2024-06-03T01:00:38.557 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-03T01:00:38.557 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764 --verbose --name teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/orig.config.yaml 2024-06-03T01:00:38.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T01:00:38.701 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7738764' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2064 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 125/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpLu1GXXHcwTHt3NpEVSwAVfyE+d28PboE8SihJxLCzmdV6hIiju9n/RuITg9652y1Oscdu+pmH/nn61XV+qRg= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMTaT72Mft9eD9H9C5iEKrrX6iJn/UNysZnIIU8soWaL1dEBgIvgEleisyNlg2ELwaj+fmPGmGvnVKYsK05Y0E= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-02_21:24:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-06-03T01:00:38.701 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa; will attempt to use it 2024-06-03T01:00:38.702 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks 2024-06-03T01:00:38.702 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-03T01:00:38.704 INFO:teuthology.task.internal:Checking packages... 2024-06-03T01:00:38.724 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-03T01:00:38.725 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-03T01:00:38.725 INFO:teuthology.packaging:ref: None 2024-06-03T01:00:38.725 INFO:teuthology.packaging:tag: None 2024-06-03T01:00:38.725 INFO:teuthology.packaging:branch: squid 2024-06-03T01:00:38.725 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:00:38.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-03T01:00:38.915 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2952.gdfcc4b3c 2024-06-03T01:00:38.916 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-03T01:00:38.922 INFO:teuthology.task.internal:no buildpackages task found 2024-06-03T01:00:38.923 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-03T01:00:38.944 INFO:teuthology.task.internal:Saving configuration 2024-06-03T01:00:38.964 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-03T01:00:38.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-03T01:00:38.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 00:55:58.481636', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpLu1GXXHcwTHt3NpEVSwAVfyE+d28PboE8SihJxLCzmdV6hIiju9n/RuITg9652y1Oscdu+pmH/nn61XV+qRg='} 2024-06-03T01:00:39.007 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 00:55:58.479894', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMTaT72Mft9eD9H9C5iEKrrX6iJn/UNysZnIIU8soWaL1dEBgIvgEleisyNlg2ELwaj+fmPGmGvnVKYsK05Y0E='} 2024-06-03T01:00:39.007 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-03T01:00:39.014 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-06-03T01:00:39.014 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-06-03T01:00:39.014 INFO:teuthology.run_tasks:Running task console_log... 2024-06-03T01:00:39.075 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8b1fb13be0>, signals=[15]) 2024-06-03T01:00:39.075 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-03T01:00:39.080 INFO:teuthology.task.internal:Opening connections... 2024-06-03T01:00:39.080 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:00:39.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:00:39.157 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-06-03T01:00:39.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:00:39.235 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-03T01:00:39.243 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-06-03T01:00:39.261 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-06-03T01:00:39.261 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-06-03T01:00:39.319 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T01:00:39.320 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T01:00:39.321 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-06-03T01:00:39.345 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-06-03T01:00:39.364 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-06-03T01:00:39.364 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-06-03T01:00:39.422 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-06-03T01:00:39.422 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-06-03T01:00:39.422 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-06-03T01:00:39.422 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-06-03T01:00:39.422 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T01:00:39.423 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T01:00:39.423 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-06-03T01:00:39.445 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-03T01:00:39.454 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-03T01:00:39.459 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-03T01:00:39.459 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-06-03T01:00:39.462 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-06-03T01:00:39.479 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-03T01:00:39.485 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-03T01:00:39.485 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-06-03T01:00:39.520 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-06-03T01:00:39.548 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-03T01:00:39.599 INFO:teuthology.run_tasks:Running task kernel... 2024-06-03T01:00:39.614 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-03T01:00:39.614 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-03T01:00:39.614 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T01:00:39.615 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T01:00:39.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:00:39.631 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-06-03T01:00:39.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:00:39.632 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-06-03T01:00:39.688 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-06-03T01:00:39.688 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-437.el9.x86_64 2024-06-03T01:00:39.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-06-03T01:00:39.690 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-437.el9.x86_64 2024-06-03T01:00:39.690 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.14.0-437.el9.x86_64 2024-06-03T01:00:39.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2024-06-03T01:00:42.180 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.1 MB 00:00 2024-06-03T01:00:42.251 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.1 MB 00:00 2024-06-03T01:00:43.865 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 37 MB/s | 19 MB 00:00 2024-06-03T01:00:43.986 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-06-03T01:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-06-03T01:00:48.632 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-06-03T01:00:49.670 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 17 kB 00:00 2024-06-03T01:00:50.274 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 17 kB 00:00 2024-06-03T01:00:51.183 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-03T01:00:51.742 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-03T01:00:56.581 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-03T01:00:57.134 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-06-03T01:00:58.695 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-03T01:00:58.695 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-03T01:00:58.793 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-03T01:00:58.800 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout:Install 4 Packages 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:00:58.801 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 80 M 2024-06-03T01:00:58.802 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 124 M 2024-06-03T01:00:58.802 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:00:59.191 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-03T01:00:59.191 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-03T01:00:59.291 INFO:teuthology.orchestra.run.smithi032.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.8 MB/s | 893 kB 00:00 2024-06-03T01:00:59.300 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:00:59.313 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:00:59.314 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-06-03T01:00:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:00:59.314 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 80 M 2024-06-03T01:00:59.314 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 124 M 2024-06-03T01:00:59.314 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:00:59.602 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 6.3 MB/s | 893 kB 00:00 2024-06-03T01:00:59.894 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-06-03T01:01:00.207 INFO:teuthology.orchestra.run.smithi032.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-06-03T01:01:00.253 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 44 MB/s | 29 MB 00:00 2024-06-03T01:01:00.595 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-06-03T01:01:00.595 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:01:00.595 INFO:teuthology.orchestra.run.smithi044.stdout:Total 63 MB/s | 80 MB 00:01 2024-06-03T01:01:00.949 INFO:teuthology.orchestra.run.smithi032.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 17 MB/s | 29 MB 00:01 2024-06-03T01:01:01.275 INFO:teuthology.orchestra.run.smithi032.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-06-03T01:01:01.275 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:01:01.275 INFO:teuthology.orchestra.run.smithi032.stdout:Total 32 MB/s | 80 MB 00:02 2024-06-03T01:01:01.323 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:01:01.747 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:01:01.747 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:01:02.001 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:01:02.427 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:01:02.427 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:01:02.717 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:01:02.718 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:01:03.393 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:01:03.393 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:01:05.288 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:01:05.943 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-03T01:01:05.946 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:01:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-03T01:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:08.982 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:09.792 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:14.083 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:14.138 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:14.885 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:19.252 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:20.064 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:50.397 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:50.439 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:50.760 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:50.760 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-03T01:01:50.760 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:50.760 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:51.331 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:51.373 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:01:51.581 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:01:51.683 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:51.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-03T01:01:51.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-03T01:01:51.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-03T01:01:51.934 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-06-03T01:01:52.023 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.023 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-03T01:01:52.024 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-03T01:01:52.024 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-03T01:01:52.024 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-03T01:01:52.024 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-06-03T01:01:52.024 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=None, version=distro) 2024-06-03T01:01:52.024 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-06-03T01:01:52.079 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.079 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-03T01:01:52.079 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-06-03T01:01:52.115 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-06-03T01:01:52.115 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-06-03T01:01:52.115 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 68.el9 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8264254 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-06-03T01:01:52.116 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:01:52.117 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-06-03T01:01:52.118 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-03T01:01:52.118 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 5.14.0-452.el9.x86_64 2024-06-03T01:01:52.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-03T01:01:52.601 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-03T01:01:52.601 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:01:52.601 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:01:52.602 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:01:52.736 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-06-03T01:01:52.977 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-06-03T01:01:53.061 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:53.061 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-03T01:01:53.061 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-03T01:01:53.062 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-03T01:01:53.062 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-03T01:01:53.062 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-06-03T01:01:53.062 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-06-03T01:01:53.062 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-06-03T01:01:53.119 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-03T01:01:53.119 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-03T01:01:53.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2024-06-03T01:01:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2024-06-03T01:01:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.06 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 68.el9 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Group : Unspecified 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 8264254 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : builder@centos.org 2024-06-03T01:01:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:01:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2024-06-03T01:01:53.154 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-03T01:01:53.154 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 5.14.0-452.el9.x86_64 2024-06-03T01:01:53.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-03T01:01:53.830 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-06-03T01:01:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-03T01:01:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-06-03T01:01:54.159 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-06-03T01:01:54.175 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.S4RV80PyPt 2024-06-03T01:01:54.175 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S4RV80PyPt 2024-06-03T01:01:54.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.S4RV80PyPt 2024-06-03T01:01:54.375 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.S4RV80PyPt is 6KB 2024-06-03T01:01:54.424 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.S4RV80PyPt 2024-06-03T01:01:54.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-06-03T01:01:54.501 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-03T01:01:54.501 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-03T01:01:54.501 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-03T01:01:54.501 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-03T01:01:54.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-03T01:01:54.640 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-06-03T01:01:55.175 INFO:teuthology.orchestra.run.smithi032.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-03T01:01:55.201 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-06-03T01:01:55.203 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-06-03T01:01:55.218 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.3cPmruPecQ 2024-06-03T01:01:55.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3cPmruPecQ 2024-06-03T01:01:55.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.3cPmruPecQ 2024-06-03T01:01:55.429 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.3cPmruPecQ is 6KB 2024-06-03T01:01:55.477 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.3cPmruPecQ 2024-06-03T01:01:55.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2024-06-03T01:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-03T01:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-03T01:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-03T01:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-03T01:01:55.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-03T01:01:55.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-06-03T01:02:24.672 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-06-03T01:02:24.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:02:24.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:02:25.705 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-06-03T01:02:25.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-06-03T01:02:25.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:02:43.231 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-06-03T01:02:52.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:02:52.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:03:25.738 DEBUG:teuthology.orchestra.remote:timed out 2024-06-03T01:03:34.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-06-03T01:03:34.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:03:35.076 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-06-03T01:03:35.433 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-06-03T01:03:35.433 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-452.el9.x86_64"... 2024-06-03T01:03:35.434 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-06-03T01:03:35.450 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-452.el9.x86_64 2024-06-03T01:03:35.451 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-03T01:03:35.451 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-03T01:03:35.451 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-03T01:03:36.452 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-06-03T01:03:36.452 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-03T01:03:36.526 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-06-03T01:03:36.552 DEBUG:teuthology.parallel:result is None 2024-06-03T01:03:52.242 DEBUG:teuthology.orchestra.remote:timed out 2024-06-03T01:04:04.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:04:04.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:04:04.598 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-06-03T01:04:04.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:04:04.975 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-452.el9.x86_64"... 2024-06-03T01:04:04.975 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-06-03T01:04:04.992 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-452.el9.x86_64 2024-06-03T01:04:04.992 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-03T01:04:04.992 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-03T01:04:04.992 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-03T01:04:05.994 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-06-03T01:04:05.994 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-03T01:04:06.064 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-06-03T01:04:06.093 DEBUG:teuthology.parallel:result is None 2024-06-03T01:04:06.093 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-03T01:04:06.101 INFO:teuthology.task.internal:Creating test directory... 2024-06-03T01:04:06.101 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T01:04:06.104 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T01:04:06.123 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-03T01:04:06.129 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-03T01:04:06.139 INFO:teuthology.task.internal:Creating archive directory... 2024-06-03T01:04:06.139 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T01:04:06.163 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T01:04:06.201 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-03T01:04:06.208 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-03T01:04:06.209 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T01:04:06.230 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T01:04:06.267 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:04:06.278 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:04:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:04:06.290 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:04:06.292 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-03T01:04:06.298 INFO:teuthology.task.internal:Configuring sudo... 2024-06-03T01:04:06.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T01:04:06.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T01:04:06.361 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-03T01:04:06.370 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-03T01:04:06.370 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T01:04:06.394 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T01:04:06.419 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:04:06.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:04:06.545 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:04:06.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:04:06.671 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:04:06.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T01:04:06.738 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:04:06.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:04:06.837 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:04:06.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:04:06.967 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:04:06.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T01:04:07.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-06-03T01:04:07.038 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-06-03T01:04:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:04:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:04:07.514 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-03T01:04:07.522 INFO:teuthology.task.internal:Starting timer... 2024-06-03T01:04:07.522 INFO:teuthology.run_tasks:Running task pcp... 2024-06-03T01:04:07.532 INFO:teuthology.run_tasks:Running task selinux... 2024-06-03T01:04:07.541 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-03T01:04:07.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-06-03T01:04:07.589 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-06-03T01:04:07.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-06-03T01:04:07.631 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-06-03T01:04:07.633 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-03T01:04:07.633 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-06-03T01:04:07.671 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-06-03T01:04:07.671 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-06-03T01:04:07.720 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-06-03T01:04:07.720 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive'} 2024-06-03T01:04:07.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:04:07.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:04:07.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:04:07.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:04:07.779 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-03T01:04:07.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-06-03T01:04:07.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-06-03T01:04:07.849 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-03T01:04:07.858 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-03T01:04:07.861 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-03T01:04:07.986 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-03T01:04:07.996 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-03T01:04:07.997 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi044.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-03T01:08:31.975 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com')] 2024-06-03T01:08:31.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-06-03T01:08:31.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:08:32.083 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-06-03T01:08:32.166 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-06-03T01:08:32.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:08:32.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:08:32.244 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-06-03T01:08:32.327 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-06-03T01:08:32.327 INFO:teuthology.run_tasks:Running task clock... 2024-06-03T01:08:32.339 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-03T01:08:32.339 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T01:08:32.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:08:32.342 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T01:08:32.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:08:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T01:08:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T01:08:32.399 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T01:08:32.411 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-06-03T01:08:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T01:08:32.422 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-06-03T01:08:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T01:08:32.441 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-06-03T01:08:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T01:08:32.453 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-06-03T01:08:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T01:08:32.483 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T01:08:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.515 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.539 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:08:32.542 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:08:32.542 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-06-03T01:08:32.543 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.543 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.543 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.543 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:08:32.544 INFO:teuthology.run_tasks:Running task install... 2024-06-03T01:08:32.553 DEBUG:teuthology.task.install:project ceph 2024-06-03T01:08:32.553 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-03T01:08:32.553 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-03T01:08:32.553 INFO:teuthology.task.install:Using flavor: default 2024-06-03T01:08:32.559 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-03T01:08:32.559 INFO:teuthology.task.install:extra packages: [] 2024-06-03T01:08:32.559 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-06-03T01:08:32.560 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:08:32.560 INFO:teuthology.packaging:ref: None 2024-06-03T01:08:32.560 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:08:32.560 INFO:teuthology.packaging:branch: None 2024-06-03T01:08:32.560 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:08:35.277 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-03T01:08:35.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-03T01:08:35.279 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-06-03T01:08:35.279 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:08:35.279 INFO:teuthology.packaging:ref: None 2024-06-03T01:08:35.280 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:08:35.280 INFO:teuthology.packaging:branch: None 2024-06-03T01:08:35.280 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:08:35.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-03T01:08:35.430 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-06-03T01:08:35.430 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-06-03T01:08:35.434 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-06-03T01:08:35.434 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-06-03T01:08:35.482 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T01:08:35.482 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:08:35.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T01:08:35.484 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T01:08:35.484 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:08:35.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T01:08:35.514 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-03T01:08:35.514 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:08:35.514 INFO:teuthology.packaging:ref: None 2024-06-03T01:08:35.514 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:08:35.514 INFO:teuthology.packaging:branch: None 2024-06-03T01:08:35.514 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:08:35.514 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T01:08:35.517 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-03T01:08:35.517 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:08:35.517 INFO:teuthology.packaging:ref: None 2024-06-03T01:08:35.517 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:08:35.517 INFO:teuthology.packaging:branch: None 2024-06-03T01:08:35.517 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:08:35.517 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T01:08:35.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T01:08:35.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T01:08:35.659 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T01:08:35.667 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T01:08:35.724 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-06-03T01:08:35.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-06-03T01:08:35.734 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-06-03T01:08:35.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-06-03T01:08:35.967 INFO:teuthology.orchestra.run.smithi044.stdout:45 files removed 2024-06-03T01:08:35.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw 2024-06-03T01:08:36.029 INFO:teuthology.orchestra.run.smithi032.stdout:45 files removed 2024-06-03T01:08:36.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw 2024-06-03T01:08:36.447 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 461 kB/s | 76 kB 00:00 2024-06-03T01:08:36.493 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 513 kB/s | 76 kB 00:00 2024-06-03T01:08:36.648 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 65 kB/s | 9.4 kB 00:00 2024-06-03T01:08:36.694 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 64 kB/s | 9.4 kB 00:00 2024-06-03T01:08:36.874 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 12 kB/s | 2.2 kB 00:00 2024-06-03T01:08:36.878 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-06-03T01:08:37.390 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-06-03T01:08:37.403 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-06-03T01:08:39.529 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-06-03T01:08:40.050 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-06-03T01:08:44.214 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-06-03T01:08:44.259 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-06-03T01:08:45.648 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 17 kB 00:00 2024-06-03T01:08:45.785 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 17 kB 00:00 2024-06-03T01:08:46.995 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-03T01:08:47.010 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-03T01:08:52.462 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-06-03T01:08:52.517 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-03T01:08:54.896 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-06-03T01:08:54.910 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T01:08:54.911 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-06-03T01:08:54.912 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:Install 28 Packages 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:08:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 53 M 2024-06-03T01:08:54.914 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:08:54.996 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-03T01:08:55.007 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T01:08:55.008 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:08:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:08:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:Install 28 Packages 2024-06-03T01:08:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 2 Packages 2024-06-03T01:08:55.010 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:08:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 53 M 2024-06-03T01:08:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:08:55.338 INFO:teuthology.orchestra.run.smithi044.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 24 MB/s | 5.2 MB 00:00 2024-06-03T01:08:55.352 INFO:teuthology.orchestra.run.smithi032.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 23 MB/s | 5.2 MB 00:00 2024-06-03T01:08:55.363 INFO:teuthology.orchestra.run.smithi044.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 962 kB/s | 24 kB 00:00 2024-06-03T01:08:55.376 INFO:teuthology.orchestra.run.smithi032.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 1.0 MB/s | 24 kB 00:00 2024-06-03T01:08:55.397 INFO:teuthology.orchestra.run.smithi044.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-06-03T01:08:55.410 INFO:teuthology.orchestra.run.smithi032.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-06-03T01:08:55.431 INFO:teuthology.orchestra.run.smithi044.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-06-03T01:08:55.444 INFO:teuthology.orchestra.run.smithi032.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-06-03T01:08:55.552 INFO:teuthology.orchestra.run.smithi032.stdout:(5/30): librgw2-18.2.1-0.el9.x86_64.rpm 41 MB/s | 4.5 MB 00:00 2024-06-03T01:08:55.556 INFO:teuthology.orchestra.run.smithi044.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 18 MB/s | 7.7 MB 00:00 2024-06-03T01:08:55.578 INFO:teuthology.orchestra.run.smithi032.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-06-03T01:08:55.581 INFO:teuthology.orchestra.run.smithi044.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-06-03T01:08:55.603 INFO:teuthology.orchestra.run.smithi032.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-06-03T01:08:55.615 INFO:teuthology.orchestra.run.smithi044.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 3.7 MB/s | 124 kB 00:00 2024-06-03T01:08:55.628 INFO:teuthology.orchestra.run.smithi032.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-06-03T01:08:55.640 INFO:teuthology.orchestra.run.smithi044.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-06-03T01:08:55.699 INFO:teuthology.orchestra.run.smithi044.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 17 MB/s | 4.5 MB 00:00 2024-06-03T01:08:55.724 INFO:teuthology.orchestra.run.smithi044.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.7 MB/s | 321 kB 00:00 2024-06-03T01:08:55.728 INFO:teuthology.orchestra.run.smithi032.stdout:(9/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 13 MB/s | 7.7 MB 00:00 2024-06-03T01:08:55.749 INFO:teuthology.orchestra.run.smithi044.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-06-03T01:08:55.775 INFO:teuthology.orchestra.run.smithi044.stdout:(12/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 3.8 MB/s | 297 kB 00:00 2024-06-03T01:08:55.796 INFO:teuthology.orchestra.run.smithi032.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 321 kB 00:00 2024-06-03T01:08:55.835 INFO:teuthology.orchestra.run.smithi032.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.7 MB/s | 297 kB 00:00 2024-06-03T01:08:55.905 INFO:teuthology.orchestra.run.smithi032.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 912 kB/s | 99 kB 00:00 2024-06-03T01:08:55.983 INFO:teuthology.orchestra.run.smithi044.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-06-03T01:08:56.050 INFO:teuthology.orchestra.run.smithi044.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 356 kB/s | 107 kB 00:00 2024-06-03T01:08:56.075 INFO:teuthology.orchestra.run.smithi044.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 533 kB/s | 160 kB 00:00 2024-06-03T01:08:56.092 INFO:teuthology.orchestra.run.smithi044.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-06-03T01:08:56.122 INFO:teuthology.orchestra.run.smithi032.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 18 MB/s | 18 MB 00:00 2024-06-03T01:08:56.143 INFO:teuthology.orchestra.run.smithi044.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-06-03T01:08:56.147 INFO:teuthology.orchestra.run.smithi032.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 345 kB/s | 107 kB 00:00 2024-06-03T01:08:56.172 INFO:teuthology.orchestra.run.smithi032.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 600 kB/s | 160 kB 00:00 2024-06-03T01:08:56.176 INFO:teuthology.orchestra.run.smithi044.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-06-03T01:08:56.201 INFO:teuthology.orchestra.run.smithi044.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 388 kB/s | 42 kB 00:00 2024-06-03T01:08:56.231 INFO:teuthology.orchestra.run.smithi032.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-06-03T01:08:56.256 INFO:teuthology.orchestra.run.smithi032.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-06-03T01:08:56.282 INFO:teuthology.orchestra.run.smithi032.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 838 kB/s | 42 kB 00:00 2024-06-03T01:08:56.307 INFO:teuthology.orchestra.run.smithi032.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-06-03T01:08:56.474 INFO:teuthology.orchestra.run.smithi032.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 510 kB/s | 111 kB 00:00 2024-06-03T01:08:56.510 INFO:teuthology.orchestra.run.smithi044.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 301 kB/s | 111 kB 00:00 2024-06-03T01:08:56.532 INFO:teuthology.orchestra.run.smithi032.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-06-03T01:08:56.558 INFO:teuthology.orchestra.run.smithi032.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-06-03T01:08:56.592 INFO:teuthology.orchestra.run.smithi032.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-06-03T01:08:56.611 INFO:teuthology.orchestra.run.smithi044.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 712 kB/s | 309 kB 00:00 2024-06-03T01:08:56.617 INFO:teuthology.orchestra.run.smithi032.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-06-03T01:08:56.636 INFO:teuthology.orchestra.run.smithi044.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-06-03T01:08:56.711 INFO:teuthology.orchestra.run.smithi044.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-06-03T01:08:56.717 INFO:teuthology.orchestra.run.smithi032.stdout:(25/30): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-06-03T01:08:56.742 INFO:teuthology.orchestra.run.smithi032.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-06-03T01:08:56.745 INFO:teuthology.orchestra.run.smithi044.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-06-03T01:08:56.801 INFO:teuthology.orchestra.run.smithi032.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-06-03T01:08:56.835 INFO:teuthology.orchestra.run.smithi032.stdout:(28/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-06-03T01:08:56.853 INFO:teuthology.orchestra.run.smithi044.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-06-03T01:08:56.893 INFO:teuthology.orchestra.run.smithi032.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 22 MB/s | 3.3 MB 00:00 2024-06-03T01:08:56.920 INFO:teuthology.orchestra.run.smithi044.stdout:(26/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-06-03T01:08:56.952 INFO:teuthology.orchestra.run.smithi032.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 20 MB/s | 3.0 MB 00:00 2024-06-03T01:08:56.954 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:08:56.954 INFO:teuthology.orchestra.run.smithi032.stdout:Total 27 MB/s | 53 MB 00:01 2024-06-03T01:08:56.996 INFO:teuthology.orchestra.run.smithi044.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-06-03T01:08:57.021 INFO:teuthology.orchestra.run.smithi044.stdout:(28/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-06-03T01:08:57.027 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:08:57.067 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:08:57.067 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:08:57.088 INFO:teuthology.orchestra.run.smithi044.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2024-06-03T01:08:57.180 INFO:teuthology.orchestra.run.smithi044.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-06-03T01:08:57.181 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:08:57.181 INFO:teuthology.orchestra.run.smithi044.stdout:Total 23 MB/s | 53 MB 00:02 2024-06-03T01:08:57.258 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:08:57.299 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:08:57.299 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:08:57.606 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:08:57.606 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:08:57.838 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:08:57.839 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:08:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:08:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-06-03T01:08:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-06-03T01:08:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-06-03T01:08:59.184 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:08:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-06-03T01:08:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-06-03T01:08:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-06-03T01:08:59.484 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-06-03T01:08:59.573 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:08:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:08:59.749 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-06-03T01:08:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-06-03T01:08:59.783 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:08:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:08:59.846 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:08:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-06-03T01:08:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-06-03T01:08:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:08:59.987 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-03T01:09:00.009 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-06-03T01:09:00.018 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:00.035 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:09:00.085 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:00.098 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-06-03T01:09:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-06-03T01:09:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-06-03T01:09:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-06-03T01:09:00.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-03T01:09:00.263 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-06-03T01:09:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-06-03T01:09:00.312 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-06-03T01:09:00.380 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-06-03T01:09:00.412 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-06-03T01:09:00.422 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-06-03T01:09:00.454 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-06-03T01:09:00.495 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-06-03T01:09:00.536 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-06-03T01:09:00.554 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-06-03T01:09:00.590 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-06-03T01:09:00.623 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-06-03T01:09:00.729 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-06-03T01:09:00.775 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-03T01:09:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-03T01:09:00.898 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-06-03T01:09:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-06-03T01:09:00.941 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-06-03T01:09:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-03T01:09:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-03T01:09:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-06-03T01:09:01.144 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-06-03T01:09:01.380 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-06-03T01:09:01.411 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-03T01:09:01.483 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-03T01:09:01.563 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-06-03T01:09:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-06-03T01:09:01.589 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-06-03T01:09:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-03T01:09:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-03T01:09:01.731 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-06-03T01:09:01.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-06-03T01:09:03.736 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:03.786 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:03.916 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:03.946 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:04.399 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-03T01:09:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-03T01:09:04.499 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T01:09:04.499 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T01:09:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:04.533 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-03T01:09:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-03T01:09:04.679 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-03T01:09:04.680 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T01:09:04.680 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T01:09:04.680 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:04.710 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-06-03T01:09:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-06-03T01:09:14.895 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-06-03T01:09:14.896 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-06-03T01:09:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:15.411 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-03T01:09:15.438 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-03T01:09:15.438 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:15.438 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T01:09:15.439 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T01:09:15.439 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T01:09:15.439 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:15.441 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-03T01:09:15.530 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-03T01:09:15.531 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:15.687 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T01:09:15.714 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:15.716 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-03T01:09:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-03T01:09:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-06-03T01:09:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-06-03T01:09:16.708 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-06-03T01:09:16.709 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-06-03T01:09:16.709 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-06-03T01:09:16.709 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:16.709 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-06-03T01:09:16.709 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-06-03T01:09:16.989 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:16.989 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-06-03T01:09:16.990 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-06-03T01:09:16.991 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-06-03T01:09:16.992 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-06-03T01:09:16.992 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-06-03T01:09:16.992 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-06-03T01:09:17.471 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: fmt-8.1.1-5.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T01:09:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:17.474 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:17.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-test 2024-06-03T01:09:17.832 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-06-03T01:09:17.832 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: fmt-8.1.1-5.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T01:09:17.833 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:17.835 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:18.174 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-test 2024-06-03T01:09:18.427 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:26 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:18.554 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-06-03T01:09:18.555 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout:Install 4 Packages 2024-06-03T01:09:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:18.557 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 40 M 2024-06-03T01:09:18.557 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 166 M 2024-06-03T01:09:18.557 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:18.764 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:26 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:18.891 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:18.891 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:18.891 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-06-03T01:09:18.892 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 40 M 2024-06-03T01:09:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 166 M 2024-06-03T01:09:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:18.904 INFO:teuthology.orchestra.run.smithi032.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 853 kB/s | 165 kB 00:00 2024-06-03T01:09:18.947 INFO:teuthology.orchestra.run.smithi032.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-06-03T01:09:19.029 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.6 MB/s | 165 kB 00:00 2024-06-03T01:09:19.096 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-06-03T01:09:19.121 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 695 kB/s | 64 kB 00:00 2024-06-03T01:09:19.347 INFO:teuthology.orchestra.run.smithi032.stdout:(3/4): ceph-test-18.2.1-0.el9.x86_64.rpm 62 MB/s | 40 MB 00:00 2024-06-03T01:09:19.388 INFO:teuthology.orchestra.run.smithi032.stdout:(4/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 132 kB/s | 64 kB 00:00 2024-06-03T01:09:19.390 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:19.391 INFO:teuthology.orchestra.run.smithi032.stdout:Total 48 MB/s | 40 MB 00:00 2024-06-03T01:09:19.418 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:19.440 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:19.440 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:19.579 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 61 MB/s | 40 MB 00:00 2024-06-03T01:09:19.581 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:19.582 INFO:teuthology.orchestra.run.smithi044.stdout:Total 58 MB/s | 40 MB 00:00 2024-06-03T01:09:19.609 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:19.777 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:19.777 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:19.966 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:19.966 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:20.268 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:20.354 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-03T01:09:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-06-03T01:09:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-03T01:09:20.454 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:20.546 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-03T01:09:20.587 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-06-03T01:09:20.615 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-03T01:09:24.379 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-03T01:09:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-03T01:09:24.569 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-03T01:09:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-03T01:09:26.617 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-03T01:09:26.617 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-06-03T01:09:26.617 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-06-03T01:09:26.617 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-06-03T01:09:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-03T01:09:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-06-03T01:09:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-06-03T01:09:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-06-03T01:09:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-03T01:09:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:26.937 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T01:09:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:26.938 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T01:09:27.135 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:27.136 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:27.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph 2024-06-03T01:09:27.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph 2024-06-03T01:09:27.825 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:35 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:27.955 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:09:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T01:09:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T01:09:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout:Install 33 Packages 2024-06-03T01:09:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:27.962 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 30 M 2024-06-03T01:09:27.962 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 110 M 2024-06-03T01:09:27.962 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:28.001 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:36 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:28.130 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:28.132 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:28.132 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-06-03T01:09:28.133 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T01:09:28.134 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T01:09:28.135 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout:Install 33 Packages 2024-06-03T01:09:28.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:28.137 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 30 M 2024-06-03T01:09:28.137 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 110 M 2024-06-03T01:09:28.137 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:28.583 INFO:teuthology.orchestra.run.smithi032.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 62 kB/s | 6.4 kB 00:00 2024-06-03T01:09:28.633 INFO:teuthology.orchestra.run.smithi032.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 1.4 MB 00:00 2024-06-03T01:09:28.639 INFO:teuthology.orchestra.run.smithi044.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 55 kB/s | 6.4 kB 00:00 2024-06-03T01:09:28.683 INFO:teuthology.orchestra.run.smithi032.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-06-03T01:09:28.690 INFO:teuthology.orchestra.run.smithi044.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 8.7 MB/s | 1.4 MB 00:00 2024-06-03T01:09:28.709 INFO:teuthology.orchestra.run.smithi032.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-06-03T01:09:28.741 INFO:teuthology.orchestra.run.smithi044.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.7 MB/s | 2.1 MB 00:00 2024-06-03T01:09:28.765 INFO:teuthology.orchestra.run.smithi044.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-06-03T01:09:28.792 INFO:teuthology.orchestra.run.smithi032.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-06-03T01:09:28.841 INFO:teuthology.orchestra.run.smithi044.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-06-03T01:09:28.916 INFO:teuthology.orchestra.run.smithi044.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-06-03T01:09:29.026 INFO:teuthology.orchestra.run.smithi032.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 45 MB/s | 18 MB 00:00 2024-06-03T01:09:29.116 INFO:teuthology.orchestra.run.smithi044.stdout:(7/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 41 MB/s | 18 MB 00:00 2024-06-03T01:09:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 473 kB/s | 106 kB 00:00 2024-06-03T01:09:29.192 INFO:teuthology.orchestra.run.smithi044.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 3.5 MB/s | 1.2 MB 00:00 2024-06-03T01:09:29.217 INFO:teuthology.orchestra.run.smithi044.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-06-03T01:09:29.242 INFO:teuthology.orchestra.run.smithi044.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 1.1 MB/s | 135 kB 00:00 2024-06-03T01:09:29.251 INFO:teuthology.orchestra.run.smithi032.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 467 kB/s | 253 kB 00:00 2024-06-03T01:09:29.267 INFO:teuthology.orchestra.run.smithi044.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-06-03T01:09:29.318 INFO:teuthology.orchestra.run.smithi032.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 2.3 MB/s | 1.2 MB 00:00 2024-06-03T01:09:29.334 INFO:teuthology.orchestra.run.smithi044.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 377 kB/s | 35 kB 00:00 2024-06-03T01:09:29.343 INFO:teuthology.orchestra.run.smithi032.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 336 kB/s | 106 kB 00:00 2024-06-03T01:09:29.368 INFO:teuthology.orchestra.run.smithi044.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-06-03T01:09:29.369 INFO:teuthology.orchestra.run.smithi032.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 1.1 MB/s | 135 kB 00:00 2024-06-03T01:09:29.393 INFO:teuthology.orchestra.run.smithi044.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-06-03T01:09:29.394 INFO:teuthology.orchestra.run.smithi032.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-06-03T01:09:29.418 INFO:teuthology.orchestra.run.smithi044.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 855 kB/s | 172 kB 00:00 2024-06-03T01:09:29.453 INFO:teuthology.orchestra.run.smithi032.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-06-03T01:09:29.478 INFO:teuthology.orchestra.run.smithi032.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-06-03T01:09:29.504 INFO:teuthology.orchestra.run.smithi032.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-06-03T01:09:29.535 INFO:teuthology.orchestra.run.smithi044.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 260 kB/s | 43 kB 00:00 2024-06-03T01:09:29.537 INFO:teuthology.orchestra.run.smithi032.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-06-03T01:09:29.562 INFO:teuthology.orchestra.run.smithi032.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-06-03T01:09:29.587 INFO:teuthology.orchestra.run.smithi032.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-06-03T01:09:29.594 INFO:teuthology.orchestra.run.smithi044.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 857 kB/s | 172 kB 00:00 2024-06-03T01:09:29.621 INFO:teuthology.orchestra.run.smithi032.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 323 kB/s | 11 kB 00:00 2024-06-03T01:09:29.621 INFO:teuthology.orchestra.run.smithi044.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 125 kB/s | 11 kB 00:00 2024-06-03T01:09:29.663 INFO:teuthology.orchestra.run.smithi032.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-06-03T01:09:29.672 INFO:teuthology.orchestra.run.smithi044.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 228 kB/s | 18 kB 00:00 2024-06-03T01:09:29.688 INFO:teuthology.orchestra.run.smithi032.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-06-03T01:09:29.697 INFO:teuthology.orchestra.run.smithi044.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-06-03T01:09:29.714 INFO:teuthology.orchestra.run.smithi032.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-06-03T01:09:29.722 INFO:teuthology.orchestra.run.smithi044.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-06-03T01:09:29.739 INFO:teuthology.orchestra.run.smithi032.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 384 kB/s | 19 kB 00:00 2024-06-03T01:09:29.764 INFO:teuthology.orchestra.run.smithi032.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-06-03T01:09:29.790 INFO:teuthology.orchestra.run.smithi032.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-06-03T01:09:29.815 INFO:teuthology.orchestra.run.smithi032.stdout:(25/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-06-03T01:09:29.831 INFO:teuthology.orchestra.run.smithi044.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 182 kB/s | 20 kB 00:00 2024-06-03T01:09:29.840 INFO:teuthology.orchestra.run.smithi032.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-06-03T01:09:29.856 INFO:teuthology.orchestra.run.smithi044.stdout:(24/33): python3-jaraco-functools-3.5.0-2.el9.n 106 kB/s | 19 kB 00:00 2024-06-03T01:09:29.866 INFO:teuthology.orchestra.run.smithi032.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-06-03T01:09:29.891 INFO:teuthology.orchestra.run.smithi032.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-06-03T01:09:29.917 INFO:teuthology.orchestra.run.smithi032.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-06-03T01:09:29.942 INFO:teuthology.orchestra.run.smithi032.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-06-03T01:09:29.967 INFO:teuthology.orchestra.run.smithi032.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-03T01:09:29.983 INFO:teuthology.orchestra.run.smithi032.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-06-03T01:09:30.017 INFO:teuthology.orchestra.run.smithi032.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-06-03T01:09:30.019 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:30.019 INFO:teuthology.orchestra.run.smithi032.stdout:Total 15 MB/s | 30 MB 00:02 2024-06-03T01:09:30.098 INFO:teuthology.orchestra.run.smithi044.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 295 kB/s | 79 kB 00:00 2024-06-03T01:09:30.123 INFO:teuthology.orchestra.run.smithi044.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-06-03T01:09:30.146 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:30.187 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:30.187 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:30.315 INFO:teuthology.orchestra.run.smithi044.stdout:(27/33): python3-logutils-0.3.5-21.el9.noarch.r 78 kB/s | 46 kB 00:00 2024-06-03T01:09:30.340 INFO:teuthology.orchestra.run.smithi044.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 68 kB/s | 16 kB 00:00 2024-06-03T01:09:30.366 INFO:teuthology.orchestra.run.smithi044.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 371 kB/s | 90 kB 00:00 2024-06-03T01:09:30.533 INFO:teuthology.orchestra.run.smithi044.stdout:(30/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-06-03T01:09:30.558 INFO:teuthology.orchestra.run.smithi044.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-06-03T01:09:30.580 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:30.581 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:30.611 INFO:teuthology.orchestra.run.smithi044.stdout:(32/33): python3-tempora-5.0.0-2.el9.noarch.rpm 121 kB/s | 36 kB 00:00 2024-06-03T01:09:30.733 INFO:teuthology.orchestra.run.smithi044.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 100 kB/s | 20 kB 00:00 2024-06-03T01:09:30.735 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:30.735 INFO:teuthology.orchestra.run.smithi044.stdout:Total 12 MB/s | 30 MB 00:02 2024-06-03T01:09:30.862 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:30.902 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:30.902 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:31.321 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:31.321 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:31.784 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:31.846 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-03T01:09:31.878 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-03T01:09:31.911 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-03T01:09:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-03T01:09:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-03T01:09:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-03T01:09:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-03T01:09:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-03T01:09:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-03T01:09:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-03T01:09:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-03T01:09:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-03T01:09:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-03T01:09:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-03T01:09:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-03T01:09:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-03T01:09:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-03T01:09:32.644 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-03T01:09:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-03T01:09:32.741 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-03T01:09:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-03T01:09:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-03T01:09:32.826 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-03T01:09:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-03T01:09:32.872 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-03T01:09:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-03T01:09:32.915 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-03T01:09:32.945 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-03T01:09:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-03T01:09:32.979 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-03T01:09:33.012 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-03T01:09:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-03T01:09:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-03T01:09:33.171 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-03T01:09:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-03T01:09:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-03T01:09:33.255 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-03T01:09:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-03T01:09:33.407 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-03T01:09:33.408 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-03T01:09:33.446 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-03T01:09:33.491 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-03T01:09:33.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-03T01:09:33.573 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-03T01:09:33.587 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-03T01:09:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-03T01:09:33.710 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-03T01:09:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-06-03T01:09:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-03T01:09:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-03T01:09:33.787 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:33.787 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T01:09:33.787 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T01:09:33.787 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T01:09:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:33.917 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-03T01:09:33.975 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-03T01:09:34.057 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-03T01:09:34.154 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-03T01:09:34.197 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-03T01:09:34.347 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-03T01:09:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-06-03T01:09:34.541 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-03T01:09:34.568 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-03T01:09:34.568 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:34.568 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T01:09:34.568 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T01:09:34.569 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T01:09:34.569 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:35.385 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-03T01:09:35.416 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-03T01:09:35.416 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:35.417 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T01:09:35.417 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T01:09:35.417 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T01:09:35.417 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:35.894 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-03T01:09:35.920 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-03T01:09:35.920 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:35.920 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T01:09:35.921 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T01:09:35.921 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T01:09:35.921 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:36.145 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-03T01:09:36.170 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-03T01:09:36.170 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:36.170 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T01:09:36.171 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T01:09:36.171 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T01:09:36.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T01:09:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-03T01:09:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T01:09:36.711 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T01:09:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:37.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-03T01:09:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-03T01:09:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-03T01:09:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-03T01:09:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-03T01:09:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-03T01:09:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-03T01:09:38.797 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T01:09:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T01:09:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:38.800 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-03T01:09:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-06-03T01:09:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-03T01:09:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-03T01:09:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-03T01:09:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-03T01:09:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-03T01:09:39.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-base 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T01:09:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T01:09:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T01:09:39.710 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:39.710 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:39.754 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:47 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:39.837 INFO:teuthology.orchestra.run.smithi032.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:09:39.881 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:39.882 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:09:39.882 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:39.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install cephadm 2024-06-03T01:09:40.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-base 2024-06-03T01:09:40.545 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:48 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:40.641 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:48 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:40.672 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:40.672 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 221 k 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 213 k 2024-06-03T01:09:40.673 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:40.725 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:09:40.753 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.7 MB/s | 221 kB 00:00 2024-06-03T01:09:40.753 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:40.754 INFO:teuthology.orchestra.run.smithi032.stdout:Total 2.7 MB/s | 221 kB 00:00 2024-06-03T01:09:40.754 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:40.760 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:40.760 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:40.762 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:40.763 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:40.770 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:40.771 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:09:40.771 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:40.834 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:40.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install cephadm 2024-06-03T01:09:41.309 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:41.435 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:49 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:41.563 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:41.563 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 221 k 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 213 k 2024-06-03T01:09:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:41.638 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.0 MB/s | 221 kB 00:00 2024-06-03T01:09:41.638 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:41.638 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-06-03T01:09:41.638 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:41.643 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:41.643 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:41.646 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:41.646 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:41.713 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:41.782 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.059 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.060 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:42.060 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:42.060 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-06-03T01:09:42.060 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:42.060 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-immutable-object-cache 2024-06-03T01:09:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.625 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.880 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:50 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:42.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-03T01:09:42.883 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:42.883 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:42.883 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-06-03T01:09:42.883 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:42.884 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:43.010 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:43.010 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repo Size 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 142 k 2024-06-03T01:09:43.011 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 395 k 2024-06-03T01:09:43.012 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:43.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-immutable-object-cache 2024-06-03T01:09:43.084 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-06-03T01:09:43.084 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:43.085 INFO:teuthology.orchestra.run.smithi032.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-06-03T01:09:43.085 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:43.093 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:43.093 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:43.153 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:43.153 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:43.395 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:43.395 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T01:09:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:43.671 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:51 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:43.799 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:43.799 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-06-03T01:09:43.799 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:43.800 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 142 k 2024-06-03T01:09:43.801 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 395 k 2024-06-03T01:09:43.801 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:43.869 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.1 MB/s | 142 kB 00:00 2024-06-03T01:09:43.869 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:43.870 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.0 MB/s | 142 kB 00:00 2024-06-03T01:09:43.870 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:43.878 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:43.878 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:43.937 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:43.938 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:44.082 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:44.170 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:44.196 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:44.196 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:09:44.196 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T01:09:44.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:44.335 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:44.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr 2024-06-03T01:09:45.135 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:09:45.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:45.136 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:45.136 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-06-03T01:09:45.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:45.136 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:45.176 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:53 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:45.261 INFO:teuthology.orchestra.run.smithi032.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:09:45.305 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:45.306 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:09:45.306 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:45.373 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr 2024-06-03T01:09:45.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-dashboard 2024-06-03T01:09:45.968 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:53 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:45.976 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:53 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:46.053 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:09:46.098 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:46.099 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:09:46.099 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:46.107 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-06-03T01:09:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout:Install 7 Packages 2024-06-03T01:09:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:46.110 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 2.1 M 2024-06-03T01:09:46.110 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 8.5 M 2024-06-03T01:09:46.110 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:46.171 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-dashboard 2024-06-03T01:09:46.421 INFO:teuthology.orchestra.run.smithi032.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 196 kB/s | 23 kB 00:00 2024-06-03T01:09:46.446 INFO:teuthology.orchestra.run.smithi032.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 102 kB/s | 15 kB 00:00 2024-06-03T01:09:46.496 INFO:teuthology.orchestra.run.smithi032.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.1 MB/s | 1.8 MB 00:00 2024-06-03T01:09:46.530 INFO:teuthology.orchestra.run.smithi032.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-06-03T01:09:46.555 INFO:teuthology.orchestra.run.smithi032.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 376 kB/s | 41 kB 00:00 2024-06-03T01:09:46.580 INFO:teuthology.orchestra.run.smithi032.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 368 kB/s | 31 kB 00:00 2024-06-03T01:09:46.605 INFO:teuthology.orchestra.run.smithi032.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-06-03T01:09:46.607 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:46.608 INFO:teuthology.orchestra.run.smithi032.stdout:Total 4.1 MB/s | 2.1 MB 00:00 2024-06-03T01:09:46.608 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:46.617 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:46.617 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:46.674 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:46.674 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:46.766 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:54 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:46.895 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-06-03T01:09:46.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:Install 7 Packages 2024-06-03T01:09:46.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.1 M 2024-06-03T01:09:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 8.5 M 2024-06-03T01:09:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:47.028 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:47.090 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-03T01:09:47.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-03T01:09:47.172 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-03T01:09:47.203 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-03T01:09:47.232 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-06-03T01:09:47.357 INFO:teuthology.orchestra.run.smithi044.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 207 kB/s | 23 kB 00:00 2024-06-03T01:09:47.383 INFO:teuthology.orchestra.run.smithi044.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 106 kB/s | 15 kB 00:00 2024-06-03T01:09:47.433 INFO:teuthology.orchestra.run.smithi044.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.4 MB/s | 1.8 MB 00:00 2024-06-03T01:09:47.481 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-06-03T01:09:47.509 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-03T01:09:47.592 INFO:teuthology.orchestra.run.smithi044.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 39 kB/s | 9.0 kB 00:00 2024-06-03T01:09:47.667 INFO:teuthology.orchestra.run.smithi044.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 144 kB/s | 41 kB 00:00 2024-06-03T01:09:47.717 INFO:teuthology.orchestra.run.smithi044.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 132 kB/s | 31 kB 00:00 2024-06-03T01:09:47.809 INFO:teuthology.orchestra.run.smithi044.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 869 kB/s | 188 kB 00:00 2024-06-03T01:09:47.812 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:47.813 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.2 MB/s | 2.1 MB 00:00 2024-06-03T01:09:47.813 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:47.822 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:47.822 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:47.881 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:47.881 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-03T01:09:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-03T01:09:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:48.310 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-03T01:09:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-03T01:09:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:48.318 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-06-03T01:09:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:48.319 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:48.345 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-03T01:09:48.393 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-03T01:09:48.423 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-03T01:09:48.451 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-06-03T01:09:48.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-03T01:09:48.691 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-06-03T01:09:48.721 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-03T01:09:49.131 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:57 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:49.220 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-03T01:09:49.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-03T01:09:49.260 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T01:09:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout:Install 12 Packages 2024-06-03T01:09:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:49.263 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 38 M 2024-06-03T01:09:49.263 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 209 M 2024-06-03T01:09:49.263 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T01:09:49.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:49.518 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:09:49.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-03T01:09:49.737 INFO:teuthology.orchestra.run.smithi032.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 46 MB/s | 7.4 MB 00:00 2024-06-03T01:09:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-06-03T01:09:49.913 INFO:teuthology.orchestra.run.smithi032.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 579 kB/s | 195 kB 00:00 2024-06-03T01:09:49.971 INFO:teuthology.orchestra.run.smithi032.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-06-03T01:09:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:(5/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-06-03T01:09:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 838 kB/s | 35 kB 00:00 2024-06-03T01:09:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:(7/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 155 kB/s | 18 kB 00:00 2024-06-03T01:09:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-06-03T01:09:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.9 MB/s | 250 kB 00:00 2024-06-03T01:09:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-06-03T01:09:50.282 INFO:teuthology.orchestra.run.smithi032.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 17 MB/s | 4.7 MB 00:00 2024-06-03T01:09:50.314 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:58 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:09:50.443 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:09:50.444 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T01:09:50.445 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout:Install 12 Packages 2024-06-03T01:09:50.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:09:50.447 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 38 M 2024-06-03T01:09:50.447 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 209 M 2024-06-03T01:09:50.447 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:09:50.498 INFO:teuthology.orchestra.run.smithi032.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-06-03T01:09:50.499 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:50.499 INFO:teuthology.orchestra.run.smithi032.stdout:Total 31 MB/s | 38 MB 00:01 2024-06-03T01:09:50.864 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:09:50.874 INFO:teuthology.orchestra.run.smithi044.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 44 MB/s | 7.4 MB 00:00 2024-06-03T01:09:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:09:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:09:50.957 INFO:teuthology.orchestra.run.smithi044.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-06-03T01:09:50.983 INFO:teuthology.orchestra.run.smithi044.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 701 kB/s | 195 kB 00:00 2024-06-03T01:09:51.016 INFO:teuthology.orchestra.run.smithi044.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-06-03T01:09:51.042 INFO:teuthology.orchestra.run.smithi044.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 311 kB/s | 18 kB 00:00 2024-06-03T01:09:51.067 INFO:teuthology.orchestra.run.smithi044.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-06-03T01:09:51.109 INFO:teuthology.orchestra.run.smithi044.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-06-03T01:09:51.167 INFO:teuthology.orchestra.run.smithi044.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-06-03T01:09:51.285 INFO:teuthology.orchestra.run.smithi044.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-06-03T01:09:51.422 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:09:51.423 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:09:51.461 INFO:teuthology.orchestra.run.smithi044.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 11 MB/s | 4.7 MB 00:00 2024-06-03T01:09:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-06-03T01:09:51.777 INFO:teuthology.orchestra.run.smithi044.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-06-03T01:09:51.778 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:09:51.778 INFO:teuthology.orchestra.run.smithi044.stdout:Total 29 MB/s | 38 MB 00:01 2024-06-03T01:09:52.149 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:09:52.178 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:09:52.178 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:09:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:09:52.282 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-03T01:09:52.322 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-03T01:09:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-03T01:09:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-03T01:09:52.651 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-03T01:09:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-03T01:09:52.701 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:09:52.701 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:09:52.788 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-03T01:09:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:09:53.501 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-03T01:09:53.565 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-03T01:09:53.609 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-03T01:09:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-03T01:09:53.719 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-03T01:09:53.752 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-03T01:09:53.936 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-03T01:09:53.967 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-03T01:09:54.058 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-03T01:09:54.763 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-03T01:09:54.887 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-03T01:09:54.976 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-03T01:09:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-03T01:09:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-03T01:09:56.258 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-03T01:09:57.367 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-03T01:09:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-03T01:09:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-03T01:09:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-03T01:09:59.018 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-03T01:09:59.018 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:59.018 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:09:59.018 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-06-03T01:09:59.018 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:09:59.019 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:09:59.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-rook 2024-06-03T01:09:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-03T01:09:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-03T01:09:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-03T01:09:59.972 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:07 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:00.101 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:10:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T01:10:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout:Install 13 Packages 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:00.103 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 2.2 M 2024-06-03T01:10:00.104 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 26 M 2024-06-03T01:10:00.104 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:00.280 INFO:teuthology.orchestra.run.smithi032.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 755 kB/s | 50 kB 00:00 2024-06-03T01:10:00.305 INFO:teuthology.orchestra.run.smithi032.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 280 kB/s | 26 kB 00:00 2024-06-03T01:10:00.330 INFO:teuthology.orchestra.run.smithi032.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 164 kB/s | 19 kB 00:00 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T01:10:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:00.387 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:00.439 INFO:teuthology.orchestra.run.smithi032.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-06-03T01:10:00.464 INFO:teuthology.orchestra.run.smithi032.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-06-03T01:10:00.489 INFO:teuthology.orchestra.run.smithi032.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-06-03T01:10:00.515 INFO:teuthology.orchestra.run.smithi032.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 710 kB/s | 54 kB 00:00 2024-06-03T01:10:00.606 INFO:teuthology.orchestra.run.smithi032.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-06-03T01:10:00.624 INFO:teuthology.orchestra.run.smithi032.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2024-06-03T01:10:00.732 INFO:teuthology.orchestra.run.smithi032.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-06-03T01:10:00.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-rook 2024-06-03T01:10:00.758 INFO:teuthology.orchestra.run.smithi032.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 846 kB/s | 206 kB 00:00 2024-06-03T01:10:00.816 INFO:teuthology.orchestra.run.smithi032.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-06-03T01:10:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-06-03T01:10:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-06-03T01:10:00.942 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:00.958 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:00.958 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:01.050 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:01.050 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:01.358 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:09 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:01.486 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T01:10:01.487 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout:Install 13 Packages 2024-06-03T01:10:01.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:01.489 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.2 M 2024-06-03T01:10:01.489 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 26 M 2024-06-03T01:10:01.489 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:01.569 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:01.748 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-03T01:10:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-03T01:10:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-03T01:10:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-03T01:10:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-03T01:10:02.057 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-03T01:10:02.177 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-06-03T01:10:02.202 INFO:teuthology.orchestra.run.smithi044.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 754 kB/s | 50 kB 00:00 2024-06-03T01:10:02.227 INFO:teuthology.orchestra.run.smithi044.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 280 kB/s | 26 kB 00:00 2024-06-03T01:10:02.228 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-03T01:10:02.244 INFO:teuthology.orchestra.run.smithi044.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 177 kB/s | 19 kB 00:00 2024-06-03T01:10:02.311 INFO:teuthology.orchestra.run.smithi044.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-06-03T01:10:02.345 INFO:teuthology.orchestra.run.smithi044.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-06-03T01:10:02.370 INFO:teuthology.orchestra.run.smithi044.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-06-03T01:10:02.400 INFO:teuthology.orchestra.run.smithi044.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 609 kB/s | 54 kB 00:00 2024-06-03T01:10:02.742 INFO:teuthology.orchestra.run.smithi044.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 40 kB/s | 15 kB 00:00 2024-06-03T01:10:02.767 INFO:teuthology.orchestra.run.smithi044.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 76 kB/s | 32 kB 00:00 2024-06-03T01:10:02.959 INFO:teuthology.orchestra.run.smithi044.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2024-06-03T01:10:02.985 INFO:teuthology.orchestra.run.smithi044.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 352 kB/s | 206 kB 00:00 2024-06-03T01:10:02.985 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-03T01:10:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-03T01:10:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-03T01:10:03.085 INFO:teuthology.orchestra.run.smithi044.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 718 kB/s | 90 kB 00:00 2024-06-03T01:10:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-03T01:10:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-03T01:10:03.195 INFO:teuthology.orchestra.run.smithi044.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.3 MB/s | 1.0 MB 00:00 2024-06-03T01:10:03.199 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:03.199 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-06-03T01:10:03.273 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:03.289 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:03.289 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:03.385 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:03.386 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-03T01:10:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-03T01:10:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-03T01:10:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-03T01:10:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-06-03T01:10:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-03T01:10:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-03T01:10:04.050 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:04.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-03T01:10:04.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:04.224 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:04.224 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-06-03T01:10:04.224 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:04.225 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-03T01:10:04.291 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-03T01:10:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-03T01:10:04.382 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-03T01:10:04.425 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-03T01:10:04.523 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-03T01:10:04.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-cephadm 2024-06-03T01:10:04.641 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-06-03T01:10:04.701 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-03T01:10:05.136 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:13 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:05.265 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T01:10:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout:Install 6 Packages 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:05.267 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 7.0 M 2024-06-03T01:10:05.268 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 33 M 2024-06-03T01:10:05.268 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-03T01:10:05.518 INFO:teuthology.orchestra.run.smithi032.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-06-03T01:10:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-03T01:10:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-03T01:10:05.604 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-03T01:10:05.618 INFO:teuthology.orchestra.run.smithi032.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-06-03T01:10:05.633 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-03T01:10:05.760 INFO:teuthology.orchestra.run.smithi032.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.2 MB/s | 548 kB 00:00 2024-06-03T01:10:05.777 INFO:teuthology.orchestra.run.smithi032.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 364 kB/s | 58 kB 00:00 2024-06-03T01:10:05.860 INFO:teuthology.orchestra.run.smithi032.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-06-03T01:10:05.885 INFO:teuthology.orchestra.run.smithi032.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 407 kB/s | 51 kB 00:00 2024-06-03T01:10:05.886 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:05.886 INFO:teuthology.orchestra.run.smithi032.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-06-03T01:10:05.959 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:06.074 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:06.075 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:06.395 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-03T01:10:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-06-03T01:10:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-03T01:10:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-03T01:10:06.540 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-03T01:10:06.541 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-03T01:10:06.567 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-03T01:10:06.623 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T01:10:06.873 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:06.874 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:07.119 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-cephadm 2024-06-03T01:10:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-03T01:10:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-03T01:10:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-03T01:10:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-03T01:10:07.716 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:15 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:07.845 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:10:07.846 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout:Install 6 Packages 2024-06-03T01:10:07.847 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:07.848 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 7.0 M 2024-06-03T01:10:07.848 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 33 M 2024-06-03T01:10:07.848 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-03T01:10:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-06-03T01:10:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-03T01:10:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-03T01:10:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-03T01:10:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-03T01:10:08.365 INFO:teuthology.orchestra.run.smithi044.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-06-03T01:10:08.474 INFO:teuthology.orchestra.run.smithi044.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-06-03T01:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-03T01:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:08.499 INFO:teuthology.orchestra.run.smithi044.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-06-03T01:10:08.549 INFO:teuthology.orchestra.run.smithi044.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 766 kB/s | 58 kB 00:00 2024-06-03T01:10:08.574 INFO:teuthology.orchestra.run.smithi044.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 679 kB/s | 51 kB 00:00 2024-06-03T01:10:08.692 INFO:teuthology.orchestra.run.smithi044.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-06-03T01:10:08.692 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:08.692 INFO:teuthology.orchestra.run.smithi044.stdout:Total 8.3 MB/s | 7.0 MB 00:00 2024-06-03T01:10:08.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-fuse 2024-06-03T01:10:08.765 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:08.778 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:08.778 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:08.881 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:08.882 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:09.336 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:17 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:09.365 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-03T01:10:09.419 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-03T01:10:09.465 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:09.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:09.467 INFO:teuthology.orchestra.run.smithi032.stdout:Install 2 Packages 2024-06-03T01:10:09.467 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:09.467 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 920 k 2024-06-03T01:10:09.467 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 2.7 M 2024-06-03T01:10:09.467 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:09.670 INFO:teuthology.orchestra.run.smithi032.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-06-03T01:10:09.704 INFO:teuthology.orchestra.run.smithi032.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 724 kB/s | 80 kB 00:00 2024-06-03T01:10:09.704 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:09.705 INFO:teuthology.orchestra.run.smithi032.stdout:Total 3.8 MB/s | 920 kB 00:00 2024-06-03T01:10:09.716 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:09.739 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:09.739 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:09.822 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:09.823 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-03T01:10:10.012 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:10.065 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-03T01:10:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-03T01:10:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-03T01:10:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-03T01:10:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-03T01:10:10.876 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-03T01:10:10.876 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-06-03T01:10:10.887 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-03T01:10:10.888 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-06-03T01:10:10.888 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-03T01:10:10.888 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-03T01:10:10.888 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-03T01:10:10.888 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-03T01:10:11.155 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-03T01:10:11.155 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:11.155 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:11.155 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-06-03T01:10:11.155 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:11.156 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:11.165 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:11.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-fuse 2024-06-03T01:10:11.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librados-devel 2024-06-03T01:10:11.969 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:19 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:12.014 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:20 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:12.099 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-06-03T01:10:12.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:12.101 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 920 k 2024-06-03T01:10:12.101 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 2.7 M 2024-06-03T01:10:12.101 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:12.142 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:10:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:12.144 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 127 k 2024-06-03T01:10:12.144 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 456 k 2024-06-03T01:10:12.144 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:12.208 INFO:teuthology.orchestra.run.smithi032.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 127 kB 00:00 2024-06-03T01:10:12.209 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:12.209 INFO:teuthology.orchestra.run.smithi032.stdout:Total 1.9 MB/s | 127 kB 00:00 2024-06-03T01:10:12.209 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:12.282 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:12.282 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:12.335 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 9.6 MB/s | 839 kB 00:00 2024-06-03T01:10:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 676 kB/s | 80 kB 00:00 2024-06-03T01:10:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:Total 3.3 MB/s | 920 kB 00:00 2024-06-03T01:10:12.381 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:12.405 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:12.405 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:12.437 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:12.488 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:12.489 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:12.533 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:12.807 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:12.945 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-03T01:10:12.948 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:13.012 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:13.221 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:13.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install libcephfs2 2024-06-03T01:10:13.645 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-03T01:10:13.645 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:14.005 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:14.056 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:22 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:14.141 INFO:teuthology.orchestra.run.smithi032.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:14.186 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:14.187 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:14.187 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:14.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install libcephfs-devel 2024-06-03T01:10:14.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados-devel 2024-06-03T01:10:14.856 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:22 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:14.859 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:22 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:14.987 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:10:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:14.989 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 31 k 2024-06-03T01:10:14.989 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 139 k 2024-06-03T01:10:14.989 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:14.995 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:14.996 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 127 k 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 456 k 2024-06-03T01:10:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:15.044 INFO:teuthology.orchestra.run.smithi032.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 557 kB/s | 31 kB 00:00 2024-06-03T01:10:15.045 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:15.045 INFO:teuthology.orchestra.run.smithi032.stdout:Total 544 kB/s | 31 kB 00:00 2024-06-03T01:10:15.046 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:15.049 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:15.049 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:15.062 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:15.063 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:15.073 INFO:teuthology.orchestra.run.smithi044.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.6 MB/s | 127 kB 00:00 2024-06-03T01:10:15.074 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:15.074 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.6 MB/s | 127 kB 00:00 2024-06-03T01:10:15.074 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:15.080 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:15.080 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:15.146 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:15.146 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:15.147 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:15.261 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:15.306 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:15.629 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:15.816 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:15.865 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:16.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librados2 2024-06-03T01:10:16.095 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:16.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:16.096 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:16.096 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:16.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:16.096 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:16.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs2 2024-06-03T01:10:16.654 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:24 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:16.740 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:16.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librbd1 2024-06-03T01:10:16.879 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:24 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:16.964 INFO:teuthology.orchestra.run.smithi044.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:17.007 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:17.008 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:17.008 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:17.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs-devel 2024-06-03T01:10:17.448 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:25 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:17.533 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:17.577 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:17.578 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:17.578 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:17.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rados 2024-06-03T01:10:17.692 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:25 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:17.821 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:17.822 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:17.822 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:17.822 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:17.822 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:17.822 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 31 k 2024-06-03T01:10:17.823 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 139 k 2024-06-03T01:10:17.824 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:17.894 INFO:teuthology.orchestra.run.smithi044.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 441 kB/s | 31 kB 00:00 2024-06-03T01:10:17.895 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:17.895 INFO:teuthology.orchestra.run.smithi044.stdout:Total 433 kB/s | 31 kB 00:00 2024-06-03T01:10:17.895 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:17.898 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:17.898 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:17.920 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:17.920 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:18.000 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:26 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:18.337 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:18.381 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:18.382 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:18.382 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:18.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rgw 2024-06-03T01:10:18.501 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:18.729 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:18.732 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:18.733 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:18.733 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:18.733 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:18.733 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:18.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados2 2024-06-03T01:10:19.047 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:27 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:19.131 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:19.175 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:19.176 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:19.176 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:19.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-cephfs 2024-06-03T01:10:19.499 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:27 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:19.584 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:19.628 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:19.629 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:19.629 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:19.704 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librbd1 2024-06-03T01:10:19.838 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:27 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:20.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rbd 2024-06-03T01:10:20.297 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:28 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:20.382 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:20.426 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:20.427 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:20.427 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:20.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rados 2024-06-03T01:10:20.633 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:28 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:20.717 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:20.761 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:20.762 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-06-03T01:10:20.762 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:20.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-fuse 2024-06-03T01:10:21.089 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:29 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:21.173 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:21.220 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:21.221 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:21.221 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:21.295 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rgw 2024-06-03T01:10:21.432 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:29 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:21.560 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:21.561 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 86 k 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 226 k 2024-06-03T01:10:21.562 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:21.640 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-06-03T01:10:21.640 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:21.641 INFO:teuthology.orchestra.run.smithi032.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-06-03T01:10:21.641 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:21.648 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:21.648 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:21.716 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:21.717 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:21.888 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:29 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:21.972 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:22.015 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:22.016 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:22.016 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:22.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-cephfs 2024-06-03T01:10:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:22.587 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:22.587 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:22.675 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:30 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:22.759 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:22.803 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:22.804 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:22.804 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:22.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-mirror 2024-06-03T01:10:22.883 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rbd 2024-06-03T01:10:23.442 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:31 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:23.480 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:31 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:23.563 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-03T01:10:23.569 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:10:23.571 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:23.571 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 3.0 M 2024-06-03T01:10:23.571 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 12 M 2024-06-03T01:10:23.571 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:23.607 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:23.608 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-06-03T01:10:23.608 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:23.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-fuse 2024-06-03T01:10:23.684 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-06-03T01:10:23.685 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:23.685 INFO:teuthology.orchestra.run.smithi032.stdout:Total 26 MB/s | 3.0 MB 00:00 2024-06-03T01:10:23.685 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:23.695 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:23.695 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:23.779 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:23.779 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:24.225 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:24.268 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:32 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:24.397 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:24.397 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:10:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:24.399 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 86 k 2024-06-03T01:10:24.399 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 226 k 2024-06-03T01:10:24.399 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:24.421 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:10:24.421 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T01:10:24.421 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T01:10:24.421 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T01:10:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:24.472 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-06-03T01:10:24.473 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:24.473 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-06-03T01:10:24.473 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:24.481 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:24.481 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:24.547 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:24.547 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:24.699 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:24.797 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:25.361 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:25.487 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:25.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-nbd 2024-06-03T01:10:25.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-mirror 2024-06-03T01:10:26.220 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:34 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:26.320 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:34 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:26.349 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-06-03T01:10:26.350 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:26.350 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:26.350 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-06-03T01:10:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:26.352 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 171 k 2024-06-03T01:10:26.352 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 494 k 2024-06-03T01:10:26.352 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-06-03T01:10:26.425 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.4 MB/s | 171 kB 00:00 2024-06-03T01:10:26.425 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:26.425 INFO:teuthology.orchestra.run.smithi032.stdout:Total 2.3 MB/s | 171 kB 00:00 2024-06-03T01:10:26.425 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-06-03T01:10:26.430 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-06-03T01:10:26.430 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-06-03T01:10:26.447 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:26.449 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.0 M 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 12 M 2024-06-03T01:10:26.450 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-06-03T01:10:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-06-03T01:10:26.550 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 30 MB/s | 3.0 MB 00:00 2024-06-03T01:10:26.551 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:26.552 INFO:teuthology.orchestra.run.smithi044.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-06-03T01:10:26.552 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:26.562 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:26.563 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:26.645 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:26.645 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:26.664 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-06-03T01:10:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:27.235 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:27.261 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T01:10:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T01:10:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T01:10:27.262 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T01:10:27.262 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:27.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:27.496 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:27.496 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-06-03T01:10:27.497 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:27.497 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:10:27.497 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-06-03T01:10:27.765 DEBUG:teuthology.parallel:result is None 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:28.194 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:28.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-nbd 2024-06-03T01:10:29.011 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:37 ago on Mon 03 Jun 2024 01:08:52 AM UTC. 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 171 k 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 494 k 2024-06-03T01:10:29.141 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-06-03T01:10:29.204 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.7 MB/s | 171 kB 00:00 2024-06-03T01:10:29.205 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:29.205 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.6 MB/s | 171 kB 00:00 2024-06-03T01:10:29.205 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-06-03T01:10:29.213 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-06-03T01:10:29.213 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-06-03T01:10:29.280 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-06-03T01:10:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-06-03T01:10:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-06-03T01:10:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:10:30.236 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-06-03T01:10:30.458 DEBUG:teuthology.parallel:result is None 2024-06-03T01:10:30.458 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:10:30.458 INFO:teuthology.packaging:ref: None 2024-06-03T01:10:30.459 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:10:30.459 INFO:teuthology.packaging:branch: None 2024-06-03T01:10:30.459 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:10:30.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-03T01:10:30.608 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T01:10:30.634 INFO:teuthology.orchestra.run.smithi032.stdout:18.2.1-0.el9 2024-06-03T01:10:30.635 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-06-03T01:10:30.635 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-06-03T01:10:30.637 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-03T01:10:30.637 INFO:teuthology.packaging:ref: None 2024-06-03T01:10:30.637 INFO:teuthology.packaging:tag: v18.2.1 2024-06-03T01:10:30.637 INFO:teuthology.packaging:branch: None 2024-06-03T01:10:30.637 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:10:30.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-03T01:10:30.779 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T01:10:30.803 INFO:teuthology.orchestra.run.smithi044.stdout:18.2.1-0.el9 2024-06-03T01:10:30.804 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-06-03T01:10:30.804 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-06-03T01:10:30.805 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-03T01:10:30.806 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:10:30.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T01:10:30.838 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:10:30.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T01:10:30.872 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-03T01:10:30.872 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:10:30.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T01:10:30.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T01:10:30.977 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:10:30.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T01:10:31.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T01:10:31.074 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-03T01:10:31.074 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:10:31.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T01:10:31.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T01:10:31.171 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:10:31.171 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T01:10:31.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T01:10:31.269 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-03T01:10:31.269 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:10:31.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T01:10:31.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T01:10:31.366 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:10:31.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T01:10:31.393 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T01:10:31.461 INFO:teuthology.run_tasks:Running task print... 2024-06-03T01:10:31.471 INFO:teuthology.task.print:**** done install task... 2024-06-03T01:10:31.471 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-03T01:10:31.575 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-03T01:10:31.575 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-06-03T01:10:31.576 INFO:tasks.cephadm:Cluster fsid is 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:10:31.576 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-03T01:10:31.576 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-06-03T01:10:31.576 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi032': '172.21.15.32', 'mon.smithi044': '172.21.15.44'} 2024-06-03T01:10:31.576 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-03T01:10:31.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2024-06-03T01:10:31.608 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-06-03T01:10:31.635 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-06-03T01:10:31.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:10:31.855 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-03T01:10:32.220 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-06-03T01:10:32.221 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/f1abf795f4fc15bf2f30ac90c811a53ab21d5d65/centos/9/x86_64/flavors/default/cephadm 2024-06-03T01:10:32.221 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/f1abf795f4fc15bf2f30ac90c811a53ab21d5d65/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:32.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r--. 1 ubuntu ubuntu 215335 Jun 3 01:10 /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:32.838 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/f1abf795f4fc15bf2f30ac90c811a53ab21d5d65/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:33.271 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 215335 Jun 3 01:10 /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:33.271 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:33.292 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T01:10:33.324 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-06-03T01:10:33.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-06-03T01:10:33.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-06-03T01:10:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-03T01:10:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-03T01:10:53.198 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-06-03T01:10:53.199 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-06-03T01:10:53.733 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:10:53.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2024-06-03T01:10:53.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-06-03T01:10:53.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2024-06-03T01:10:53.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-06-03T01:10:53.906 INFO:tasks.cephadm:Writing seed config... 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-03T01:10:53.907 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-03T01:10:53.908 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-03T01:10:53.909 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-03T01:10:53.910 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-03T01:10:53.911 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:10:53.911 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-03T01:10:53.928 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 12a144f0-2146-11ef-bc9b-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-03T01:10:53.928 DEBUG:teuthology.orchestra.run.smithi032:mon.smithi032> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service 2024-06-03T01:10:53.971 INFO:tasks.cephadm:Bootstrapping... 2024-06-03T01:10:53.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:54.135 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-06-03T01:10:54.136 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '12a144f0-2146-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2024-06-03T01:10:54.161 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5.0.2 2024-06-03T01:10:54.162 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-03T01:10:54.162 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2024-06-03T01:10:54.185 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5.0.2 2024-06-03T01:10:54.185 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2024-06-03T01:10:54.185 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2024-06-03T01:10:54.193 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T01:10:54.193 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T01:10:54.199 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T01:10:54.199 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:10:54.206 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-06-03T01:10:54.212 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-06-03T01:10:54.212 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-06-03T01:10:54.213 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2024-06-03T01:10:54.236 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5.0.2 2024-06-03T01:10:54.237 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-03T01:10:54.237 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2024-06-03T01:10:54.237 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2024-06-03T01:10:54.243 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T01:10:54.243 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T01:10:54.249 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T01:10:54.250 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:10:54.256 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 139719403700528 on /run/cephadm/12a144f0-2146-11ef-bc9b-c7b262605968.lock 2024-06-03T01:10:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 139719403700528 acquired on /run/cephadm/12a144f0-2146-11ef-bc9b-c7b262605968.lock 2024-06-03T01:10:54.264 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2024-06-03T01:10:54.264 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2024-06-03T01:10:54.264 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-06-03T01:10:54.267 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.32 metric 100 2024-06-03T01:10:54.267 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2024-06-03T01:10:54.270 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-03T01:10:54.270 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-03T01:10:54.270 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-03T01:10:54.272 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T01:10:54.273 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-06-03T01:10:54.274 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-06-03T01:10:54.274 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-03T01:10:54.274 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-03T01:10:54.275 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-03T01:10:55.054 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-06-03T01:10:55.054 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-06-03T01:10:55.054 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2024-06-03T01:10:55.054 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-06-03T01:10:55.055 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-06-03T01:10:55.055 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-06-03T01:10:55.055 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-03T01:11:04.202 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-06-03T01:11:04.202 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-06-03T01:11:04.202 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2024-06-03T01:11:06.744 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2024-06-03T01:11:06.744 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2024-06-03T01:11:07.567 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQArGF1mmynQABAAM2Vrrzcs6fFR1/kDuval9g== 2024-06-03T01:11:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQArGF1m8lbaNhAAare9dKi9zCvAJttXxmVkhg== 2024-06-03T01:11:09.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQAsGF1m/z1ELRAAMbQE+zxq/SFLwrrX5xCcOA== 2024-06-03T01:11:09.520 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2024-06-03T01:11:10.859 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T01:11:10.859 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-06-03T01:11:10.859 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:10.859 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for smithi032 [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:11:10.860 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2024-06-03T01:11:12.972 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.smithi032 on 2024-06-03T01:11:13.202 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-03T01:11:13.356 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-03T01:11:13.533 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target. 2024-06-03T01:11:13.534 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target. 2024-06-03T01:11:13.761 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032 2024-06-03T01:11:13.762 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Unit ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service not loaded. 2024-06-03T01:11:13.939 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target.wants/ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service → /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968@.service. 2024-06-03T01:11:14.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:14 smithi032 podman[28972]: 2024-06-03 01:11:14.41081804 +0000 UTC m=+0.346907110 container start 3b2fd8a11564206f52337211a22fa9a96e8c05c8cf55eae96df780dbcfa0ca8f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-06-03T01:11:14.495 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T01:11:14.495 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2024-06-03T01:11:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2024-06-03T01:11:14.703 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:14 smithi032 ceph-mon[28991]: mkfs 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:14.703 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:14 smithi032 ceph-mon[28991]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-06-03T01:11:14.703 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:14 smithi032 bash[28972]: 3b2fd8a11564206f52337211a22fa9a96e8c05c8cf55eae96df780dbcfa0ca8f 2024-06-03T01:11:14.704 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:14 smithi032 systemd[1]: Started Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:11:15.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2024-06-03T01:11:15.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:15.567 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-03T01:11:15.567 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:15.567 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi032 (age 0.551725s) 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-03T01:11:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae9537a700 1 Processor -- start 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae9537a700 1 -- start start 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae9537a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae9537a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90105240 con 0x7fae90104850 2024-06-03T01:11:15.569 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57762/0 (socket says 172.21.15.32:57762) 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.921+0000 7fae8effd700 1 -- 172.21.15.32:0/304414755 learned_addr learned my addr 172.21.15.32:0/304414755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae8effd700 1 -- 172.21.15.32:0/304414755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae901077e0 con 0x7fae90104850 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae8effd700 1 --2- 172.21.15.32:0/304414755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae800098a0 tx=0x7fae80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98e24f96006707 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae8dffb700 1 -- 172.21.15.32:0/304414755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae80004180 con 0x7fae90104850 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae8dffb700 1 -- 172.21.15.32:0/304414755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae800042e0 con 0x7fae90104850 2024-06-03T01:11:15.570 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae8dffb700 1 -- 172.21.15.32:0/304414755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae800036a0 con 0x7fae90104850 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae9537a700 1 -- 172.21.15.32:0/304414755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 msgr2=0x7fae90104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.922+0000 7fae9537a700 1 --2- 172.21.15.32:0/304414755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae800098a0 tx=0x7fae80009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 -- 172.21.15.32:0/304414755 shutdown_connections 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 --2- 172.21.15.32:0/304414755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90104c70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 -- 172.21.15.32:0/304414755 >> 172.21.15.32:0/304414755 conn(0x7fae901001a0 msgr2=0x7fae901025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 -- 172.21.15.32:0/304414755 shutdown_connections 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 -- 172.21.15.32:0/304414755 wait complete. 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 Processor -- start 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.923+0000 7fae9537a700 1 -- start start 2024-06-03T01:11:15.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae9537a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae9537a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90199eb0 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57768/0 (socket says 172.21.15.32:57768) 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae8effd700 1 -- 172.21.15.32:0/2646235022 learned_addr learned my addr 172.21.15.32:0/2646235022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae8effd700 1 -- 172.21.15.32:0/2646235022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009550 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae8effd700 1 --2- 172.21.15.32:0/2646235022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae80004000 tx=0x7fae80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae80003ce0 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80020070 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9019a0b0 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae80011630 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.924+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9019a550 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.925+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae80022030 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.925+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae80025d00 con 0x7fae90104850 2024-06-03T01:11:15.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.925+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90193290 con 0x7fae90104850 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:14.928+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fae80025480 con 0x7fae90104850 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.000+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fae9002cc70 con 0x7fae90104850 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.001+0000 7fae77fff700 1 -- 172.21.15.32:0/2646235022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fae80011c20 con 0x7fae90104850 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 msgr2=0x7fae90199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 --2- 172.21.15.32:0/2646235022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae80004000 tx=0x7fae80004750 comp rx=0 tx=0).stop 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 shutdown_connections 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 --2- 172.21.15.32:0/2646235022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae90104850 0x7fae90199970 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 >> 172.21.15.32:0/2646235022 conn(0x7fae901001a0 msgr2=0x7fae9018dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 shutdown_connections 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:15.002+0000 7fae9537a700 1 -- 172.21.15.32:0/2646235022 wait complete. 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2024-06-03T01:11:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2024-06-03T01:11:15.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-06-03T01:11:15.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: monmap e1: 1 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-03T01:11:15.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: fsmap 2024-06-03T01:11:15.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T01:11:15.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: mgrmap e1: no daemons active 2024-06-03T01:11:15.686 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:15 smithi032 ceph-mon[28991]: from='client.? 172.21.15.32:0/2646235022' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-03T01:11:16.656 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:16.656 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T01:11:16.656 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:16.656 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-06-03T01:11:16.656 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T01:11:16.657 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.019+0000 7fc0c7838700 1 Processor -- start 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.019+0000 7fc0c7838700 1 -- start start 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c7838700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c7838700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0074680 con 0x7fc0c0106fe0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c55d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c55d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57774/0 (socket says 172.21.15.32:57774) 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c55d4700 1 -- 172.21.15.32:0/1543807584 learned_addr learned my addr 172.21.15.32:0/1543807584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c55d4700 1 -- 172.21.15.32:0/1543807584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0109910 con 0x7fc0c0106fe0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0c55d4700 1 --2- 172.21.15.32:0/1543807584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0098a0 tx=0x7fc0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11dd248f53ded90 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0b3fff700 1 -- 172.21.15.32:0/1543807584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0bc004180 con 0x7fc0c0106fe0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0b3fff700 1 -- 172.21.15.32:0/1543807584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc0042e0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.020+0000 7fc0b3fff700 1 -- 172.21.15.32:0/1543807584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0bc0036a0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.658 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 -- 172.21.15.32:0/1543807584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 msgr2=0x7fc0c01093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 --2- 172.21.15.32:0/1543807584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0098a0 tx=0x7fc0bc009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 -- 172.21.15.32:0/1543807584 shutdown_connections 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 --2- 172.21.15.32:0/1543807584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c01093d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 -- 172.21.15.32:0/1543807584 >> 172.21.15.32:0/1543807584 conn(0x7fc0c01009f0 msgr2=0x7fc0c0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 -- 172.21.15.32:0/1543807584 shutdown_connections 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.021+0000 7fc0c7838700 1 -- 172.21.15.32:0/1543807584 wait complete. 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c7838700 1 Processor -- start 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c7838700 1 -- start start 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c7838700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c7838700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0199eb0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c55d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c55d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57778/0 (socket says 172.21.15.32:57778) 2024-06-03T01:11:16.659 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c55d4700 1 -- 172.21.15.32:0/819960566 learned_addr learned my addr 172.21.15.32:0/819960566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.022+0000 7fc0c55d4700 1 -- 172.21.15.32:0/819960566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009550 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0c55d4700 1 --2- 172.21.15.32:0/819960566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0073ac0 tx=0x7fc0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0bc003ce0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc020070 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c019a0b0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0bc011630 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c019a550 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc0bc022030 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.023+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc0bc025d00 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.024+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c0193870 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.026+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc0bc025480 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.096+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc0c002cf20 con 0x7fc0c0106fe0 2024-06-03T01:11:16.660 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.099+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0bc011bd0 con 0x7fc0c0106fe0 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.099+0000 7fc0b27fc700 1 -- 172.21.15.32:0/819960566 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fc0bc039210 con 0x7fc0c0106fe0 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 msgr2=0x7fc0c0199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 --2- 172.21.15.32:0/819960566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0073ac0 tx=0x7fc0bc004750 comp rx=0 tx=0).stop 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 shutdown_connections 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 --2- 172.21.15.32:0/819960566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c0106fe0 0x7fc0c0199970 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 >> 172.21.15.32:0/819960566 conn(0x7fc0c01009f0 msgr2=0x7fc0c01022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 shutdown_connections 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:16.100+0000 7fc0c7838700 1 -- 172.21.15.32:0/819960566 wait complete. 2024-06-03T01:11:16.661 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2024-06-03T01:11:16.773 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:16 smithi032 ceph-mon[28991]: from='client.? 172.21.15.32:0/819960566' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T01:11:16.774 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:16 smithi032 ceph-mon[28991]: from='client.? 172.21.15.32:0/819960566' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-03T01:11:17.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:17 smithi032 ceph-mon[28991]: from='client.? 172.21.15.32:0/514497748' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a8d3d700 1 Processor -- start 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a8d3d700 1 -- start start 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a8d3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a8d3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a407a8b0 con 0x7f54a407be80 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57620/0 (socket says 172.21.15.32:57620) 2024-06-03T01:11:17.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a259c700 1 -- 172.21.15.32:0/3862110442 learned_addr learned my addr 172.21.15.32:0/3862110442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.083+0000 7f54a259c700 1 -- 172.21.15.32:0/3862110442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a407a9f0 con 0x7f54a407be80 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a259c700 1 --2- 172.21.15.32:0/3862110442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f54940098a0 tx=0x7f5494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d15d94fd68689ad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a159a700 1 -- 172.21.15.32:0/3862110442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5494004180 con 0x7f54a407be80 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a159a700 1 -- 172.21.15.32:0/3862110442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54940042e0 con 0x7f54a407be80 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a159a700 1 -- 172.21.15.32:0/3862110442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5494011a50 con 0x7f54a407be80 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a8d3d700 1 -- 172.21.15.32:0/3862110442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 msgr2=0x7f54a407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a8d3d700 1 --2- 172.21.15.32:0/3862110442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f54940098a0 tx=0x7f5494009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a8d3d700 1 -- 172.21.15.32:0/3862110442 shutdown_connections 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a8d3d700 1 --2- 172.21.15.32:0/3862110442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a407a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.084+0000 7f54a8d3d700 1 -- 172.21.15.32:0/3862110442 >> 172.21.15.32:0/3862110442 conn(0x7f54a4101b40 msgr2=0x7f54a4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.085+0000 7f54a8d3d700 1 -- 172.21.15.32:0/3862110442 shutdown_connections 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.085+0000 7f54a8d3d700 1 -- 172.21.15.32:0/3862110442 wait complete. 2024-06-03T01:11:17.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.085+0000 7f54a8d3d700 1 Processor -- start 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.085+0000 7f54a8d3d700 1 -- start start 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.085+0000 7f54a8d3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a8d3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a419c960 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57634/0 (socket says 172.21.15.32:57634) 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a259c700 1 -- 172.21.15.32:0/514497748 learned_addr learned my addr 172.21.15.32:0/514497748 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a259c700 1 -- 172.21.15.32:0/514497748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494009550 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a259c700 1 --2- 172.21.15.32:0/514497748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5494003de0 tx=0x7f5494011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5494003930 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a419cb60 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5494003a90 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54940196c0 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a419d000 con 0x7f54a407be80 2024-06-03T01:11:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5494019c50 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.086+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f549402c3a0 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.087+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a40623c0 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.089+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f549401abc0 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.161+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f54a419f9c0 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.161+0000 7f549b7fe700 1 -- 172.21.15.32:0/514497748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f549401a430 con 0x7f54a407be80 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 msgr2=0x7f54a419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 --2- 172.21.15.32:0/514497748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5494003de0 tx=0x7f5494011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 shutdown_connections 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 --2- 172.21.15.32:0/514497748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a407be80 0x7f54a419c420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 >> 172.21.15.32:0/514497748 conn(0x7f54a4101b40 msgr2=0x7f54a41031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 shutdown_connections 2024-06-03T01:11:17.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:17.162+0000 7f54a8d3d700 1 -- 172.21.15.32:0/514497748 wait complete. 2024-06-03T01:11:17.971 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2024-06-03T01:11:18.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:17 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:11:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[28987]: 2024-06-03T01:11:18.228+0000 7fbc39a80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:11:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[28987]: 2024-06-03T01:11:18.228+0000 7fbc39a80700 -1 mon.smithi032@0(leader) e1 *** Got Signal Terminated *** 2024-06-03T01:11:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 podman[29236]: 2024-06-03 01:11:18.404399416 +0000 UTC m=+0.259214886 container died 3b2fd8a11564206f52337211a22fa9a96e8c05c8cf55eae96df780dbcfa0ca8f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-06-03T01:11:18.921 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 podman[29236]: 2024-06-03 01:11:18.74611331 +0000 UTC m=+0.600928773 container cleanup 3b2fd8a11564206f52337211a22fa9a96e8c05c8cf55eae96df780dbcfa0ca8f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T01:11:18.921 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 bash[29236]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032 2024-06-03T01:11:19.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:18 smithi032 podman[29249]: 2024-06-03 01:11:18.922044184 +0000 UTC m=+0.516046155 container remove 3b2fd8a11564206f52337211a22fa9a96e8c05c8cf55eae96df780dbcfa0ca8f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, RELEASE=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.29.1) 2024-06-03T01:11:19.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Deactivated successfully. 2024-06-03T01:11:19.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 systemd[1]: Stopped Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:11:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 systemd[1]: Starting Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 podman[29342]: 2024-06-03 01:11:19.35621755 +0000 UTC m=+0.095240286 container create 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 podman[29342]: 2024-06-03 01:11:19.278608748 +0000 UTC m=+0.017631503 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 podman[29342]: 2024-06-03 01:11:19.566680389 +0000 UTC m=+0.305703143 container init 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 podman[29342]: 2024-06-03 01:11:19.569832818 +0000 UTC m=+0.308855554 container start 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T01:11:19.633 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: pidfile_write: ignore empty --pid-file 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: load: jerasure load: lrc 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: RocksDB version: 7.9.2 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Git sha 0 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Compile date 2023-12-11 22:07:34 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: DB SUMMARY 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: DB Session ID: 0O55H18KI54RBOE96DK0 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: CURRENT file: CURRENT 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T01:11:19.634 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi032/store.db dir, Total Num: 1, files: 000008.sst 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi032/store.db: 000009.log size: 98351 ; 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.error_if_exists: 0 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.create_if_missing: 0 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.paranoid_checks: 1 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.env: 0x55d69837c720 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.info_log: 0x55d69b2952e0 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.statistics: (nil) 2024-06-03T01:11:19.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.use_fsync: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_log_file_size: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_fallocate: 1 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.use_direct_reads: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T01:11:19.636 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.db_log_dir: 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.wal_dir: 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.write_buffer_manager: 0x55d69a506140 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T01:11:19.637 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T01:11:19.638 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T01:11:19.638 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T01:11:19.638 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T01:11:19.638 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.unordered_write: 0 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T01:11:19.639 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.row_cache: None 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.wal_filter: None 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.two_write_queues: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.wal_compression: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.atomic_flush: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.log_readahead_size: 0 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T01:11:19.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_background_jobs: 2 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_background_compactions: -1 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_subcompactions: 1 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T01:11:19.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_open_files: -1 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_background_flushes: -1 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Compression algorithms supported: 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T01:11:19.642 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kZSTD supported: 0 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kXpressCompression supported: 0 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kZlibCompression supported: 1 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kSnappyCompression supported: 1 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kLZ4Compression supported: 1 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: kBZip2Compression supported: 0 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000010 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.merge_operator: 2024-06-03T01:11:19.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_filter: None 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d69b2953e0) 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_type: 0 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_index_type: 0 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_shortening: 1 2024-06-03T01:11:19.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: checksum: 4 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: no_block_cache: 0 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache: 0x55d69a5a71f0 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_options: 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: capacity : 536870912 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_shard_bits : 4 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: strict_capacity_limit : 0 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_compressed: (nil) 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: persistent_cache: (nil) 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size: 4096 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size_deviation: 10 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_restart_interval: 16 2024-06-03T01:11:19.645 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_block_restart_interval: 1 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_block_size: 4096 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: partition_filters: 0 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: use_delta_encoding: 1 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: filter_policy: bloomfilter 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: whole_key_filtering: 1 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: verify_compression: 0 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: format_version: 5 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_index_compression: 1 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_align: 0 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_auto_readahead_size: 262144 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: prepopulate_block_cache: 0 2024-06-03T01:11:19.646 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: initial_auto_readahead_size: 8192 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression: NoCompression 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.num_levels: 7 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T01:11:19.647 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T01:11:19.648 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T01:11:19.649 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T01:11:19.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.table_properties_collectors: 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.inplace_update_support: 0 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.bloom_locality: 0 2024-06-03T01:11:19.651 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.max_successive_merges: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.ttl: 2592000 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enable_blob_files: false 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.min_blob_size: 0 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T01:11:19.652 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cdeefeec-51c4-4327-a8c6-47189881b844 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377079599320, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377079600427, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93595, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91617, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84226, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717377079, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cdeefeec-51c4-4327-a8c6-47189881b844", "db_session_id": "0O55H18KI54RBOE96DK0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377079600510, "job": 1, "event": "recovery_finished"} 2024-06-03T01:11:19.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi032/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d69a644000 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: DB pointer 0x55d69a630000 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** DB Stats ** 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:11:19.654 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L0 2/0 93.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Sum 2/0 93.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 126.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T01:11:19.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative compaction: 0.00 GB write, 27.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval compaction: 0.00 GB write, 27.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache BinnedLRUCache@0x55d69a5a71f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T01:11:19.656 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: starting mon.smithi032 rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi032 fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???) e1 preinit fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).mds e1 new map 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).mds e1 print_map 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: e1 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: legacy client fscid: -1 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: No filesystems configured 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:11:19.657 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 ceph-mon[29360]: mon.smithi032@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-03T01:11:19.658 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 bash[29342]: 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 2024-06-03T01:11:19.658 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:19 smithi032 systemd[1]: Started Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:11:19.658 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-03T01:11:20.737 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.091+0000 7f3ff4273700 1 Processor -- start 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.091+0000 7f3ff4273700 1 -- start start 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.091+0000 7f3ff4273700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.091+0000 7f3ff4273700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec074620 con 0x7f3fec104e30 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff200f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff200f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57638/0 (socket says 172.21.15.32:57638) 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff200f700 1 -- 172.21.15.32:0/759311730 learned_addr learned my addr 172.21.15.32:0/759311730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff200f700 1 -- 172.21.15.32:0/759311730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec105790 con 0x7f3fec104e30 2024-06-03T01:11:20.738 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff200f700 1 --2- 172.21.15.32:0/759311730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009e80 tx=0x7f3fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3603138d2fe5ca1b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff100d700 1 -- 172.21.15.32:0/759311730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe800e070 con 0x7f3fec104e30 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.092+0000 7f3ff100d700 1 -- 172.21.15.32:0/759311730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3fe8003e90 con 0x7f3fec104e30 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff100d700 1 -- 172.21.15.32:0/759311730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe8004380 con 0x7f3fec104e30 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 -- 172.21.15.32:0/759311730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 msgr2=0x7f3fec105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 --2- 172.21.15.32:0/759311730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009e80 tx=0x7f3fe800b330 comp rx=0 tx=0).stop 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 -- 172.21.15.32:0/759311730 shutdown_connections 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 --2- 172.21.15.32:0/759311730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec105250 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 -- 172.21.15.32:0/759311730 >> 172.21.15.32:0/759311730 conn(0x7f3fec1009f0 msgr2=0x7f3fec102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 -- 172.21.15.32:0/759311730 shutdown_connections 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.093+0000 7f3ff4273700 1 -- 172.21.15.32:0/759311730 wait complete. 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff4273700 1 Processor -- start 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff4273700 1 -- start start 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff4273700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff4273700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19c870 con 0x7f3fec104e30 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff200f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff200f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57648/0 (socket says 172.21.15.32:57648) 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff200f700 1 -- 172.21.15.32:0/126783669 learned_addr learned my addr 172.21.15.32:0/126783669 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.094+0000 7f3ff200f700 1 -- 172.21.15.32:0/126783669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009b30 con 0x7f3fec104e30 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3ff200f700 1 --2- 172.21.15.32:0/126783669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b7b0 tx=0x7f3fe8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe800e040 con 0x7f3fec104e30 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3fe8003c70 con 0x7f3fec104e30 2024-06-03T01:11:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec19ca70 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe801b760 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec19cf70 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3fe8019040 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.095+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fe801a6d0 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.096+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec195c10 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.098+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3fe8029060 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.168+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3fec02ce70 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.171+0000 7f3fe37fe700 1 -- 172.21.15.32:0/126783669 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3fe801f070 con 0x7f3fec104e30 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 msgr2=0x7f3fec19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 --2- 172.21.15.32:0/126783669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b7b0 tx=0x7f3fe8004430 comp rx=0 tx=0).stop 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 shutdown_connections 2024-06-03T01:11:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 --2- 172.21.15.32:0/126783669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fec104e30 0x7f3fec19c330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:20.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 >> 172.21.15.32:0/126783669 conn(0x7f3fec1009f0 msgr2=0x7f3fec1909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:20.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 shutdown_connections 2024-06-03T01:11:20.745 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:20.172+0000 7f3ff4273700 1 -- 172.21.15.32:0/126783669 wait complete. 2024-06-03T01:11:20.745 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-03T01:11:20.755 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:11:20.755 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2024-06-03T01:11:20.755 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-03T01:11:20.756 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-03T01:11:20.756 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8443 ... 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: monmap e1: 1 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: fsmap 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: mgrmap e1: no daemons active 2024-06-03T01:11:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:20 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/126783669' entity='client.admin' 2024-06-03T01:11:21.016 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mgr.smithi032.kfcfaa 2024-06-03T01:11:21.016 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mgr.smithi032.kfcfaa.service: Unit ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mgr.smithi032.kfcfaa.service not loaded. 2024-06-03T01:11:21.175 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968.target.wants/ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mgr.smithi032.kfcfaa.service → /etc/systemd/system/ceph-12a144f0-2146-11ef-bc9b-c7b262605968@.service. 2024-06-03T01:11:21.700 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T01:11:21.700 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-06-03T01:11:21.708 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T01:11:21.708 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:11:21.708 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-06-03T01:11:21.708 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T01:11:21.716 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T01:11:21.716 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-06-03T01:11:21.724 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T01:11:21.724 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:11:21.724 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-06-03T01:11:21.725 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-06-03T01:11:21.725 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2024-06-03T01:11:21.725 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2024-06-03T01:11:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:22 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3050823207' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T01:11:23.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:23.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:23.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "12a144f0-2146-11ef-bc9b-c7b262605968", 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-06-03T01:11:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T01:11:23.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T01:11:23.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T01:11:23.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T01:11:14.451395+0000", 2024-06-03T01:11:23.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.170+0000 7fbda2fb5700 1 Processor -- start 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.170+0000 7fbda2fb5700 1 -- start start 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda2fb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda2fb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c074640 con 0x7fbd9c107020 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda0d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda0d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57670/0 (socket says 172.21.15.32:57670) 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda0d51700 1 -- 172.21.15.32:0/1689151162 learned_addr learned my addr 172.21.15.32:0/1689151162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.171+0000 7fbda0d51700 1 -- 172.21.15.32:0/1689151162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c109950 con 0x7fbd9c107020 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbda0d51700 1 --2- 172.21.15.32:0/1689151162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098a0 tx=0x7fbd90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f002dde4d8641b86 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:23.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbd9b7fe700 1 -- 172.21.15.32:0/1689151162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd90004180 con 0x7fbd9c107020 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbd9b7fe700 1 -- 172.21.15.32:0/1689151162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd900042e0 con 0x7fbd9c107020 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbd9b7fe700 1 -- 172.21.15.32:0/1689151162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd90011a50 con 0x7fbd9c107020 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbda2fb5700 1 -- 172.21.15.32:0/1689151162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 msgr2=0x7fbd9c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.172+0000 7fbda2fb5700 1 --2- 172.21.15.32:0/1689151162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098a0 tx=0x7fbd90009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 -- 172.21.15.32:0/1689151162 shutdown_connections 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 --2- 172.21.15.32:0/1689151162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c109410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 -- 172.21.15.32:0/1689151162 >> 172.21.15.32:0/1689151162 conn(0x7fbd9c1009f0 msgr2=0x7fbd9c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 -- 172.21.15.32:0/1689151162 shutdown_connections 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 -- 172.21.15.32:0/1689151162 wait complete. 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 Processor -- start 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.173+0000 7fbda2fb5700 1 -- start start 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda2fb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda2fb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c106ec0 con 0x7fbd9c107020 2024-06-03T01:11:23.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda0d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda0d51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57682/0 (socket says 172.21.15.32:57682) 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda0d51700 1 -- 172.21.15.32:0/3050823207 learned_addr learned my addr 172.21.15.32:0/3050823207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda0d51700 1 -- 172.21.15.32:0/3050823207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009550 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda0d51700 1 --2- 172.21.15.32:0/3050823207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd90000c00 tx=0x7fbd90011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd90003b90 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd900248b0 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c104fd0 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.174+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd9001a8e0 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.175+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c105470 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.175+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbd9002d3f0 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.175+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbd90019580 con 0x7fbd9c107020 2024-06-03T01:11:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.176+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c10a6a0 con 0x7fbd9c107020 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.180+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fbd90024db0 con 0x7fbd9c107020 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.253+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbd9c105ad0 con 0x7fbd9c107020 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.254+0000 7fbd99ffb700 1 -- 172.21.15.32:0/3050823207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fbd90024a20 con 0x7fbd9c107020 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 msgr2=0x7fbd9c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 --2- 172.21.15.32:0/3050823207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd90000c00 tx=0x7fbd90011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 shutdown_connections 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 --2- 172.21.15.32:0/3050823207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd9c107020 0x7fbd9c106980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 >> 172.21.15.32:0/3050823207 conn(0x7fbd9c1009f0 msgr2=0x7fbd9c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 shutdown_connections 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:22.255+0000 7fbda2fb5700 1 -- 172.21.15.32:0/3050823207 wait complete. 2024-06-03T01:11:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2024-06-03T01:11:25.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:25 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/807599353' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T01:11:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "12a144f0-2146-11ef-bc9b-c7b262605968", 2024-06-03T01:11:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T01:11:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-06-03T01:11:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T01:11:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T01:11:26.190 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T01:11:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T01:11:14.451395+0000", 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:26.192 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88ed41b700 1 Processor -- start 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88ed41b700 1 -- start start 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88ed41b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88ed41b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8105e60 con 0x7f88e8105470 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57694/0 (socket says 172.21.15.32:57694) 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88e6ffd700 1 -- 172.21.15.32:0/2162474481 learned_addr learned my addr 172.21.15.32:0/2162474481 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.566+0000 7f88e6ffd700 1 -- 172.21.15.32:0/2162474481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8106670 con 0x7f88e8105470 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.567+0000 7f88e6ffd700 1 --2- 172.21.15.32:0/2162474481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88d80098a0 tx=0x7f88d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd74a7639563b07f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.567+0000 7f88e67fc700 1 -- 172.21.15.32:0/2162474481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88d8004180 con 0x7f88e8105470 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.567+0000 7f88e67fc700 1 -- 172.21.15.32:0/2162474481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d80042e0 con 0x7f88e8105470 2024-06-03T01:11:26.193 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.567+0000 7f88e67fc700 1 -- 172.21.15.32:0/2162474481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88d8011a50 con 0x7f88e8105470 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 -- 172.21.15.32:0/2162474481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 msgr2=0x7f88e8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 --2- 172.21.15.32:0/2162474481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88d80098a0 tx=0x7f88d8009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 -- 172.21.15.32:0/2162474481 shutdown_connections 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 --2- 172.21.15.32:0/2162474481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8105890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 -- 172.21.15.32:0/2162474481 >> 172.21.15.32:0/2162474481 conn(0x7f88e81009f0 msgr2=0x7f88e8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 -- 172.21.15.32:0/2162474481 shutdown_connections 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 -- 172.21.15.32:0/2162474481 wait complete. 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.568+0000 7f88ed41b700 1 Processor -- start 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88ed41b700 1 -- start start 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88ed41b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88ed41b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8190b80 con 0x7f88e8105470 2024-06-03T01:11:26.194 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57706/0 (socket says 172.21.15.32:57706) 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e6ffd700 1 -- 172.21.15.32:0/807599353 learned_addr learned my addr 172.21.15.32:0/807599353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e6ffd700 1 -- 172.21.15.32:0/807599353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d8009550 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e6ffd700 1 --2- 172.21.15.32:0/807599353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006af0 tx=0x7f88d8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88d8003b90 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d801a8a0 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.569+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e818ec90 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.570+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e818f130 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.570+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88d8019880 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.570+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f88d802d440 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.570+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f88d8024800 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.570+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c8005320 con 0x7f88e8105470 2024-06-03T01:11:26.195 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.573+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f88d801a430 con 0x7f88e8105470 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.645+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f88c8005190 con 0x7f88e8105470 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.646+0000 7f88e4ff9700 1 -- 172.21.15.32:0/807599353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f88d801a5d0 con 0x7f88e8105470 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.646+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 msgr2=0x7f88e8190640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.646+0000 7f88ed41b700 1 --2- 172.21.15.32:0/807599353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006af0 tx=0x7f88d8011a90 comp rx=0 tx=0).stop 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.647+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 shutdown_connections 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.647+0000 7f88ed41b700 1 --2- 172.21.15.32:0/807599353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e8105470 0x7f88e8190640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.647+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 >> 172.21.15.32:0/807599353 conn(0x7f88e81009f0 msgr2=0x7f88e8074130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.647+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 shutdown_connections 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:25.647+0000 7f88ed41b700 1 -- 172.21.15.32:0/807599353 wait complete. 2024-06-03T01:11:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2024-06-03T01:11:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: mgrmap e2: smithi032.kfcfaa(active, starting, since 0.0033823s) 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:11:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:28.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:28.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:28 smithi032 ceph-mon[29360]: from='mgr.14100 172.21.15.32:0/3536918272' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "12a144f0-2146-11ef-bc9b-c7b262605968", 2024-06-03T01:11:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T01:11:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T01:11:29.153 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T01:11:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T01:11:29.167 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T01:11:29.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T01:11:29.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T01:11:29.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T01:11:29.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:29.172 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T01:11:14.451395+0000", 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e34c6700 1 Processor -- start 2024-06-03T01:11:29.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e34c6700 1 -- start start 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e34c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e34c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc074680 con 0x7f60dc106fe0 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e1262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e1262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39044/0 (socket says 172.21.15.32:39044) 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.588+0000 7f60e1262700 1 -- 172.21.15.32:0/74381725 learned_addr learned my addr 172.21.15.32:0/74381725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:29.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.589+0000 7f60e1262700 1 -- 172.21.15.32:0/74381725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc109910 con 0x7f60dc106fe0 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.589+0000 7f60e1262700 1 --2- 172.21.15.32:0/74381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098a0 tx=0x7f60d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a570c02d84f18c8b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.589+0000 7f60d3fff700 1 -- 172.21.15.32:0/74381725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8004180 con 0x7f60dc106fe0 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.589+0000 7f60d3fff700 1 -- 172.21.15.32:0/74381725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60d80042e0 con 0x7f60dc106fe0 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.589+0000 7f60d3fff700 1 -- 172.21.15.32:0/74381725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8011a50 con 0x7f60dc106fe0 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- 172.21.15.32:0/74381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 msgr2=0x7f60dc1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 --2- 172.21.15.32:0/74381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098a0 tx=0x7f60d8009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- 172.21.15.32:0/74381725 shutdown_connections 2024-06-03T01:11:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 --2- 172.21.15.32:0/74381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc1093d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- 172.21.15.32:0/74381725 >> 172.21.15.32:0/74381725 conn(0x7f60dc1009f0 msgr2=0x7f60dc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- 172.21.15.32:0/74381725 shutdown_connections 2024-06-03T01:11:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- 172.21.15.32:0/74381725 wait complete. 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 Processor -- start 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.590+0000 7f60e34c6700 1 -- start start 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e34c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e34c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc19a670 con 0x7f60dc106fe0 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e1262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e1262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39046/0 (socket says 172.21.15.32:39046) 2024-06-03T01:11:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e1262700 1 -- 172.21.15.32:0/3234154936 learned_addr learned my addr 172.21.15.32:0/3234154936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e1262700 1 -- 172.21.15.32:0/3234154936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009550 con 0x7f60dc106fe0 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e1262700 1 --2- 172.21.15.32:0/3234154936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60d80119e0 tx=0x7f60d8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8003b90 con 0x7f60dc106fe0 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc19a870 con 0x7f60dc106fe0 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60d801a8a0 con 0x7f60dc106fe0 2024-06-03T01:11:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.591+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8019930 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.592+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc19ad10 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.592+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f60d802d440 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.592+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f60d80249d0 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.593+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc0623c0 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.595+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f60d801aa10 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.607+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7f60d801a430 con 0x7f60dc106fe0 2024-06-03T01:11:29.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.674+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f60dc19d8a0 con 0x7f60dc106fe0 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.674+0000 7f60d27fc700 1 -- 172.21.15.32:0/3234154936 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f60d801a6e0 con 0x7f60dc106fe0 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.675+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 msgr2=0x7f60dc19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.675+0000 7f60e34c6700 1 --2- 172.21.15.32:0/3234154936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60d80119e0 tx=0x7f60d8011ac0 comp rx=0 tx=0).stop 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.675+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 shutdown_connections 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.675+0000 7f60e34c6700 1 --2- 172.21.15.32:0/3234154936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc106fe0 0x7f60dc19a130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.675+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 >> 172.21.15.32:0/3234154936 conn(0x7f60dc1009f0 msgr2=0x7f60dc101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.676+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 shutdown_connections 2024-06-03T01:11:29.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:28.676+0000 7f60e34c6700 1 -- 172.21.15.32:0/3234154936 wait complete. 2024-06-03T01:11:29.181 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (3/15)... 2024-06-03T01:11:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:29 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3234154936' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T01:11:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:29 smithi032 ceph-mon[29360]: mgrmap e3: smithi032.kfcfaa(active, since 1.00858s) 2024-06-03T01:11:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:31 smithi032 ceph-mon[29360]: mgrmap e4: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:11:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "12a144f0-2146-11ef-bc9b-c7b262605968", 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-06-03T01:11:32.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T01:11:32.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T01:11:32.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T01:11:32.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T01:11:32.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T01:11:32.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T01:11:32.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T01:11:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T01:11:14.451395+0000", 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.567+0000 7fb39509f700 1 Processor -- start 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.567+0000 7fb39509f700 1 -- start start 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.567+0000 7fb39509f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.567+0000 7fb39509f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390079e30 con 0x7fb390107020 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39126/0 (socket says 172.21.15.32:39126) 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38ed9d700 1 -- 172.21.15.32:0/933832124 learned_addr learned my addr 172.21.15.32:0/933832124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38ed9d700 1 -- 172.21.15.32:0/933832124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390109950 con 0x7fb390107020 2024-06-03T01:11:32.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38ed9d700 1 --2- 172.21.15.32:0/933832124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3800098a0 tx=0x7fb380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d627d5d64a18282 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38dd9b700 1 -- 172.21.15.32:0/933832124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb380004180 con 0x7fb390107020 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38dd9b700 1 -- 172.21.15.32:0/933832124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3800042e0 con 0x7fb390107020 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.568+0000 7fb38dd9b700 1 -- 172.21.15.32:0/933832124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb380011a50 con 0x7fb390107020 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 -- 172.21.15.32:0/933832124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 msgr2=0x7fb390109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 --2- 172.21.15.32:0/933832124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3800098a0 tx=0x7fb380009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 -- 172.21.15.32:0/933832124 shutdown_connections 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 --2- 172.21.15.32:0/933832124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb390109410 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 -- 172.21.15.32:0/933832124 >> 172.21.15.32:0/933832124 conn(0x7fb3901009f0 msgr2=0x7fb390102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 -- 172.21.15.32:0/933832124 shutdown_connections 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.569+0000 7fb39509f700 1 -- 172.21.15.32:0/933832124 wait complete. 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb39509f700 1 Processor -- start 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb39509f700 1 -- start start 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb39509f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb39509f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39019a710 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39136/0 (socket says 172.21.15.32:39136) 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.570+0000 7fb38ed9d700 1 -- 172.21.15.32:0/2379890842 learned_addr learned my addr 172.21.15.32:0/2379890842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb38ed9d700 1 -- 172.21.15.32:0/2379890842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380009550 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb38ed9d700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb380003de0 tx=0x7fb380011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb380003930 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb380003a90 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39019a910 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb380019730 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.571+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39019adb0 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.572+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fb380019980 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.572+0000 7fb377fff700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 0x7fb37803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.572+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39010a670 con 0x7fb390107020 2024-06-03T01:11:32.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.572+0000 7fb38e59c700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 0x7fb37803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.572+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb38004bfe0 con 0x7fb390107020 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.573+0000 7fb38e59c700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 0x7fb37803a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb384006fd0 tx=0x7fb384006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.578+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb380010de0 con 0x7fb390107020 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.856+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb39010a890 con 0x7fb390107020 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.856+0000 7fb377fff700 1 -- 172.21.15.32:0/2379890842 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb38000eb20 con 0x7fb390107020 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 msgr2=0x7fb37803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 0x7fb37803a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb384006fd0 tx=0x7fb384006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 msgr2=0x7fb39019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb380003de0 tx=0x7fb380011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 shutdown_connections 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fb378038320 0x7fb37803a7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 --2- 172.21.15.32:0/2379890842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb390107020 0x7fb39019a1d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.858+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 >> 172.21.15.32:0/2379890842 conn(0x7fb3901009f0 msgr2=0x7fb3901017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.859+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 shutdown_connections 2024-06-03T01:11:32.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:31.859+0000 7fb39509f700 1 -- 172.21.15.32:0/2379890842 wait complete. 2024-06-03T01:11:32.450 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2024-06-03T01:11:32.825 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:32 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2379890842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T01:11:33.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T01:11:33.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.872+0000 7faadfad0700 1 Processor -- start 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.872+0000 7faadfad0700 1 -- start start 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.872+0000 7faadfad0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.872+0000 7faadfad0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8079db0 con 0x7faad8107080 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadd86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadd86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39152/0 (socket says 172.21.15.32:39152) 2024-06-03T01:11:33.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadd86c700 1 -- 172.21.15.32:0/3515696881 learned_addr learned my addr 172.21.15.32:0/3515696881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadd86c700 1 -- 172.21.15.32:0/3515696881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8079ef0 con 0x7faad8107080 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadd86c700 1 --2- 172.21.15.32:0/3515696881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faad40098a0 tx=0x7faad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d5a3bb8a8c0bea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadc86a700 1 -- 172.21.15.32:0/3515696881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad4004180 con 0x7faad8107080 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadc86a700 1 -- 172.21.15.32:0/3515696881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad40042e0 con 0x7faad8107080 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.873+0000 7faadc86a700 1 -- 172.21.15.32:0/3515696881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad4011a50 con 0x7faad8107080 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 -- 172.21.15.32:0/3515696881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 msgr2=0x7faad8109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 --2- 172.21.15.32:0/3515696881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faad40098a0 tx=0x7faad4009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 -- 172.21.15.32:0/3515696881 shutdown_connections 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 --2- 172.21.15.32:0/3515696881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad8109470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 -- 172.21.15.32:0/3515696881 >> 172.21.15.32:0/3515696881 conn(0x7faad81009b0 msgr2=0x7faad8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 -- 172.21.15.32:0/3515696881 shutdown_connections 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.874+0000 7faadfad0700 1 -- 172.21.15.32:0/3515696881 wait complete. 2024-06-03T01:11:33.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadfad0700 1 Processor -- start 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadfad0700 1 -- start start 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadfad0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadfad0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8106f20 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadd86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadd86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39168/0 (socket says 172.21.15.32:39168) 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadd86c700 1 -- 172.21.15.32:0/2665590767 learned_addr learned my addr 172.21.15.32:0/2665590767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadd86c700 1 -- 172.21.15.32:0/2665590767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4009550 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.875+0000 7faadd86c700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faad40119e0 tx=0x7faad4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad4003b90 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad401a8a0 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad4019930 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad8105030 con 0x7faad8107080 2024-06-03T01:11:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81054d0 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.876+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7faad4019bf0 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.877+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad810a660 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.877+0000 7faaceffd700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 0x7faac403a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.877+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faad404c680 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.877+0000 7faadd06b700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 0x7faac403a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.877+0000 7faadd06b700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 0x7faac403a750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faac8006fd0 tx=0x7faac8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:32.882+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faad401a430 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.078+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7faad810a880 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.079+0000 7faaceffd700 1 -- 172.21.15.32:0/2665590767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7faad4018350 con 0x7faad8107080 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 msgr2=0x7faac403a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 0x7faac403a750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faac8006fd0 tx=0x7faac8006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 msgr2=0x7faad81069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:33.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faad40119e0 tx=0x7faad4011ac0 comp rx=0 tx=0).stop 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 shutdown_connections 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7faac4038290 0x7faac403a750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 --2- 172.21.15.32:0/2665590767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8107080 0x7faad81069e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 >> 172.21.15.32:0/2665590767 conn(0x7faad81009b0 msgr2=0x7faad8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 shutdown_connections 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:33.082+0000 7faadfad0700 1 -- 172.21.15.32:0/2665590767 wait complete. 2024-06-03T01:11:33.608 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2024-06-03T01:11:33.922 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:33 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2665590767' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T01:11:34.984 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:34 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2443793760' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc49bd3f700 1 Processor -- start 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc49bd3f700 1 -- start start 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc49bd3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc49bd3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494079e10 con 0x7fc494104e30 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc499adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc499adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39178/0 (socket says 172.21.15.32:39178) 2024-06-03T01:11:35.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.029+0000 7fc499adb700 1 -- 172.21.15.32:0/1732155992 learned_addr learned my addr 172.21.15.32:0/1732155992 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.030+0000 7fc499adb700 1 -- 172.21.15.32:0/1732155992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494105790 con 0x7fc494104e30 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.030+0000 7fc499adb700 1 --2- 172.21.15.32:0/1732155992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc4900098a0 tx=0x7fc490009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df52db58f435684a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.030+0000 7fc498ad9700 1 -- 172.21.15.32:0/1732155992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc490004180 con 0x7fc494104e30 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.030+0000 7fc498ad9700 1 -- 172.21.15.32:0/1732155992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4900042e0 con 0x7fc494104e30 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.030+0000 7fc498ad9700 1 -- 172.21.15.32:0/1732155992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc490011a50 con 0x7fc494104e30 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- 172.21.15.32:0/1732155992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 msgr2=0x7fc494105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/1732155992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc4900098a0 tx=0x7fc490009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- 172.21.15.32:0/1732155992 shutdown_connections 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/1732155992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc494105250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- 172.21.15.32:0/1732155992 >> 172.21.15.32:0/1732155992 conn(0x7fc4941009f0 msgr2=0x7fc494102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- 172.21.15.32:0/1732155992 shutdown_connections 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- 172.21.15.32:0/1732155992 wait complete. 2024-06-03T01:11:35.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 Processor -- start 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.031+0000 7fc49bd3f700 1 -- start start 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc49bd3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc49bd3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49419c910 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc499adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc499adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39188/0 (socket says 172.21.15.32:39188) 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc499adb700 1 -- 172.21.15.32:0/2443793760 learned_addr learned my addr 172.21.15.32:0/2443793760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc499adb700 1 -- 172.21.15.32:0/2443793760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490009550 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc499adb700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc490003de0 tx=0x7fc490011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc490003930 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc490003a90 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49419cb10 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc490019730 con 0x7fc494104e30 2024-06-03T01:11:35.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.032+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49419cfb0 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.033+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fc4900199b0 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.033+0000 7fc48affd700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 0x7fc48003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.033+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc49004c090 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.033+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494195cd0 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.033+0000 7fc4992da700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 0x7fc48003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.034+0000 7fc4992da700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 0x7fc48003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.038+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc490010de0 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.291+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc49402cc70 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.663+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc49000fd80 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.664+0000 7fc48affd700 1 -- 172.21.15.32:0/2443793760 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc49002c960 con 0x7fc494104e30 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 msgr2=0x7fc48003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 0x7fc48003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:35.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 msgr2=0x7fc49419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc490003de0 tx=0x7fc490011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 shutdown_connections 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7fc4800382d0 0x7fc48003a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 --2- 172.21.15.32:0/2443793760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc494104e30 0x7fc49419c3d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 >> 172.21.15.32:0/2443793760 conn(0x7fc4941009f0 msgr2=0x7fc4941909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 shutdown_connections 2024-06-03T01:11:35.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:34.666+0000 7fc49bd3f700 1 -- 172.21.15.32:0/2443793760 wait complete. 2024-06-03T01:11:35.787 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:35 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2443793760' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-03T01:11:35.787 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:35 smithi032 ceph-mon[29360]: mgrmap e5: smithi032.kfcfaa(active, since 6s) 2024-06-03T01:11:36.551 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "smithi032.kfcfaa", 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.691+0000 7f7ef62a8700 1 Processor -- start 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.691+0000 7f7ef62a8700 1 -- start start 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.691+0000 7f7ef62a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:36.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.691+0000 7f7ef62a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0074680 con 0x7f7ef0106fe0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39220/0 (socket says 172.21.15.32:39220) 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeffff700 1 -- 172.21.15.32:0/4143000849 learned_addr learned my addr 172.21.15.32:0/4143000849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeffff700 1 -- 172.21.15.32:0/4143000849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0109910 con 0x7f7ef0106fe0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeffff700 1 --2- 172.21.15.32:0/4143000849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00098a0 tx=0x7f7ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fed9250cca68aa server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeeffd700 1 -- 172.21.15.32:0/4143000849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ee0004180 con 0x7f7ef0106fe0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.692+0000 7f7eeeffd700 1 -- 172.21.15.32:0/4143000849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ee00042e0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.693+0000 7f7eeeffd700 1 -- 172.21.15.32:0/4143000849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ee0011a50 con 0x7f7ef0106fe0 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.693+0000 7f7ef62a8700 1 -- 172.21.15.32:0/4143000849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 msgr2=0x7f7ef01093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.693+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/4143000849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00098a0 tx=0x7f7ee0009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.693+0000 7f7ef62a8700 1 -- 172.21.15.32:0/4143000849 shutdown_connections 2024-06-03T01:11:36.558 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/4143000849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef01093d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 -- 172.21.15.32:0/4143000849 >> 172.21.15.32:0/4143000849 conn(0x7f7ef01009f0 msgr2=0x7f7ef0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 -- 172.21.15.32:0/4143000849 shutdown_connections 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 -- 172.21.15.32:0/4143000849 wait complete. 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 Processor -- start 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.694+0000 7f7ef62a8700 1 -- start start 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7ef62a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7ef62a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0106e80 con 0x7f7ef0106fe0 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39222/0 (socket says 172.21.15.32:39222) 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7eeffff700 1 -- 172.21.15.32:0/3555940328 learned_addr learned my addr 172.21.15.32:0/3555940328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7eeffff700 1 -- 172.21.15.32:0/3555940328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0009550 con 0x7f7ef0106fe0 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.695+0000 7f7eeffff700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.696+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ee0003b90 con 0x7f7ef0106fe0 2024-06-03T01:11:36.559 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.696+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef0104f90 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.696+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ee00248b0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.696+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ee001a8e0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.696+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef0105430 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7ee001aba0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7eed7fa700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 0x7f7ed803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7ee004c0a0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7eef7fe700 1 -- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 msgr2=0x7f7ed803a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7eef7fe700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 0x7f7ed803a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.697+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef010a690 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.702+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7ee00233a0 con 0x7f7ef0106fe0 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.898+0000 7f7eef7fe700 1 -- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 msgr2=0x7f7ed803a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:36.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:35.898+0000 7f7eef7fe700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 0x7f7ed803a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.022+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7ef01a0180 con 0x7f7ef0106fe0 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.022+0000 7f7eed7fa700 1 -- 172.21.15.32:0/3555940328 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f7ee0018b00 con 0x7f7ef0106fe0 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.024+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 msgr2=0x7f7ed803a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.024+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 0x7f7ed803a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.024+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 msgr2=0x7f7ef0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.024+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 shutdown_connections 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f7ed8038320 0x7f7ed803a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 --2- 172.21.15.32:0/3555940328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0106fe0 0x7f7ef0106940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 >> 172.21.15.32:0/3555940328 conn(0x7f7ef01009f0 msgr2=0x7f7ef0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 shutdown_connections 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.025+0000 7f7ef62a8700 1 -- 172.21.15.32:0/3555940328 wait complete. 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-06-03T01:11:36.561 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2024-06-03T01:11:36.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:36 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3555940328' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T01:11:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: Active manager daemon smithi032.kfcfaa restarted 2024-06-03T01:11:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: osdmap e2: 0 total, 0 up, 0 in 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: mgrmap e6: smithi032.kfcfaa(active, starting, since 0.00307949s) 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:42.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:11:42.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:41 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:11:42.912 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:42 smithi032 ceph-mon[29360]: Found migration_current of "None". Setting to last migration. 2024-06-03T01:11:42.912 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:42 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:42.912 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:42 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:42.912 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:42 smithi032 ceph-mon[29360]: mgrmap e7: smithi032.kfcfaa(active, since 1.00503s) 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.950+0000 7f79e2758700 1 Processor -- start 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.950+0000 7f79e2758700 1 -- start start 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79e2758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79e2758700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc105e60 con 0x7f79dc105470 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39780/0 (socket says 172.21.15.32:39780) 2024-06-03T01:11:43.239 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79dbfff700 1 -- 172.21.15.32:0/736036129 learned_addr learned my addr 172.21.15.32:0/736036129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79dbfff700 1 -- 172.21.15.32:0/736036129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc106670 con 0x7f79dc105470 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79dbfff700 1 --2- 172.21.15.32:0/736036129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098a0 tx=0x7f79cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210d8bd77c7a57ff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.951+0000 7f79daffd700 1 -- 172.21.15.32:0/736036129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc004180 con 0x7f79dc105470 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79daffd700 1 -- 172.21.15.32:0/736036129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79cc0042e0 con 0x7f79dc105470 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79daffd700 1 -- 172.21.15.32:0/736036129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc011a50 con 0x7f79dc105470 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 -- 172.21.15.32:0/736036129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 msgr2=0x7f79dc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 --2- 172.21.15.32:0/736036129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098a0 tx=0x7f79cc009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 -- 172.21.15.32:0/736036129 shutdown_connections 2024-06-03T01:11:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 --2- 172.21.15.32:0/736036129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc105890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 -- 172.21.15.32:0/736036129 >> 172.21.15.32:0/736036129 conn(0x7f79dc1009f0 msgr2=0x7f79dc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 -- 172.21.15.32:0/736036129 shutdown_connections 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.952+0000 7f79e2758700 1 -- 172.21.15.32:0/736036129 wait complete. 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.953+0000 7f79e2758700 1 Processor -- start 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.953+0000 7f79e2758700 1 -- start start 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.953+0000 7f79e2758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.953+0000 7f79e2758700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19a6e0 con 0x7f79dc105470 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.953+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39784/0 (socket says 172.21.15.32:39784) 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79dbfff700 1 -- 172.21.15.32:0/3435506000 learned_addr learned my addr 172.21.15.32:0/3435506000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79dbfff700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009550 con 0x7f79dc105470 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79dbfff700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c00 tx=0x7f79cc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc003b90 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc19a8e0 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79cc0248b0 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc01a8e0 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.954+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc19ad80 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f79cc01aba0 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79d97fa700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79c403aef0 con 0x7f79c4038320 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f79cc04c190 con 0x7f79dc105470 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79db7fe700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:36.955+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:37.156+0000 7f79db7fe700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:37.156+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:11:43.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:37.557+0000 7f79db7fe700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:37.557+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:38.358+0000 7f79db7fe700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:38.358+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:39.959+0000 7f79db7fe700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/458796220 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:39.959+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:41.611+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f79cc02d3f0 con 0x7f79dc105470 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:41.612+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 msgr2=0x7f79c403a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:41.612+0000 7f79d97fa700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.612+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f79cc04dcc0 con 0x7f79dc105470 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.613+0000 7f79d97fa700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 0x7f79c403dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.613+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79c403aef0 con 0x7f79c403bc00 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.613+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 0x7f79c403dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.614+0000 7f79db7fe700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 0x7f79c403dff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79d0003a10 tx=0x7f79d0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.614+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f79c403aef0 con 0x7f79c403bc00 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f79dc19b4e0 con 0x7f79c403bc00 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79d97fa700 1 -- 172.21.15.32:0/3435506000 <== mgr.14120 v2:172.21.15.32:6800/3147998267 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f79dc19b4e0 con 0x7f79c403bc00 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 msgr2=0x7f79c403dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79e2758700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 0x7f79c403dff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79d0003a10 tx=0x7f79d0009290 comp rx=0 tx=0).stop 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 msgr2=0x7f79dc19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.621+0000 7f79e2758700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c00 tx=0x7f79cc011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 shutdown_connections 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f79c403bc00 0x7f79c403dff0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc105470 0x7f79dc19a1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 --2- 172.21.15.32:0/3435506000 >> [v2:172.21.15.32:6800/458796220,v1:172.21.15.32:6801/458796220] conn(0x7f79c4038320 0x7f79c403a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 >> 172.21.15.32:0/3435506000 conn(0x7f79dc1009f0 msgr2=0x7f79dc079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 shutdown_connections 2024-06-03T01:11:43.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:42.622+0000 7f79e2758700 1 -- 172.21.15.32:0/3435506000 wait complete. 2024-06-03T01:11:43.245 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2024-06-03T01:11:43.245 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2024-06-03T01:11:43.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:43 smithi032 ceph-mon[29360]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T01:11:43.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:43 smithi032 ceph-mon[29360]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T01:11:43.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:43 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:44.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.653+0000 7f5d6dc21700 1 Processor -- start 2024-06-03T01:11:44.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d6dc21700 1 -- start start 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d6dc21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d6dc21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68079e50 con 0x7f5d68107010 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39866/0 (socket says 172.21.15.32:39866) 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d677fe700 1 -- 172.21.15.32:0/1192179262 learned_addr learned my addr 172.21.15.32:0/1192179262 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.654+0000 7f5d677fe700 1 -- 172.21.15.32:0/1192179262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68109940 con 0x7f5d68107010 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d677fe700 1 --2- 172.21.15.32:0/1192179262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098a0 tx=0x7f5d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21affe8acc4a62d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d667fc700 1 -- 172.21.15.32:0/1192179262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d58004180 con 0x7f5d68107010 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d667fc700 1 -- 172.21.15.32:0/1192179262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d580042e0 con 0x7f5d68107010 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d667fc700 1 -- 172.21.15.32:0/1192179262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d58011a50 con 0x7f5d68107010 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d6dc21700 1 -- 172.21.15.32:0/1192179262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 msgr2=0x7f5d68109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/1192179262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098a0 tx=0x7f5d58009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d6dc21700 1 -- 172.21.15.32:0/1192179262 shutdown_connections 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/1192179262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d68109400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.655+0000 7f5d6dc21700 1 -- 172.21.15.32:0/1192179262 >> 172.21.15.32:0/1192179262 conn(0x7f5d68100a20 msgr2=0x7f5d68102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.656+0000 7f5d6dc21700 1 -- 172.21.15.32:0/1192179262 shutdown_connections 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.656+0000 7f5d6dc21700 1 -- 172.21.15.32:0/1192179262 wait complete. 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.656+0000 7f5d6dc21700 1 Processor -- start 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.656+0000 7f5d6dc21700 1 -- start start 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d6dc21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d6dc21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6819a720 con 0x7f5d68107010 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39868/0 (socket says 172.21.15.32:39868) 2024-06-03T01:11:44.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d677fe700 1 -- 172.21.15.32:0/141923739 learned_addr learned my addr 172.21.15.32:0/141923739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d677fe700 1 -- 172.21.15.32:0/141923739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58009550 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d677fe700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d58003de0 tx=0x7f5d58011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d58003930 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d58003a90 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d58019730 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6819a920 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.657+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6819adc0 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.658+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f5d58019960 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.658+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6810a690 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.658+0000 7f5d64ff9700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 0x7f5d5003a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.658+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d5804c240 con 0x7f5d68107010 2024-06-03T01:11:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.659+0000 7f5d66ffd700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 0x7f5d5003a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.659+0000 7f5d66ffd700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 0x7f5d5003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c006fd0 tx=0x7f5d5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.663+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d58010770 con 0x7f5d68107010 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.668+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5d5801a430 con 0x7f5d68107010 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.881+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d68101eb0 con 0x7f5d50038250 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.887+0000 7f5d64ff9700 1 -- 172.21.15.32:0/141923739 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5d68101eb0 con 0x7f5d50038250 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 msgr2=0x7f5d5003a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 0x7f5d5003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c006fd0 tx=0x7f5d5c006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 msgr2=0x7f5d6819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d58003de0 tx=0x7f5d58011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 shutdown_connections 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d50038250 0x7f5d5003a710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 --2- 172.21.15.32:0/141923739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d68107010 0x7f5d6819a1e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 >> 172.21.15.32:0/141923739 conn(0x7f5d68100a20 msgr2=0x7f5d681017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:44.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 shutdown_connections 2024-06-03T01:11:44.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:43.890+0000 7f5d6dc21700 1 -- 172.21.15.32:0/141923739 wait complete. 2024-06-03T01:11:44.730 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: [03/Jun/2024:01:11:43] ENGINE Bus STARTING 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: [03/Jun/2024:01:11:43] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: [03/Jun/2024:01:11:43] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: [03/Jun/2024:01:11:43] ENGINE Bus STARTED 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: mgrmap e8: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:44.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:44 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:45.592 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789cf21700 1 Processor -- start 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789cf21700 1 -- start start 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789cf21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789cf21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898079e30 con 0x7f7898107020 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39874/0 (socket says 172.21.15.32:39874) 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789659c700 1 -- 172.21.15.32:0/1078030832 learned_addr learned my addr 172.21.15.32:0/1078030832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.841+0000 7f789659c700 1 -- 172.21.15.32:0/1078030832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898109950 con 0x7f7898107020 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789659c700 1 --2- 172.21.15.32:0/1078030832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78880098a0 tx=0x7f7888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16d746fac44e68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789559a700 1 -- 172.21.15.32:0/1078030832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7888004180 con 0x7f7898107020 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789559a700 1 -- 172.21.15.32:0/1078030832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78880042e0 con 0x7f7898107020 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789559a700 1 -- 172.21.15.32:0/1078030832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7888011a50 con 0x7f7898107020 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789cf21700 1 -- 172.21.15.32:0/1078030832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 msgr2=0x7f7898109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.842+0000 7f789cf21700 1 --2- 172.21.15.32:0/1078030832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78880098a0 tx=0x7f7888009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 -- 172.21.15.32:0/1078030832 shutdown_connections 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 --2- 172.21.15.32:0/1078030832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f7898109410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 -- 172.21.15.32:0/1078030832 >> 172.21.15.32:0/1078030832 conn(0x7f78981009b0 msgr2=0x7f7898102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 -- 172.21.15.32:0/1078030832 shutdown_connections 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 -- 172.21.15.32:0/1078030832 wait complete. 2024-06-03T01:11:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 Processor -- start 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.843+0000 7f789cf21700 1 -- start start 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789cf21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789cf21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789819a750 con 0x7f7898107020 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39888/0 (socket says 172.21.15.32:39888) 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789659c700 1 -- 172.21.15.32:0/701519664 learned_addr learned my addr 172.21.15.32:0/701519664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789659c700 1 -- 172.21.15.32:0/701519664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888009550 con 0x7f7898107020 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789659c700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7888003de0 tx=0x7f7888011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7888003930 con 0x7f7898107020 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7888003a90 con 0x7f7898107020 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789819a950 con 0x7f7898107020 2024-06-03T01:11:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7888019730 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.844+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789819adf0 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.845+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f78880199b0 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.846+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789810a670 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.846+0000 7f78877fe700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 0x7f788003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.846+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f788804c600 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.846+0000 7f7895d9b700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 0x7f788003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.846+0000 7f7895d9b700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 0x7f788003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:44.851+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7888010dc0 con 0x7f7898107020 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.052+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7898102cc0 con 0x7f7880038320 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.053+0000 7f78877fe700 1 -- 172.21.15.32:0/701519664 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7898102cc0 con 0x7f7880038320 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 msgr2=0x7f788003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 0x7f788003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:45.596 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 msgr2=0x7f789819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7888003de0 tx=0x7f7888011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 shutdown_connections 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7880038320 0x7f788003a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 --2- 172.21.15.32:0/701519664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7898107020 0x7f789819a210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 >> 172.21.15.32:0/701519664 conn(0x7f78981009b0 msgr2=0x7f7898102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 shutdown_connections 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:45.055+0000 7f789cf21700 1 -- 172.21.15.32:0/701519664 wait complete. 2024-06-03T01:11:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2024-06-03T01:11:45.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:45 smithi032 ceph-mon[29360]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:47.007 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:46 smithi032 ceph-mon[29360]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:47.008 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:46 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:47.008 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:46 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efeeea700 1 Processor -- start 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efeeea700 1 -- start start 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efeeea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efeeea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8079e10 con 0x7f9ef8107050 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efcc86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efcc86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39894/0 (socket says 172.21.15.32:39894) 2024-06-03T01:11:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efcc86700 1 -- 172.21.15.32:0/211870766 learned_addr learned my addr 172.21.15.32:0/211870766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.007+0000 7f9efcc86700 1 -- 172.21.15.32:0/211870766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8109980 con 0x7f9ef8107050 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9efcc86700 1 --2- 172.21.15.32:0/211870766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098a0 tx=0x7f9eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fb7b4800c93b0d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9ef77fe700 1 -- 172.21.15.32:0/211870766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec004180 con 0x7f9ef8107050 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9ef77fe700 1 -- 172.21.15.32:0/211870766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9eec0042e0 con 0x7f9ef8107050 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9ef77fe700 1 -- 172.21.15.32:0/211870766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec011a50 con 0x7f9ef8107050 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9efeeea700 1 -- 172.21.15.32:0/211870766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 msgr2=0x7f9ef8109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9efeeea700 1 --2- 172.21.15.32:0/211870766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098a0 tx=0x7f9eec009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9efeeea700 1 -- 172.21.15.32:0/211870766 shutdown_connections 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.008+0000 7f9efeeea700 1 --2- 172.21.15.32:0/211870766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef8109440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 -- 172.21.15.32:0/211870766 >> 172.21.15.32:0/211870766 conn(0x7f9ef8100a20 msgr2=0x7f9ef8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 -- 172.21.15.32:0/211870766 shutdown_connections 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 -- 172.21.15.32:0/211870766 wait complete. 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 Processor -- start 2024-06-03T01:11:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 -- start start 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.009+0000 7f9efeeea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef819a760 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efcc86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efcc86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39900/0 (socket says 172.21.15.32:39900) 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efcc86700 1 -- 172.21.15.32:0/1875841060 learned_addr learned my addr 172.21.15.32:0/1875841060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efcc86700 1 -- 172.21.15.32:0/1875841060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec009550 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efcc86700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9eec003de0 tx=0x7f9eec011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec003930 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9eec003a90 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef819a960 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec019730 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.010+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef819ae00 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.011+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9eec019960 con 0x7f9ef8107050 2024-06-03T01:11:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.011+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef810a6a0 con 0x7f9ef8107050 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.011+0000 7f9ef5ffb700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 0x7f9ee803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.012+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9eec04c3d0 con 0x7f9ef8107050 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.012+0000 7f9ef7fff700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 0x7f9ee803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.012+0000 7f9ef7fff700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 0x7f9ee803a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.016+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9eec01abc0 con 0x7f9ef8107050 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.214+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ef8101ee0 con 0x7f9ee8038370 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.412+0000 7f9ef5ffb700 1 -- 172.21.15.32:0/1875841060 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9ef8101ee0 con 0x7f9ee8038370 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 msgr2=0x7f9ee803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 0x7f9ee803a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 msgr2=0x7f9ef819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9eec003de0 tx=0x7f9eec011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:47.018 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 shutdown_connections 2024-06-03T01:11:47.019 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ee8038370 0x7f9ee803a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:47.019 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 --2- 172.21.15.32:0/1875841060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ef8107050 0x7f9ef819a220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:47.019 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 >> 172.21.15.32:0/1875841060 conn(0x7f9ef8100a20 msgr2=0x7f9ef81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:47.019 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 shutdown_connections 2024-06-03T01:11:47.019 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:46.414+0000 7f9efeeea700 1 -- 172.21.15.32:0/1875841060 wait complete. 2024-06-03T01:11:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:47 smithi032 ceph-mon[29360]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:47 smithi032 ceph-mon[29360]: Generating ssh key... 2024-06-03T01:11:48.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1ha3nMV9AHJ8oh58MpwmsalUDE8BeM6zTvmui+kxSiRYtEE+pTAIAGVyz0mDKEsX2H/kAjyvLwI9twWHbCJwCmrzcCw9R7afgRD0aL+uk5bft9tY3iMJGTObXGUIOkAm0HrHu2QWvskQ1ewDdxecC+Ymgeph0Jf0vWx9V+sl453rHVIoZetmyn4WmmdM3dQxYAba5dlb9FpRpFN0kkMd8MfyE4Up3Gg56WKd0WG3EkKfV5Afb/RnilqpCLBQO2fBz46ywagOhAMSz6xo4FzS9jgB6sM5W0YW+Ov425hE0VkJklIkCzg+sgM9dRjrx9wdqfkQkILYS9kKLzxEAyOGI721ob/o1Nd37whhY2Io/MauFHw6goJLyfdmycrBVnFOAN4jkMFiqjSj+IkehjfMRnBf81LiENn+2no2ld8LC7xzmLYJ4e9/NuNhqp/5lTZReJPTUgE9/jkppQiHQcDS1V9iHf1a491DK13+vO+uyBp2+FgGH6OnvU9qHi2OZgbk= ceph-12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:11:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b76d9c700 1 Processor -- start 2024-06-03T01:11:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b76d9c700 1 -- start start 2024-06-03T01:11:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b76d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b76d9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70105e50 con 0x7f2b70105460 2024-06-03T01:11:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b74b38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b74b38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b74b38700 1 -- 172.21.15.32:0/2204048482 learned_addr learned my addr 172.21.15.32:0/2204048482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.432+0000 7f2b74b38700 1 -- 172.21.15.32:0/2204048482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70106660 con 0x7f2b70105460 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b74b38700 1 --2- 172.21.15.32:0/2204048482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e80 tx=0x7f2b6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a466575b644b8720 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b6f7fe700 1 -- 172.21.15.32:0/2204048482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b6000e070 con 0x7f2b70105460 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b6f7fe700 1 -- 172.21.15.32:0/2204048482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2b60003e90 con 0x7f2b70105460 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b6f7fe700 1 -- 172.21.15.32:0/2204048482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b60004380 con 0x7f2b70105460 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 -- 172.21.15.32:0/2204048482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 msgr2=0x7f2b70105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/2204048482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e80 tx=0x7f2b6000b330 comp rx=0 tx=0).stop 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 -- 172.21.15.32:0/2204048482 shutdown_connections 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/2204048482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70105880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 -- 172.21.15.32:0/2204048482 >> 172.21.15.32:0/2204048482 conn(0x7f2b70100a20 msgr2=0x7f2b70102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 -- 172.21.15.32:0/2204048482 shutdown_connections 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.433+0000 7f2b76d9c700 1 -- 172.21.15.32:0/2204048482 wait complete. 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b76d9c700 1 Processor -- start 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b76d9c700 1 -- start start 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b76d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b76d9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70190b70 con 0x7f2b70105460 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b74b38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b74b38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34964/0 (socket says 172.21.15.32:34964) 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.434+0000 7f2b74b38700 1 -- 172.21.15.32:0/1758854982 learned_addr learned my addr 172.21.15.32:0/1758854982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b74b38700 1 -- 172.21.15.32:0/1758854982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009b30 con 0x7f2b70105460 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b74b38700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e50 tx=0x7f2b60004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b6000e040 con 0x7f2b70105460 2024-06-03T01:11:48.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2b60003cf0 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7018ec80 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b6001b770 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.435+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7018f180 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.436+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2b60019040 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.436+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700623c0 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.436+0000 7f2b6dffb700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 0x7f2b5c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.436+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2b6004bdb0 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.436+0000 7f2b6ffff700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 0x7f2b5c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.437+0000 7f2b6ffff700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 0x7f2b5c03a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006fd0 tx=0x7f2b58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.441+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b60011570 con 0x7f2b70105460 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.638+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b70102b50 con 0x7f2b5c038320 2024-06-03T01:11:48.184 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.639+0000 7f2b6dffb700 1 -- 172.21.15.32:0/1758854982 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2b70102b50 con 0x7f2b5c038320 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 msgr2=0x7f2b5c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 0x7f2b5c03a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006fd0 tx=0x7f2b58006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 msgr2=0x7f2b70190630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e50 tx=0x7f2b60004920 comp rx=0 tx=0).stop 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 shutdown_connections 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f2b5c038320 0x7f2b5c03a7e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 --2- 172.21.15.32:0/1758854982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b70105460 0x7f2b70190630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 >> 172.21.15.32:0/1758854982 conn(0x7f2b70100a20 msgr2=0x7f2b70079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 shutdown_connections 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:47.641+0000 7f2b76d9c700 1 -- 172.21.15.32:0/1758854982 wait complete. 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2024-06-03T01:11:48.185 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2024-06-03T01:11:48.817 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:48 smithi032 ceph-mon[29360]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:50.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:50 smithi032 ceph-mon[29360]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:50.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:50 smithi032 ceph-mon[29360]: Deploying cephadm binary to smithi032 2024-06-03T01:11:51.804 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2024-06-03T01:11:51.804 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f0870812700 1 Processor -- start 2024-06-03T01:11:51.804 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f0870812700 1 -- start start 2024-06-03T01:11:51.804 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f0870812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f0870812700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c106f50 con 0x7f086c106560 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f086a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f086a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34980/0 (socket says 172.21.15.32:34980) 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.601+0000 7f086a59c700 1 -- 172.21.15.32:0/552773009 learned_addr learned my addr 172.21.15.32:0/552773009 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.602+0000 7f086a59c700 1 -- 172.21.15.32:0/552773009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c107760 con 0x7f086c106560 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.602+0000 7f086a59c700 1 --2- 172.21.15.32:0/552773009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f085c0098a0 tx=0x7f085c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4858d448618a49 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.602+0000 7f086959a700 1 -- 172.21.15.32:0/552773009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f085c004180 con 0x7f086c106560 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.602+0000 7f086959a700 1 -- 172.21.15.32:0/552773009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f085c0042e0 con 0x7f086c106560 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.602+0000 7f086959a700 1 -- 172.21.15.32:0/552773009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f085c011a50 con 0x7f086c106560 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 -- 172.21.15.32:0/552773009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 msgr2=0x7f086c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 --2- 172.21.15.32:0/552773009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f085c0098a0 tx=0x7f085c009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 -- 172.21.15.32:0/552773009 shutdown_connections 2024-06-03T01:11:51.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 --2- 172.21.15.32:0/552773009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c106980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 -- 172.21.15.32:0/552773009 >> 172.21.15.32:0/552773009 conn(0x7f086c101b40 msgr2=0x7f086c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 -- 172.21.15.32:0/552773009 shutdown_connections 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.603+0000 7f0870812700 1 -- 172.21.15.32:0/552773009 wait complete. 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f0870812700 1 Processor -- start 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f0870812700 1 -- start start 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f0870812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f0870812700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c07c170 con 0x7f086c106560 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f086a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f086a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34994/0 (socket says 172.21.15.32:34994) 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f086a59c700 1 -- 172.21.15.32:0/642842868 learned_addr learned my addr 172.21.15.32:0/642842868 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.604+0000 7f086a59c700 1 -- 172.21.15.32:0/642842868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c009550 con 0x7f086c106560 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f086a59c700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f085c003de0 tx=0x7f085c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:51.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f085c003930 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f085c003a90 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c07a280 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f085c0196c0 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.605+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c07a720 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.606+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f085c01a430 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.606+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c0623c0 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.606+0000 7f085b7fe700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 0x7f085403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.606+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f085c04c3e0 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.606+0000 7f0869d9b700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 0x7f085403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.607+0000 7f0869d9b700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 0x7f085403a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.611+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f085c010560 con 0x7f086c106560 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:48.816+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) v1 -- 0x7f086c103010 con 0x7f0854038320 2024-06-03T01:11:51.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.268+0000 7f085b7fe700 1 -- 172.21.15.32:0/642842868 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f086c103010 con 0x7f0854038320 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 msgr2=0x7f085403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 0x7f085403a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 msgr2=0x7f086c07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f085c003de0 tx=0x7f085c011ab0 comp rx=0 tx=0).stop 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 shutdown_connections 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0854038320 0x7f085403a7e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 --2- 172.21.15.32:0/642842868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f086c106560 0x7f086c07bc30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.270+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 >> 172.21.15.32:0/642842868 conn(0x7f086c101b40 msgr2=0x7f086c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.271+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 shutdown_connections 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:51.271+0000 7f0870812700 1 -- 172.21.15.32:0/642842868 wait complete. 2024-06-03T01:11:51.808 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying mon service with default placement... 2024-06-03T01:11:52.443 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:52 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:52.443 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:52 smithi032 ceph-mon[29360]: Added host smithi032 2024-06-03T01:11:52.443 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:52 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:53.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:53 smithi032 ceph-mon[29360]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:53.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:53 smithi032 ceph-mon[29360]: Saving service mon spec with placement count:5 2024-06-03T01:11:53.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:53 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:53.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:53 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.230+0000 7f9aff0e2700 1 Processor -- start 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.230+0000 7f9aff0e2700 1 -- start start 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9aff0e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9aff0e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8106f50 con 0x7f9af8106560 2024-06-03T01:11:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9afce7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9afce7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34998/0 (socket says 172.21.15.32:34998) 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9afce7e700 1 -- 172.21.15.32:0/2817281641 learned_addr learned my addr 172.21.15.32:0/2817281641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9afce7e700 1 -- 172.21.15.32:0/2817281641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8107760 con 0x7f9af8106560 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9afce7e700 1 --2- 172.21.15.32:0/2817281641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009e80 tx=0x7f9ae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9228a0bb0a008885 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.231+0000 7f9af77fe700 1 -- 172.21.15.32:0/2817281641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ae800e070 con 0x7f9af8106560 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9af77fe700 1 -- 172.21.15.32:0/2817281641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ae8003e90 con 0x7f9af8106560 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9af77fe700 1 -- 172.21.15.32:0/2817281641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ae8004380 con 0x7f9af8106560 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2817281641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 msgr2=0x7f9af8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2817281641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009e80 tx=0x7f9ae800b330 comp rx=0 tx=0).stop 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2817281641 shutdown_connections 2024-06-03T01:11:53.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2817281641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af8106980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2817281641 >> 172.21.15.32:0/2817281641 conn(0x7f9af8101b40 msgr2=0x7f9af8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2817281641 shutdown_connections 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.232+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2817281641 wait complete. 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.233+0000 7f9aff0e2700 1 Processor -- start 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.233+0000 7f9aff0e2700 1 -- start start 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.233+0000 7f9aff0e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.233+0000 7f9aff0e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af807c170 con 0x7f9af8106560 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.234+0000 7f9afce7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.234+0000 7f9afce7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35000/0 (socket says 172.21.15.32:35000) 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.234+0000 7f9afce7e700 1 -- 172.21.15.32:0/2232385797 learned_addr learned my addr 172.21.15.32:0/2232385797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.234+0000 7f9afce7e700 1 -- 172.21.15.32:0/2232385797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009b30 con 0x7f9af8106560 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.234+0000 7f9afce7e700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d010 tx=0x7f9ae8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:53.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.235+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ae800e040 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.235+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ae8003c30 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.235+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af807a280 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.235+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ae801b790 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.235+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af807a780 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.236+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9ae8019040 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.236+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80623c0 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.236+0000 7f9af5ffb700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 0x7f9ae403a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.236+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ae804ccd0 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.236+0000 7f9af7fff700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 0x7f9ae403a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.237+0000 7f9af7fff700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 0x7f9ae403a7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006fd0 tx=0x7f9ae0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.241+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9ae8011980 con 0x7f9af8106560 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.453+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f9af8102f90 con 0x7f9ae40382e0 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.457+0000 7f9af5ffb700 1 -- 172.21.15.32:0/2232385797 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9af8102f90 con 0x7f9ae40382e0 2024-06-03T01:11:53.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 msgr2=0x7f9ae403a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 0x7f9ae403a7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006fd0 tx=0x7f9ae0006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 msgr2=0x7f9af807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d010 tx=0x7f9ae8004430 comp rx=0 tx=0).stop 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 shutdown_connections 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f9ae40382e0 0x7f9ae403a7a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 --2- 172.21.15.32:0/2232385797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9af8106560 0x7f9af807bc30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 >> 172.21.15.32:0/2232385797 conn(0x7f9af8101b40 msgr2=0x7f9af8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 shutdown_connections 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:52.459+0000 7f9aff0e2700 1 -- 172.21.15.32:0/2232385797 wait complete. 2024-06-03T01:11:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying mgr service with default placement... 2024-06-03T01:11:54.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:54 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:54.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:54 smithi032 ceph-mon[29360]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:54.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:54 smithi032 ceph-mon[29360]: Saving service mgr spec with placement count:2 2024-06-03T01:11:54.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:54 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:55.461 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-03T01:11:55.461 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.217+0000 7f141c25b700 1 Processor -- start 2024-06-03T01:11:55.461 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.217+0000 7f141c25b700 1 -- start start 2024-06-03T01:11:55.461 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f141c25b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f141c25b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414106f30 con 0x7f1414106540 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1419ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1419ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35008/0 (socket says 172.21.15.32:35008) 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1419ff7700 1 -- 172.21.15.32:0/592866948 learned_addr learned my addr 172.21.15.32:0/592866948 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1419ff7700 1 -- 172.21.15.32:0/592866948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414107740 con 0x7f1414106540 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1419ff7700 1 --2- 172.21.15.32:0/592866948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14100098a0 tx=0x7f1410009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd93573332395f09 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1418ff5700 1 -- 172.21.15.32:0/592866948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1410004180 con 0x7f1414106540 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1418ff5700 1 -- 172.21.15.32:0/592866948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14100042e0 con 0x7f1414106540 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.218+0000 7f1418ff5700 1 -- 172.21.15.32:0/592866948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1410011af0 con 0x7f1414106540 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 -- 172.21.15.32:0/592866948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 msgr2=0x7f1414106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:55.462 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 --2- 172.21.15.32:0/592866948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14100098a0 tx=0x7f1410009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 -- 172.21.15.32:0/592866948 shutdown_connections 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 --2- 172.21.15.32:0/592866948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f1414106960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 -- 172.21.15.32:0/592866948 >> 172.21.15.32:0/592866948 conn(0x7f1414101b20 msgr2=0x7f1414103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 -- 172.21.15.32:0/592866948 shutdown_connections 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.219+0000 7f141c25b700 1 -- 172.21.15.32:0/592866948 wait complete. 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f141c25b700 1 Processor -- start 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f141c25b700 1 -- start start 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f141c25b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f141c25b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141419a730 con 0x7f1414106540 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f1419ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f1419ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35012/0 (socket says 172.21.15.32:35012) 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.220+0000 7f1419ff7700 1 -- 172.21.15.32:0/2089692198 learned_addr learned my addr 172.21.15.32:0/2089692198 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f1419ff7700 1 -- 172.21.15.32:0/2089692198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009550 con 0x7f1414106540 2024-06-03T01:11:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f1419ff7700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1410006ba0 tx=0x7f1410011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1410003b00 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f141001a8e0 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1410019990 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141419a930 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.221+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141419add0 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.222+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1410019c00 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.222+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14140623c0 con 0x7f1414106540 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.222+0000 7f140affd700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 0x7f140003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.222+0000 7f14197f6700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 0x7f140003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.223+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f141004c5a0 con 0x7f1414106540 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.223+0000 7f14197f6700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 0x7f140003a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.228+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f141002b930 con 0x7f1414106540 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.446+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f1414102f90 con 0x7f1400038320 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.450+0000 7f140affd700 1 -- 172.21.15.32:0/2089692198 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1414102f90 con 0x7f1400038320 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 msgr2=0x7f140003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 0x7f140003a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 msgr2=0x7f141419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1410006ba0 tx=0x7f1410011b50 comp rx=0 tx=0).stop 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 shutdown_connections 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f1400038320 0x7f140003a7e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 --2- 172.21.15.32:0/2089692198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414106540 0x7f141419a1f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:55.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 >> 172.21.15.32:0/2089692198 conn(0x7f1414101b20 msgr2=0x7f1414102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:55.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 shutdown_connections 2024-06-03T01:11:55.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:54.452+0000 7f141c25b700 1 -- 172.21.15.32:0/2089692198 wait complete. 2024-06-03T01:11:55.466 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying crash service with default placement... 2024-06-03T01:11:56.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:56 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:56.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:56 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ef12700 1 Processor -- start 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ef12700 1 -- start start 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ef12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ef12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077807a8d0 con 0x7f077807bea0 2024-06-03T01:11:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ccae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ccae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35028/0 (socket says 172.21.15.32:35028) 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ccae700 1 -- 172.21.15.32:0/3888477161 learned_addr learned my addr 172.21.15.32:0/3888477161 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.860+0000 7f077ccae700 1 -- 172.21.15.32:0/3888477161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077807aa10 con 0x7f077807bea0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ccae700 1 --2- 172.21.15.32:0/3888477161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0768009e80 tx=0x7f076800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96635d0651559d2f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f07777fe700 1 -- 172.21.15.32:0/3888477161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f076800e070 con 0x7f077807bea0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f07777fe700 1 -- 172.21.15.32:0/3888477161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f076800b8c0 con 0x7f077807bea0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f07777fe700 1 -- 172.21.15.32:0/3888477161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07680044d0 con 0x7f077807bea0 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 -- 172.21.15.32:0/3888477161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 msgr2=0x7f077807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 --2- 172.21.15.32:0/3888477161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0768009e80 tx=0x7f076800b330 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 -- 172.21.15.32:0/3888477161 shutdown_connections 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 --2- 172.21.15.32:0/3888477161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077807a300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 -- 172.21.15.32:0/3888477161 >> 172.21.15.32:0/3888477161 conn(0x7f0778101b40 msgr2=0x7f0778103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 -- 172.21.15.32:0/3888477161 shutdown_connections 2024-06-03T01:11:56.652 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.861+0000 7f077ef12700 1 -- 172.21.15.32:0/3888477161 wait complete. 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ef12700 1 Processor -- start 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ef12700 1 -- start start 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ef12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ef12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819c8e0 con 0x7f077807bea0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ccae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ccae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35040/0 (socket says 172.21.15.32:35040) 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.862+0000 7f077ccae700 1 -- 172.21.15.32:0/1613518850 learned_addr learned my addr 172.21.15.32:0/1613518850 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f077ccae700 1 -- 172.21.15.32:0/1613518850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009b30 con 0x7f077807bea0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f077ccae700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07680072a0 tx=0x7f07680049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f076800e040 con 0x7f077807bea0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f076800bd80 con 0x7f077807bea0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077819cae0 con 0x7f077807bea0 2024-06-03T01:11:56.653 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07680126c0 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.863+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077819cfe0 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.864+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0768019040 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.864+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077810c900 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.864+0000 7f0775ffb700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 0x7f076403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.864+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f076804bce0 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.865+0000 7f0777fff700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 0x7f076403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.865+0000 7f0777fff700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 0x7f076403a7f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0760006fd0 tx=0x7f0760006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:55.869+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f076800f250 con 0x7f077807bea0 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.082+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f0778102f60 con 0x7f0764038330 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.089+0000 7f0775ffb700 1 -- 172.21.15.32:0/1613518850 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f0778102f60 con 0x7f0764038330 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 msgr2=0x7f076403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.654 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 0x7f076403a7f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0760006fd0 tx=0x7f0760006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 msgr2=0x7f077819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07680072a0 tx=0x7f07680049f0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 shutdown_connections 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f0764038330 0x7f076403a7f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 --2- 172.21.15.32:0/1613518850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f077807bea0 0x7f077819c3a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 >> 172.21.15.32:0/1613518850 conn(0x7f0778101b40 msgr2=0x7f0778102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 shutdown_connections 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:56.091+0000 7f077ef12700 1 -- 172.21.15.32:0/1613518850 wait complete. 2024-06-03T01:11:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying ceph-exporter service with default placement... 2024-06-03T01:11:57.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:57 smithi032 ceph-mon[29360]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:57.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:57 smithi032 ceph-mon[29360]: Saving service crash spec with placement * 2024-06-03T01:11:57.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:57 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.047+0000 7efc5e6f3700 1 Processor -- start 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.047+0000 7efc5e6f3700 1 -- start start 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc5e6f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc5e6f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc58108580 con 0x7efc58107b90 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53574/0 (socket says 172.21.15.32:53574) 2024-06-03T01:11:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc57fff700 1 -- 172.21.15.32:0/278302480 learned_addr learned my addr 172.21.15.32:0/278302480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:57.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc57fff700 1 -- 172.21.15.32:0/278302480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc581086c0 con 0x7efc58107b90 2024-06-03T01:11:57.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc57fff700 1 --2- 172.21.15.32:0/278302480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc480098a0 tx=0x7efc48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa105d15a860fd00 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:57.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc56ffd700 1 -- 172.21.15.32:0/278302480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc48004180 con 0x7efc58107b90 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.048+0000 7efc56ffd700 1 -- 172.21.15.32:0/278302480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc480042e0 con 0x7efc58107b90 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc56ffd700 1 -- 172.21.15.32:0/278302480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc48011af0 con 0x7efc58107b90 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 -- 172.21.15.32:0/278302480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 msgr2=0x7efc58107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/278302480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc480098a0 tx=0x7efc48009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 -- 172.21.15.32:0/278302480 shutdown_connections 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/278302480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 -- 172.21.15.32:0/278302480 >> 172.21.15.32:0/278302480 conn(0x7efc58100190 msgr2=0x7efc581025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 -- 172.21.15.32:0/278302480 shutdown_connections 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.049+0000 7efc5e6f3700 1 -- 172.21.15.32:0/278302480 wait complete. 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc5e6f3700 1 Processor -- start 2024-06-03T01:11:57.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc5e6f3700 1 -- start start 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc5e6f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc5e6f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581077d0 con 0x7efc58107b90 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53588/0 (socket says 172.21.15.32:53588) 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.050+0000 7efc57fff700 1 -- 172.21.15.32:0/3295195241 learned_addr learned my addr 172.21.15.32:0/3295195241 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc57fff700 1 -- 172.21.15.32:0/3295195241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48009550 con 0x7efc58107b90 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc57fff700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc48006ba0 tx=0x7efc48011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc48003b00 con 0x7efc58107b90 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc581058e0 con 0x7efc58107b90 2024-06-03T01:11:57.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc4801a8e0 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc48019990 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.051+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc58105d80 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.052+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7efc48019c00 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.052+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc580623c0 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.053+0000 7efc557fa700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 0x7efc4403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.053+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc4804c590 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.053+0000 7efc577fe700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 0x7efc4403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.053+0000 7efc577fe700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 0x7efc4403a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.058+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efc4802d440 con 0x7efc58107b90 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.275+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7efc58101640 con 0x7efc44038320 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.280+0000 7efc557fa700 1 -- 172.21.15.32:0/3295195241 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7efc58101640 con 0x7efc44038320 2024-06-03T01:11:57.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 msgr2=0x7efc4403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 0x7efc4403a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 msgr2=0x7efc58107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc48006ba0 tx=0x7efc48011b50 comp rx=0 tx=0).stop 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 shutdown_connections 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7efc44038320 0x7efc4403a7e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 --2- 172.21.15.32:0/3295195241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc58107b90 0x7efc58107290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 >> 172.21.15.32:0/3295195241 conn(0x7efc58100190 msgr2=0x7efc58100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 shutdown_connections 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:57.282+0000 7efc5e6f3700 1 -- 172.21.15.32:0/3295195241 wait complete. 2024-06-03T01:11:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying prometheus service with default placement... 2024-06-03T01:11:58.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:58 smithi032 ceph-mon[29360]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:11:58.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:58 smithi032 ceph-mon[29360]: Saving service ceph-exporter spec with placement * 2024-06-03T01:11:58.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:11:58 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.124+0000 7fd1f0c7f700 1 Processor -- start 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.124+0000 7fd1f0c7f700 1 -- start start 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1f0c7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1f0c7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec106f50 con 0x7fd1ec106560 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53594/0 (socket says 172.21.15.32:53594) 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1ea59c700 1 -- 172.21.15.32:0/1458327036 learned_addr learned my addr 172.21.15.32:0/1458327036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.125+0000 7fd1ea59c700 1 -- 172.21.15.32:0/1458327036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec107760 con 0x7fd1ec106560 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1ea59c700 1 --2- 172.21.15.32:0/1458327036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098a0 tx=0x7fd1dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df893d92d8317b46 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1e959a700 1 -- 172.21.15.32:0/1458327036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc004180 con 0x7fd1ec106560 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1e959a700 1 -- 172.21.15.32:0/1458327036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc0042e0 con 0x7fd1ec106560 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1e959a700 1 -- 172.21.15.32:0/1458327036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc011af0 con 0x7fd1ec106560 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1458327036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 msgr2=0x7fd1ec106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1458327036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098a0 tx=0x7fd1dc009bb0 comp rx=0 tx=0).stop 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1458327036 shutdown_connections 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1458327036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec106980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1458327036 >> 172.21.15.32:0/1458327036 conn(0x7fd1ec101b40 msgr2=0x7fd1ec103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1458327036 shutdown_connections 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.126+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1458327036 wait complete. 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.127+0000 7fd1f0c7f700 1 Processor -- start 2024-06-03T01:11:59.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.127+0000 7fd1f0c7f700 1 -- start start 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.127+0000 7fd1f0c7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.127+0000 7fd1f0c7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec07c170 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53610/0 (socket says 172.21.15.32:53610) 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1ea59c700 1 -- 172.21.15.32:0/1039009265 learned_addr learned my addr 172.21.15.32:0/1039009265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1ea59c700 1 -- 172.21.15.32:0/1039009265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009550 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1ea59c700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc011670 tx=0x7fd1dc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc0039f0 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc003b50 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc0247c0 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec07a280 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.128+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec07a720 con 0x7fd1ec106560 2024-06-03T01:11:59.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.129+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd1dc01a430 con 0x7fd1ec106560 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.129+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec0623c0 con 0x7fd1ec106560 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.129+0000 7fd1e37fe700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 0x7fd1d803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.130+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd1dc04c360 con 0x7fd1ec106560 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.130+0000 7fd1e9d9b700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 0x7fd1d803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.130+0000 7fd1e9d9b700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 0x7fd1d803a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.134+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1dc018b40 con 0x7fd1ec106560 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.345+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1ec10a960 con 0x7fd1d8038370 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.350+0000 7fd1e37fe700 1 -- 172.21.15.32:0/1039009265 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fd1ec10a960 con 0x7fd1d8038370 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 msgr2=0x7fd1d803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 0x7fd1d803a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4006e40 comp rx=0 tx=0).stop 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 msgr2=0x7fd1ec07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:59.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc011670 tx=0x7fd1dc011b50 comp rx=0 tx=0).stop 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 shutdown_connections 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fd1d8038370 0x7fd1d803a830 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 --2- 172.21.15.32:0/1039009265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec106560 0x7fd1ec07bc30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 >> 172.21.15.32:0/1039009265 conn(0x7fd1ec101b40 msgr2=0x7fd1ec102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 shutdown_connections 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:11:59.352+0000 7fd1f0c7f700 1 -- 172.21.15.32:0/1039009265 wait complete. 2024-06-03T01:11:59.991 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying grafana service with default placement... 2024-06-03T01:12:00.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:00 smithi032 ceph-mon[29360]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:00.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:00 smithi032 ceph-mon[29360]: Saving service prometheus spec with placement count:1 2024-06-03T01:12:00.641 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:00 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.423+0000 7f8c08e3c700 1 Processor -- start 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.423+0000 7f8c08e3c700 1 -- start start 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c08e3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c08e3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04106f70 con 0x7f8c04106580 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53612/0 (socket says 172.21.15.32:53612) 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c0259c700 1 -- 172.21.15.32:0/2011616664 learned_addr learned my addr 172.21.15.32:0/2011616664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c0259c700 1 -- 172.21.15.32:0/2011616664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04107780 con 0x7f8c04106580 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.424+0000 7f8c0259c700 1 --2- 172.21.15.32:0/2011616664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0098a0 tx=0x7f8bec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1336543d5ad027a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c0159a700 1 -- 172.21.15.32:0/2011616664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bec004180 con 0x7f8c04106580 2024-06-03T01:12:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c0159a700 1 -- 172.21.15.32:0/2011616664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bec0042e0 con 0x7f8c04106580 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c0159a700 1 -- 172.21.15.32:0/2011616664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bec011af0 con 0x7f8c04106580 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2011616664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 msgr2=0x7f8c041069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2011616664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0098a0 tx=0x7f8bec009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2011616664 shutdown_connections 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2011616664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c041069a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2011616664 >> 172.21.15.32:0/2011616664 conn(0x7f8c04101b00 msgr2=0x7f8c04103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2011616664 shutdown_connections 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.425+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2011616664 wait complete. 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.426+0000 7f8c08e3c700 1 Processor -- start 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.426+0000 7f8c08e3c700 1 -- start start 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.426+0000 7f8c08e3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.426+0000 7f8c08e3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419a7d0 con 0x7f8c04106580 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.426+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:01.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53620/0 (socket says 172.21.15.32:53620) 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c0259c700 1 -- 172.21.15.32:0/2053302302 learned_addr learned my addr 172.21.15.32:0/2053302302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c0259c700 1 -- 172.21.15.32:0/2053302302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec009550 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c0259c700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011670 tx=0x7f8bec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bec0039f0 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bec003b50 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0419a9d0 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bec0247c0 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.427+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0419ae70 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.428+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f8bec02d3f0 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.428+0000 7f8bfb7fe700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 0x7f8bf003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.428+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8bec04c3c0 con 0x7f8c04106580 2024-06-03T01:12:01.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.428+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c040623c0 con 0x7f8c04106580 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.428+0000 7f8c01d9b700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 0x7f8bf003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.429+0000 7f8c01d9b700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 0x7f8bf003a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4006fd0 tx=0x7f8bf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.433+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8bec02d6f0 con 0x7f8c04106580 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.640+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c04103e00 con 0x7f8bf0038320 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.645+0000 7f8bfb7fe700 1 -- 172.21.15.32:0/2053302302 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f8c04103e00 con 0x7f8bf0038320 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 msgr2=0x7f8bf003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 0x7f8bf003a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4006fd0 tx=0x7f8bf4006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 msgr2=0x7f8c0419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011670 tx=0x7f8bec011b50 comp rx=0 tx=0).stop 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 shutdown_connections 2024-06-03T01:12:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f8bf0038320 0x7f8bf003a7e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 --2- 172.21.15.32:0/2053302302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04106580 0x7f8c0419a290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 >> 172.21.15.32:0/2053302302 conn(0x7f8c04101b00 msgr2=0x7f8c041031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 shutdown_connections 2024-06-03T01:12:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:00.647+0000 7f8c08e3c700 1 -- 172.21.15.32:0/2053302302 wait complete. 2024-06-03T01:12:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying node-exporter service with default placement... 2024-06-03T01:12:01.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:01 smithi032 ceph-mon[29360]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:01.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:01 smithi032 ceph-mon[29360]: Saving service grafana spec with placement count:1 2024-06-03T01:12:01.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:01 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-06-03T01:12:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.619+0000 7f785cfd4700 1 Processor -- start 2024-06-03T01:12:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.619+0000 7f785cfd4700 1 -- start start 2024-06-03T01:12:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785cfd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785cfd4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858074640 con 0x7f7858107020 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53630/0 (socket says 172.21.15.32:53630) 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785659c700 1 -- 172.21.15.32:0/1442362354 learned_addr learned my addr 172.21.15.32:0/1442362354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785659c700 1 -- 172.21.15.32:0/1442362354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858109950 con 0x7f7858107020 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.620+0000 7f785659c700 1 --2- 172.21.15.32:0/1442362354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7848009e80 tx=0x7f784800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd42e0cbee6134e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785559a700 1 -- 172.21.15.32:0/1442362354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f784800e070 con 0x7f7858107020 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785559a700 1 -- 172.21.15.32:0/1442362354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f784800b8c0 con 0x7f7858107020 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785559a700 1 -- 172.21.15.32:0/1442362354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78480044d0 con 0x7f7858107020 2024-06-03T01:12:02.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 -- 172.21.15.32:0/1442362354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 msgr2=0x7f7858109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1442362354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7848009e80 tx=0x7f784800b330 comp rx=0 tx=0).stop 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 -- 172.21.15.32:0/1442362354 shutdown_connections 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1442362354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858109410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 -- 172.21.15.32:0/1442362354 >> 172.21.15.32:0/1442362354 conn(0x7f78581009f0 msgr2=0x7f7858102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 -- 172.21.15.32:0/1442362354 shutdown_connections 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.621+0000 7f785cfd4700 1 -- 172.21.15.32:0/1442362354 wait complete. 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.622+0000 7f785cfd4700 1 Processor -- start 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.622+0000 7f785cfd4700 1 -- start start 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.622+0000 7f785cfd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.622+0000 7f785cfd4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858106ec0 con 0x7f7858107020 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.622+0000 7f785659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53646/0 (socket says 172.21.15.32:53646) 2024-06-03T01:12:02.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785659c700 1 -- 172.21.15.32:0/1465710516 learned_addr learned my addr 172.21.15.32:0/1465710516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785659c700 1 -- 172.21.15.32:0/1465710516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009b30 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785659c700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7848004070 tx=0x7f78480049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f784800e040 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f784800be00 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7848012770 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7858104fd0 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.623+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78581054d0 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.624+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7848019040 con 0x7f7858107020 2024-06-03T01:12:02.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.624+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78580623c0 con 0x7f7858107020 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.624+0000 7f784f7fe700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 0x7f784403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.624+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f784804bcc0 con 0x7f7858107020 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.625+0000 7f7855d9b700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 0x7f784403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.625+0000 7f7855d9b700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 0x7f784403a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7840006fd0 tx=0x7f7840006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.629+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f784801a570 con 0x7f7858107020 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.846+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f7858101e30 con 0x7f7844038370 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.851+0000 7f784f7fe700 1 -- 172.21.15.32:0/1465710516 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f7858101e30 con 0x7f7844038370 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.852+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 msgr2=0x7f784403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.852+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 0x7f784403a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7840006fd0 tx=0x7f7840006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.852+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 msgr2=0x7f7858106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.852+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7848004070 tx=0x7f78480049f0 comp rx=0 tx=0).stop 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 shutdown_connections 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f7844038370 0x7f784403a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 --2- 172.21.15.32:0/1465710516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7858107020 0x7f7858106980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 >> 172.21.15.32:0/1465710516 conn(0x7f78581009f0 msgr2=0x7f7858101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 shutdown_connections 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:01.853+0000 7f785cfd4700 1 -- 172.21.15.32:0/1465710516 wait complete. 2024-06-03T01:12:02.480 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying alertmanager service with default placement... 2024-06-03T01:12:02.811 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:02 smithi032 ceph-mon[29360]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:02.811 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:02 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:03.732 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:03 smithi032 ceph-mon[29360]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:03.732 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:03 smithi032 ceph-mon[29360]: Saving service node-exporter spec with placement * 2024-06-03T01:12:03.732 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:03 smithi032 ceph-mon[29360]: from='mgr.14120 172.21.15.32:0/1829281398' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-06-03T01:12:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.925+0000 7fecdb42d700 1 Processor -- start 2024-06-03T01:12:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.925+0000 7fecdb42d700 1 -- start start 2024-06-03T01:12:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.925+0000 7fecdb42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.925+0000 7fecdb42d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4079e30 con 0x7fecd4107030 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7fecd91c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7fecd91c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53660/0 (socket says 172.21.15.32:53660) 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7fecd91c9700 1 -- 172.21.15.32:0/33850309 learned_addr learned my addr 172.21.15.32:0/33850309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7fecd91c9700 1 -- 172.21.15.32:0/33850309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4109960 con 0x7fecd4107030 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7fecd91c9700 1 --2- 172.21.15.32:0/33850309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd00098a0 tx=0x7fecd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9965c46f6dc4e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7feccbfff700 1 -- 172.21.15.32:0/33850309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd0004180 con 0x7fecd4107030 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7feccbfff700 1 -- 172.21.15.32:0/33850309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fecd00042e0 con 0x7fecd4107030 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.926+0000 7feccbfff700 1 -- 172.21.15.32:0/33850309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd0011af0 con 0x7fecd4107030 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 -- 172.21.15.32:0/33850309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 msgr2=0x7fecd4109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 --2- 172.21.15.32:0/33850309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd00098a0 tx=0x7fecd0009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 -- 172.21.15.32:0/33850309 shutdown_connections 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 --2- 172.21.15.32:0/33850309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4109420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 -- 172.21.15.32:0/33850309 >> 172.21.15.32:0/33850309 conn(0x7fecd4100a20 msgr2=0x7fecd4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 -- 172.21.15.32:0/33850309 shutdown_connections 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.927+0000 7fecdb42d700 1 -- 172.21.15.32:0/33850309 wait complete. 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecdb42d700 1 Processor -- start 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecdb42d700 1 -- start start 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecdb42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecdb42d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4106ed0 con 0x7fecd4107030 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecd91c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecd91c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53670/0 (socket says 172.21.15.32:53670) 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecd91c9700 1 -- 172.21.15.32:0/1751224057 learned_addr learned my addr 172.21.15.32:0/1751224057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecd91c9700 1 -- 172.21.15.32:0/1751224057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009550 con 0x7fecd4107030 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.928+0000 7fecd91c9700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fecd0011670 tx=0x7fecd0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd0003a40 con 0x7fecd4107030 2024-06-03T01:12:03.741 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fecd0003ba0 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd4104fe0 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd0024820 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd4105480 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.929+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fecd001a430 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.930+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd4062430 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.930+0000 7fecca7fc700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 0x7fecc003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.930+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fecd004c310 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.930+0000 7fecd89c8700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 0x7fecc003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.930+0000 7fecd89c8700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 0x7fecc003a7a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006fd0 tx=0x7fecc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:02.935+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fecd4062430 con 0x7fecd4107030 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.149+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fecd40623c0 con 0x7fecc00382e0 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.154+0000 7fecca7fc700 1 -- 172.21.15.32:0/1751224057 <== mgr.14120 v2:172.21.15.32:6800/3147998267 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fecd40623c0 con 0x7fecc00382e0 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 msgr2=0x7fecc003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 0x7fecc003a7a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006fd0 tx=0x7fecc4006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 msgr2=0x7fecd4106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fecd0011670 tx=0x7fecd0011aa0 comp rx=0 tx=0).stop 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 shutdown_connections 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fecc00382e0 0x7fecc003a7a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 --2- 172.21.15.32:0/1751224057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4107030 0x7fecd4106990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 >> 172.21.15.32:0/1751224057 conn(0x7fecd4100a20 msgr2=0x7fecd41017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 shutdown_connections 2024-06-03T01:12:03.743 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:03.156+0000 7fecdb42d700 1 -- 172.21.15.32:0/1751224057 wait complete. 2024-06-03T01:12:04.930 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:04 smithi032 ceph-mon[29360]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:04.930 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:04 smithi032 ceph-mon[29360]: Saving service alertmanager spec with placement count:1 2024-06-03T01:12:04.930 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:04 smithi032 ceph-mon[29360]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:04.931 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:04 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1566436940' entity='client.admin' 2024-06-03T01:12:04.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.161+0000 7f5da679e700 1 Processor -- start 2024-06-03T01:12:04.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.161+0000 7f5da679e700 1 -- start start 2024-06-03T01:12:04.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5da679e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:04.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5da679e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0105e40 con 0x7f5da0105450 2024-06-03T01:12:04.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53682/0 (socket says 172.21.15.32:53682) 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9ffff700 1 -- 172.21.15.32:0/827218066 learned_addr learned my addr 172.21.15.32:0/827218066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9ffff700 1 -- 172.21.15.32:0/827218066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0106650 con 0x7f5da0105450 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9ffff700 1 --2- 172.21.15.32:0/827218066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000bb40 tx=0x7f5d9000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda5675c95b799ab server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9effd700 1 -- 172.21.15.32:0/827218066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d9000dc00 con 0x7f5da0105450 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9effd700 1 -- 172.21.15.32:0/827218066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d900044e0 con 0x7f5da0105450 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.162+0000 7f5d9effd700 1 -- 172.21.15.32:0/827218066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d90010470 con 0x7f5da0105450 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 -- 172.21.15.32:0/827218066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 msgr2=0x7f5da0105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 --2- 172.21.15.32:0/827218066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000bb40 tx=0x7f5d9000be50 comp rx=0 tx=0).stop 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 -- 172.21.15.32:0/827218066 shutdown_connections 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 --2- 172.21.15.32:0/827218066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0105870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 -- 172.21.15.32:0/827218066 >> 172.21.15.32:0/827218066 conn(0x7f5da01009b0 msgr2=0x7f5da0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:04.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 -- 172.21.15.32:0/827218066 shutdown_connections 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.163+0000 7f5da679e700 1 -- 172.21.15.32:0/827218066 wait complete. 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5da679e700 1 Processor -- start 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5da679e700 1 -- start start 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5da679e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5da679e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0198590 con 0x7f5da0105450 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5d9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5d9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53684/0 (socket says 172.21.15.32:53684) 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.164+0000 7f5d9ffff700 1 -- 172.21.15.32:0/1566436940 learned_addr learned my addr 172.21.15.32:0/1566436940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5d9ffff700 1 -- 172.21.15.32:0/1566436940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9000b7f0 con 0x7f5da0105450 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5d9ffff700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b340 tx=0x7f5d90004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d9001e070 con 0x7f5da0105450 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da0198790 con 0x7f5da0105450 2024-06-03T01:12:04.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d9000de20 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d90022a00 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.165+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0198c30 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.166+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5d90010730 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.166+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da0191ce0 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.166+0000 7f5d9d7fa700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 0x7f5d8803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.166+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d9004e5f0 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.166+0000 7f5d9f7fe700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 0x7f5d8803a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.167+0000 7f5d9f7fe700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 0x7f5d8803a790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.171+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d90021ba0 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.368+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f5da002d090 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.370+0000 7f5d9d7fa700 1 -- 172.21.15.32:0/1566436940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f5d90027030 con 0x7f5da0105450 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 msgr2=0x7f5d8803a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 0x7f5d8803a790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 msgr2=0x7f5da0198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b340 tx=0x7f5d90004070 comp rx=0 tx=0).stop 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 shutdown_connections 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f5d880382d0 0x7f5d8803a790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 --2- 172.21.15.32:0/1566436940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da0105450 0x7f5da0198050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 >> 172.21.15.32:0/1566436940 conn(0x7f5da01009b0 msgr2=0x7f5da018ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 shutdown_connections 2024-06-03T01:12:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:04.373+0000 7f5da679e700 1 -- 172.21.15.32:0/1566436940 wait complete. 2024-06-03T01:12:06.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.374+0000 7f21c1dca700 1 Processor -- start 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.374+0000 7f21c1dca700 1 -- start start 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21c1dca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21c1dca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc106f50 con 0x7f21bc106560 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21bb7fe700 1 -- 172.21.15.32:0/354122737 learned_addr learned my addr 172.21.15.32:0/354122737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21bb7fe700 1 -- 172.21.15.32:0/354122737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc107760 con 0x7f21bc106560 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21bb7fe700 1 --2- 172.21.15.32:0/354122737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b37618c351576e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:06.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.375+0000 7f21ba7fc700 1 -- 172.21.15.32:0/354122737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21ac004180 con 0x7f21bc106560 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21ba7fc700 1 -- 172.21.15.32:0/354122737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21ac0042e0 con 0x7f21bc106560 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21ba7fc700 1 -- 172.21.15.32:0/354122737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21ac011af0 con 0x7f21bc106560 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 -- 172.21.15.32:0/354122737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 msgr2=0x7f21bc106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 --2- 172.21.15.32:0/354122737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 -- 172.21.15.32:0/354122737 shutdown_connections 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 --2- 172.21.15.32:0/354122737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc106980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 -- 172.21.15.32:0/354122737 >> 172.21.15.32:0/354122737 conn(0x7f21bc101b40 msgr2=0x7f21bc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 -- 172.21.15.32:0/354122737 shutdown_connections 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.376+0000 7f21c1dca700 1 -- 172.21.15.32:0/354122737 wait complete. 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.377+0000 7f21c1dca700 1 Processor -- start 2024-06-03T01:12:06.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.377+0000 7f21c1dca700 1 -- start start 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.377+0000 7f21c1dca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.377+0000 7f21c1dca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19a730 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.377+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53712/0 (socket says 172.21.15.32:53712) 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21bb7fe700 1 -- 172.21.15.32:0/1798699745 learned_addr learned my addr 172.21.15.32:0/1798699745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21bb7fe700 1 -- 172.21.15.32:0/1798699745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac009550 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21bb7fe700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21ac003a70 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc19a930 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21ac003bd0 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc19add0 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.378+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21ac0238c0 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.379+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f21ac023b30 con 0x7f21bc106560 2024-06-03T01:12:06.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.379+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc0623c0 con 0x7f21bc106560 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.379+0000 7f21b8ff9700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 0x7f21a803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.380+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f21ac04c180 con 0x7f21bc106560 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.380+0000 7f21baffd700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 0x7f21a803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.380+0000 7f21baffd700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 0x7f21a803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.384+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f21ac010c20 con 0x7f21bc106560 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.583+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f21bc19dc10 con 0x7f21bc106560 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.587+0000 7f21b8ff9700 1 -- 172.21.15.32:0/1798699745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f21ac010830 con 0x7f21bc106560 2024-06-03T01:12:06.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 msgr2=0x7f21a803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 0x7f21a803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 msgr2=0x7f21bc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac011b50 comp rx=0 tx=0).stop 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 shutdown_connections 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f21a8038370 0x7f21a803a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 --2- 172.21.15.32:0/1798699745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc106560 0x7f21bc19a1f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 >> 172.21.15.32:0/1798699745 conn(0x7f21bc101b40 msgr2=0x7f21bc102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 shutdown_connections 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:05.589+0000 7f21c1dca700 1 -- 172.21.15.32:0/1798699745 wait complete. 2024-06-03T01:12:06.325 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2024-06-03T01:12:07.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:06 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1798699745' entity='client.admin' 2024-06-03T01:12:07.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:06 smithi032 ceph-mon[29360]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:07.910 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:07 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3393980743' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.778+0000 7fb76dfc4700 1 Processor -- start 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.778+0000 7fb76dfc4700 1 -- start start 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb76dfc4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb76dfc4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768074640 con 0x7fb768107020 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53724/0 (socket says 172.21.15.32:53724) 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb7677fe700 1 -- 172.21.15.32:0/2360597911 learned_addr learned my addr 172.21.15.32:0/2360597911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:08.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb7677fe700 1 -- 172.21.15.32:0/2360597911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768109950 con 0x7fb768107020 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb7677fe700 1 --2- 172.21.15.32:0/2360597911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098a0 tx=0x7fb758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7284addaff54585 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb766ffd700 1 -- 172.21.15.32:0/2360597911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb758004180 con 0x7fb768107020 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb766ffd700 1 -- 172.21.15.32:0/2360597911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb7580042e0 con 0x7fb768107020 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.779+0000 7fb766ffd700 1 -- 172.21.15.32:0/2360597911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb758011af0 con 0x7fb768107020 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 -- 172.21.15.32:0/2360597911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 msgr2=0x7fb768109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/2360597911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098a0 tx=0x7fb758009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 -- 172.21.15.32:0/2360597911 shutdown_connections 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/2360597911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb768109410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 -- 172.21.15.32:0/2360597911 >> 172.21.15.32:0/2360597911 conn(0x7fb7681009f0 msgr2=0x7fb768102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 -- 172.21.15.32:0/2360597911 shutdown_connections 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.780+0000 7fb76dfc4700 1 -- 172.21.15.32:0/2360597911 wait complete. 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb76dfc4700 1 Processor -- start 2024-06-03T01:12:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb76dfc4700 1 -- start start 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb76dfc4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb76dfc4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76819a6c0 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53736/0 (socket says 172.21.15.32:53736) 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb7677fe700 1 -- 172.21.15.32:0/3393980743 learned_addr learned my addr 172.21.15.32:0/3393980743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.781+0000 7fb7677fe700 1 -- 172.21.15.32:0/3393980743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009550 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb7677fe700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb758011ad0 tx=0x7fb758011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb758003ab0 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb758003c10 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76819a8c0 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7580198e0 con 0x7fb768107020 2024-06-03T01:12:08.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.782+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76819ad60 con 0x7fb768107020 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb758019b50 con 0x7fb768107020 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb7657fa700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 0x7fb75403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb75804c290 con 0x7fb768107020 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7680623c0 con 0x7fb768107020 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb75effd700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 0x7fb75403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.783+0000 7fb75effd700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 0x7fb75403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:06.788+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb758010da0 con 0x7fb768107020 2024-06-03T01:12:08.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.041+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb76819ddd0 con 0x7fb768107020 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.592+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fb75800b9a0 con 0x7fb768107020 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.594+0000 7fb7657fa700 1 -- 172.21.15.32:0/3393980743 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb758020d90 con 0x7fb768107020 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 msgr2=0x7fb75403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 0x7fb75403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 msgr2=0x7fb76819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb758011ad0 tx=0x7fb758011bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 shutdown_connections 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7fb754038370 0x7fb75403a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 --2- 172.21.15.32:0/3393980743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768107020 0x7fb76819a180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:08.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 >> 172.21.15.32:0/3393980743 conn(0x7fb7681009f0 msgr2=0x7fb768101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:08.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 shutdown_connections 2024-06-03T01:12:08.183 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:07.597+0000 7fb76dfc4700 1 -- 172.21.15.32:0/3393980743 wait complete. 2024-06-03T01:12:08.830 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:08 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3393980743' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-03T01:12:08.830 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:08 smithi032 ceph-mon[29360]: mgrmap e9: smithi032.kfcfaa(active, since 25s) 2024-06-03T01:12:09.526 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "smithi032.kfcfaa", 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.623+0000 7f41c1f15700 1 Processor -- start 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.623+0000 7f41c1f15700 1 -- start start 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.623+0000 7f41c1f15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.623+0000 7f41c1f15700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc105e60 con 0x7f41bc105470 2024-06-03T01:12:09.527 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43296/0 (socket says 172.21.15.32:43296) 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41bb7fe700 1 -- 172.21.15.32:0/234376167 learned_addr learned my addr 172.21.15.32:0/234376167 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41bb7fe700 1 -- 172.21.15.32:0/234376167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc106670 con 0x7f41bc105470 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41bb7fe700 1 --2- 172.21.15.32:0/234376167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00bb40 tx=0x7f41ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3cc5aa80bac7d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41ba7fc700 1 -- 172.21.15.32:0/234376167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41ac00dc00 con 0x7f41bc105470 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41ba7fc700 1 -- 172.21.15.32:0/234376167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41ac0044e0 con 0x7f41bc105470 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.624+0000 7f41ba7fc700 1 -- 172.21.15.32:0/234376167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41ac010470 con 0x7f41bc105470 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 -- 172.21.15.32:0/234376167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 msgr2=0x7f41bc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 --2- 172.21.15.32:0/234376167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00bb40 tx=0x7f41ac00be50 comp rx=0 tx=0).stop 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 -- 172.21.15.32:0/234376167 shutdown_connections 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 --2- 172.21.15.32:0/234376167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc105890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 -- 172.21.15.32:0/234376167 >> 172.21.15.32:0/234376167 conn(0x7f41bc1009f0 msgr2=0x7f41bc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.528 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 -- 172.21.15.32:0/234376167 shutdown_connections 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.625+0000 7f41c1f15700 1 -- 172.21.15.32:0/234376167 wait complete. 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41c1f15700 1 Processor -- start 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41c1f15700 1 -- start start 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41c1f15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41c1f15700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc198500 con 0x7f41bc105470 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43308/0 (socket says 172.21.15.32:43308) 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41bb7fe700 1 -- 172.21.15.32:0/3012587363 learned_addr learned my addr 172.21.15.32:0/3012587363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.626+0000 7f41bb7fe700 1 -- 172.21.15.32:0/3012587363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac00b7f0 con 0x7f41bc105470 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41bb7fe700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41ac003fc0 tx=0x7f41ac0040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.529 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41ac01e070 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41ac00de20 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc198700 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41ac0229d0 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.627+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc198ba0 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f41ac0106e0 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc191d00 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41b8ff9700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 0x7f41a403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f41ac04e560 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41baffd700 1 -- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 msgr2=0x7f41a403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.628+0000 7f41baffd700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 0x7f41a403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.633+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f41ac021590 con 0x7f41bc105470 2024-06-03T01:12:09.530 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.829+0000 7f41baffd700 1 -- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 msgr2=0x7f41a403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.829+0000 7f41baffd700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 0x7f41a403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.936+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f41bc02d090 con 0x7f41bc105470 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.937+0000 7f41b8ff9700 1 -- 172.21.15.32:0/3012587363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f41ac027020 con 0x7f41bc105470 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 msgr2=0x7f41a403a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 0x7f41a403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 msgr2=0x7f41bc197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41ac003fc0 tx=0x7f41ac0040a0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 shutdown_connections 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f41a40383c0 0x7f41a403a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 --2- 172.21.15.32:0/3012587363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc105470 0x7f41bc197fc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 >> 172.21.15.32:0/3012587363 conn(0x7f41bc1009f0 msgr2=0x7f41bc18ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.939+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 shutdown_connections 2024-06-03T01:12:09.531 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:08.940+0000 7f41c1f15700 1 -- 172.21.15.32:0/3012587363 wait complete. 2024-06-03T01:12:09.532 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-06-03T01:12:09.532 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2024-06-03T01:12:09.802 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:09 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3012587363' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T01:12:15.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: Active manager daemon smithi032.kfcfaa restarted 2024-06-03T01:12:15.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: osdmap e3: 0 total, 0 up, 0 in 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: mgrmap e10: smithi032.kfcfaa(active, starting, since 0.00334059s) 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:12:15.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:14 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.906+0000 7f4afeaf0700 1 Processor -- start 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.906+0000 7f4afeaf0700 1 -- start start 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.906+0000 7f4afeaf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.906+0000 7f4afeaf0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8074680 con 0x7f4af8106fe0 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4afc88c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4afc88c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43324/0 (socket says 172.21.15.32:43324) 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4afc88c700 1 -- 172.21.15.32:0/4145886946 learned_addr learned my addr 172.21.15.32:0/4145886946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4afc88c700 1 -- 172.21.15.32:0/4145886946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8109910 con 0x7f4af8106fe0 2024-06-03T01:12:16.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4afc88c700 1 --2- 172.21.15.32:0/4145886946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098a0 tx=0x7f4ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826847730eb6d18d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4af77fe700 1 -- 172.21.15.32:0/4145886946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ae8004180 con 0x7f4af8106fe0 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4af77fe700 1 -- 172.21.15.32:0/4145886946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ae80042e0 con 0x7f4af8106fe0 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.907+0000 7f4af77fe700 1 -- 172.21.15.32:0/4145886946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ae8011af0 con 0x7f4af8106fe0 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 -- 172.21.15.32:0/4145886946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 msgr2=0x7f4af81093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/4145886946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098a0 tx=0x7f4ae8009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 -- 172.21.15.32:0/4145886946 shutdown_connections 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/4145886946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af81093d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 -- 172.21.15.32:0/4145886946 >> 172.21.15.32:0/4145886946 conn(0x7f4af81009f0 msgr2=0x7f4af8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 -- 172.21.15.32:0/4145886946 shutdown_connections 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.908+0000 7f4afeaf0700 1 -- 172.21.15.32:0/4145886946 wait complete. 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afeaf0700 1 Processor -- start 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afeaf0700 1 -- start start 2024-06-03T01:12:16.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afeaf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afeaf0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af819a6b0 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afc88c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afc88c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43330/0 (socket says 172.21.15.32:43330) 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afc88c700 1 -- 172.21.15.32:0/1131352738 learned_addr learned my addr 172.21.15.32:0/1131352738 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afc88c700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009550 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.909+0000 7f4afc88c700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8011ad0 tx=0x7f4ae8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.910+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ae8003ab0 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.910+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ae8003c10 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.910+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af819a8b0 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.910+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ae80198e0 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.910+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af819ad50 con 0x7f4af8106fe0 2024-06-03T01:12:16.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f4ae8019b50 con 0x7f4af8106fe0 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af5ffb700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae403af90 con 0x7f4ae40383c0 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ae804c150 con 0x7f4af8106fe0 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af7fff700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:09.911+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:10.112+0000 7f4af7fff700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:10.112+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:10.513+0000 7f4af7fff700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:10.513+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:11.314+0000 7f4af7fff700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:11.314+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:12.915+0000 7f4af7fff700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3147998267 2024-06-03T01:12:16.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:12.915+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:14.540+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f4ae802ad00 con 0x7f4af8106fe0 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:14.541+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 msgr2=0x7f4ae403a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:14.541+0000 7f4af5ffb700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.542+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f4ae804dca0 con 0x7f4af8106fe0 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.542+0000 7f4af5ffb700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 0x7f4ae403e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.542+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae403af90 con 0x7f4ae403be30 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.543+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 0x7f4ae403e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.543+0000 7f4af7fff700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 0x7f4ae403e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0003a10 tx=0x7f4ae0009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.544+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4ae403af90 con 0x7f4ae403be30 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.550+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4af819b4b0 con 0x7f4ae403be30 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.550+0000 7f4af5ffb700 1 -- 172.21.15.32:0/1131352738 <== mgr.14164 v2:172.21.15.32:6800/1578498944 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4af819b4b0 con 0x7f4ae403be30 2024-06-03T01:12:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 msgr2=0x7f4ae403e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 0x7f4ae403e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0003a10 tx=0x7f4ae0009270 comp rx=0 tx=0).stop 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 msgr2=0x7f4af819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8011ad0 tx=0x7f4ae8011bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 shutdown_connections 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f4ae403be30 0x7f4ae403e220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af8106fe0 0x7f4af819a170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 --2- 172.21.15.32:0/1131352738 >> [v2:172.21.15.32:6800/3147998267,v1:172.21.15.32:6801/3147998267] conn(0x7f4ae40383c0 0x7f4ae403a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 >> 172.21.15.32:0/1131352738 conn(0x7f4af81009f0 msgr2=0x7f4af8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 shutdown_connections 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:15.551+0000 7f4afeaf0700 1 -- 172.21.15.32:0/1131352738 wait complete. 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2024-06-03T01:12:16.324 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2024-06-03T01:12:16.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:16 smithi032 ceph-mon[29360]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T01:12:16.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:16 smithi032 ceph-mon[29360]: mgrmap e11: smithi032.kfcfaa(active, since 1.00574s) 2024-06-03T01:12:16.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:16 smithi032 ceph-mon[29360]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T01:12:16.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:16 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:17.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199cae0700 1 Processor -- start 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199cae0700 1 -- start start 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199cae0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199cae0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1998074620 con 0x7f1998107040 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43416/0 (socket says 172.21.15.32:43416) 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199659c700 1 -- 172.21.15.32:0/2345949923 learned_addr learned my addr 172.21.15.32:0/2345949923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.786+0000 7f199659c700 1 -- 172.21.15.32:0/2345949923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998109970 con 0x7f1998107040 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f199659c700 1 --2- 172.21.15.32:0/2345949923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7ab5243a2a7b61 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f1995d9b700 1 -- 172.21.15.32:0/2345949923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1988004180 con 0x7f1998107040 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f1995d9b700 1 -- 172.21.15.32:0/2345949923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19880042e0 con 0x7f1998107040 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f1995d9b700 1 -- 172.21.15.32:0/2345949923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1988011af0 con 0x7f1998107040 2024-06-03T01:12:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f199cae0700 1 -- 172.21.15.32:0/2345949923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 msgr2=0x7f1998109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f199cae0700 1 --2- 172.21.15.32:0/2345949923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.787+0000 7f199cae0700 1 -- 172.21.15.32:0/2345949923 shutdown_connections 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 --2- 172.21.15.32:0/2345949923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f1998109430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 -- 172.21.15.32:0/2345949923 >> 172.21.15.32:0/2345949923 conn(0x7f19981009f0 msgr2=0x7f1998102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 -- 172.21.15.32:0/2345949923 shutdown_connections 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 -- 172.21.15.32:0/2345949923 wait complete. 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 Processor -- start 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.788+0000 7f199cae0700 1 -- start start 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199cae0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199cae0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199819c8b0 con 0x7f1998107040 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43420/0 (socket says 172.21.15.32:43420) 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199659c700 1 -- 172.21.15.32:0/236587945 learned_addr learned my addr 172.21.15.32:0/236587945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199659c700 1 -- 172.21.15.32:0/236587945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988009550 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199659c700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1988006ba0 tx=0x7f1988011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1988003b00 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f198801a8e0 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199819cab0 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.789+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19880199c0 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.790+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199819cf50 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.790+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f1988019c30 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.791+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199810a700 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.791+0000 7f198f7fe700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 0x7f198403a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.791+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f198804c520 con 0x7f1998107040 2024-06-03T01:12:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.791+0000 7f198ffff700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 0x7f198403a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.791+0000 7f198ffff700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 0x7f198403a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.796+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f198801a430 con 0x7f1998107040 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:16.892+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f198802d960 con 0x7f1998107040 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.013+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f1998101e50 con 0x7f1984038240 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.143+0000 7f198f7fe700 1 -- 172.21.15.32:0/236587945 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f1998101e50 con 0x7f1984038240 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 msgr2=0x7f198403a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 0x7f198403a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 msgr2=0x7f199819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1988006ba0 tx=0x7f1988011b50 comp rx=0 tx=0).stop 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 shutdown_connections 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f1984038240 0x7f198403a700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 --2- 172.21.15.32:0/236587945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1998107040 0x7f199819c370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:17.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 >> 172.21.15.32:0/236587945 conn(0x7f19981009f0 msgr2=0x7f1998101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:17.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 shutdown_connections 2024-06-03T01:12:17.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:17.145+0000 7f199cae0700 1 -- 172.21.15.32:0/236587945 wait complete. 2024-06-03T01:12:17.679 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: [03/Jun/2024:01:12:15] ENGINE Bus STARTING 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: [03/Jun/2024:01:12:16] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: [03/Jun/2024:01:12:16] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: [03/Jun/2024:01:12:16] ENGINE Bus STARTED 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:17.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: mgrmap e12: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:12:17.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:17.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:17 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:18.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:18 smithi032 ceph-mon[29360]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:19.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KfH.cbaU4zTwFfUQO9V1i.ODdqabZ3/LG2uzUBRgEgMxoO8qFIu3a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717377138, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-03T01:12:19.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.067+0000 7fd718276700 1 Processor -- start 2024-06-03T01:12:19.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.067+0000 7fd718276700 1 -- start start 2024-06-03T01:12:19.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd718276700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd718276700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100797a0 con 0x7fd71007ad70 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd716012700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd716012700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39824/0 (socket says 172.21.15.32:39824) 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd716012700 1 -- 172.21.15.32:0/56155872 learned_addr learned my addr 172.21.15.32:0/56155872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd716012700 1 -- 172.21.15.32:0/56155872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100798e0 con 0x7fd71007ad70 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd716012700 1 --2- 172.21.15.32:0/56155872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd700009e80 tx=0x7fd70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d491b8ee1e04f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd715010700 1 -- 172.21.15.32:0/56155872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd70000e070 con 0x7fd71007ad70 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd715010700 1 -- 172.21.15.32:0/56155872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd70000b8c0 con 0x7fd71007ad70 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.068+0000 7fd715010700 1 -- 172.21.15.32:0/56155872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7000044d0 con 0x7fd71007ad70 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 -- 172.21.15.32:0/56155872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 msgr2=0x7fd7100791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 --2- 172.21.15.32:0/56155872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd700009e80 tx=0x7fd70000b330 comp rx=0 tx=0).stop 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 -- 172.21.15.32:0/56155872 shutdown_connections 2024-06-03T01:12:19.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 --2- 172.21.15.32:0/56155872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd7100791d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 -- 172.21.15.32:0/56155872 >> 172.21.15.32:0/56155872 conn(0x7fd710101b20 msgr2=0x7fd710103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 -- 172.21.15.32:0/56155872 shutdown_connections 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.069+0000 7fd718276700 1 -- 172.21.15.32:0/56155872 wait complete. 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd718276700 1 Processor -- start 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd718276700 1 -- start start 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd718276700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd718276700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71019c8e0 con 0x7fd71007ad70 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd716012700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd716012700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39838/0 (socket says 172.21.15.32:39838) 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd716012700 1 -- 172.21.15.32:0/2051394814 learned_addr learned my addr 172.21.15.32:0/2051394814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.070+0000 7fd716012700 1 -- 172.21.15.32:0/2051394814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009b30 con 0x7fd71007ad70 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd716012700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd700004940 tx=0x7fd700004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:19.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd70000e040 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd70000bc40 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7000126e0 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71019cae0 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.071+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71019cfe0 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.072+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fd700019040 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.072+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71010c920 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.072+0000 7fd70f7fe700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 0x7fd6f803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.072+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd70004bde0 con 0x7fd71007ad70 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.072+0000 7fd715811700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 0x7fd6f803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.073+0000 7fd715811700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 0x7fd6f803a8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:19.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.077+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd700012950 con 0x7fd71007ad70 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.307+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7100621e0 con 0x7fd6f8038410 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.553+0000 7fd70f7fe700 1 -- 172.21.15.32:0/2051394814 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd7100621e0 con 0x7fd6f8038410 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 msgr2=0x7fd6f803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 0x7fd6f803a8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 msgr2=0x7fd71019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd700004940 tx=0x7fd700004a20 comp rx=0 tx=0).stop 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 shutdown_connections 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fd6f8038410 0x7fd6f803a8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 --2- 172.21.15.32:0/2051394814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71007ad70 0x7fd71019c3a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 >> 172.21.15.32:0/2051394814 conn(0x7fd710101b20 msgr2=0x7fd7101028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 shutdown_connections 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:18.555+0000 7fd718276700 1 -- 172.21.15.32:0/2051394814 wait complete. 2024-06-03T01:12:19.231 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2024-06-03T01:12:19.774 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:19 smithi032 ceph-mon[29360]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:19.774 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:19 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:20.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:20 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2664263223' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee817ac700 1 Processor -- start 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee817ac700 1 -- start start 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee817ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee817ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c105e60 con 0x7fee7c105470 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee7affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee7affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39840/0 (socket says 172.21.15.32:39840) 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.038+0000 7fee7affd700 1 -- 172.21.15.32:0/3918391452 learned_addr learned my addr 172.21.15.32:0/3918391452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.039+0000 7fee7affd700 1 -- 172.21.15.32:0/3918391452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c106670 con 0x7fee7c105470 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.039+0000 7fee7affd700 1 --2- 172.21.15.32:0/3918391452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009e80 tx=0x7fee6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9406cf7e4662915 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.039+0000 7fee79ffb700 1 -- 172.21.15.32:0/3918391452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee6c00e070 con 0x7fee7c105470 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.039+0000 7fee79ffb700 1 -- 172.21.15.32:0/3918391452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee6c00b8c0 con 0x7fee7c105470 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.039+0000 7fee79ffb700 1 -- 172.21.15.32:0/3918391452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee6c0044d0 con 0x7fee7c105470 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 -- 172.21.15.32:0/3918391452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 msgr2=0x7fee7c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 --2- 172.21.15.32:0/3918391452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009e80 tx=0x7fee6c00b330 comp rx=0 tx=0).stop 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 -- 172.21.15.32:0/3918391452 shutdown_connections 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 --2- 172.21.15.32:0/3918391452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c105890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 -- 172.21.15.32:0/3918391452 >> 172.21.15.32:0/3918391452 conn(0x7fee7c1009f0 msgr2=0x7fee7c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 -- 172.21.15.32:0/3918391452 shutdown_connections 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 -- 172.21.15.32:0/3918391452 wait complete. 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.040+0000 7fee817ac700 1 Processor -- start 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee817ac700 1 -- start start 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee817ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:20.846 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee817ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c19a6c0 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee7affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee7affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39842/0 (socket says 172.21.15.32:39842) 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee7affd700 1 -- 172.21.15.32:0/2664263223 learned_addr learned my addr 172.21.15.32:0/2664263223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee7affd700 1 -- 172.21.15.32:0/2664263223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c009b30 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.041+0000 7fee7affd700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004940 tx=0x7fee6c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee6c00e040 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c19a8c0 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee6c00bc40 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee6c0126e0 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c19adc0 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.042+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fee6c019040 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.043+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c0623c0 con 0x7fee7c105470 2024-06-03T01:12:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.043+0000 7fee73fff700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 0x7fee6803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.043+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee6c04bd40 con 0x7fee7c105470 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.043+0000 7fee7a7fc700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 0x7fee6803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.043+0000 7fee7a7fc700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 0x7fee6803a880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.048+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fee6c011570 con 0x7fee7c105470 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.244+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fee7c109870 con 0x7fee7c105470 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.244+0000 7fee73fff700 1 -- 172.21.15.32:0/2664263223 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fee6c01f020 con 0x7fee7c105470 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 msgr2=0x7fee6803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 0x7fee6803a880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 msgr2=0x7fee7c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004940 tx=0x7fee6c004a20 comp rx=0 tx=0).stop 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 shutdown_connections 2024-06-03T01:12:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fee680383c0 0x7fee6803a880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:20.849 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 --2- 172.21.15.32:0/2664263223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c105470 0x7fee7c19a180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:20.849 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 >> 172.21.15.32:0/2664263223 conn(0x7fee7c1009f0 msgr2=0x7fee7c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:20.849 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 shutdown_connections 2024-06-03T01:12:20.849 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:20.246+0000 7fee817ac700 1 -- 172.21.15.32:0/2664263223 wait complete. 2024-06-03T01:12:20.852 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T01:12:20.852 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-06-03T01:12:20.860 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T01:12:20.860 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-06-03T01:12:20.860 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-06-03T01:12:20.860 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-03T01:12:20.861 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032.front.sepia.ceph.com:8443/ 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: Password: mjdr37nltn 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config directory 2024-06-03T01:12:20.862 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2024-06-03T01:12:21.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:21 smithi032 ceph-mon[29360]: mgrmap e13: smithi032.kfcfaa(active, since 6s) 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92b4ea5700 1 Processor -- start 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92b4ea5700 1 -- start start 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92b4ea5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92b4ea5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b0106f50 con 0x7f92b0106560 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92ae59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:22.126 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92ae59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39854/0 (socket says 172.21.15.32:39854) 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.332+0000 7f92ae59c700 1 -- 172.21.15.32:0/2197534163 learned_addr learned my addr 172.21.15.32:0/2197534163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.333+0000 7f92ae59c700 1 -- 172.21.15.32:0/2197534163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0107760 con 0x7f92b0106560 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.333+0000 7f92ae59c700 1 --2- 172.21.15.32:0/2197534163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92a00098a0 tx=0x7f92a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ad23181303d0fc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.333+0000 7f92ad59a700 1 -- 172.21.15.32:0/2197534163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92a0004180 con 0x7f92b0106560 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.333+0000 7f92ad59a700 1 -- 172.21.15.32:0/2197534163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92a00042e0 con 0x7f92b0106560 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.333+0000 7f92ad59a700 1 -- 172.21.15.32:0/2197534163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92a0011af0 con 0x7f92b0106560 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2197534163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 msgr2=0x7f92b0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2197534163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92a00098a0 tx=0x7f92a0009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2197534163 shutdown_connections 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2197534163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b0106980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2197534163 >> 172.21.15.32:0/2197534163 conn(0x7f92b0101b40 msgr2=0x7f92b0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2197534163 shutdown_connections 2024-06-03T01:12:22.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2197534163 wait complete. 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.334+0000 7f92b4ea5700 1 Processor -- start 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92b4ea5700 1 -- start start 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92b4ea5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92b4ea5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b019a780 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92ae59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92ae59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39856/0 (socket says 172.21.15.32:39856) 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92ae59c700 1 -- 172.21.15.32:0/2150472309 learned_addr learned my addr 172.21.15.32:0/2150472309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92ae59c700 1 -- 172.21.15.32:0/2150472309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0009550 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92ae59c700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92a0011670 tx=0x7f92a0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92a00039c0 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b019a980 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92a0003b20 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92a00247f0 con 0x7f92b0106560 2024-06-03T01:12:22.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.335+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b019ae20 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.336+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f92a001a430 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.336+0000 7f929f7fe700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 0x7f929803a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.337+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b00623c0 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.337+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f92a004c450 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.337+0000 7f92add9b700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 0x7f929803a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.337+0000 7f92add9b700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 0x7f929803a8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006fd0 tx=0x7f92a4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.341+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f92a001a730 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.539+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f92b010a960 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.539+0000 7f929f7fe700 1 -- 172.21.15.32:0/2150472309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f92a0018350 con 0x7f92b0106560 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 msgr2=0x7f929803a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 0x7f929803a8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006fd0 tx=0x7f92a4006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 msgr2=0x7f92b019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92a0011670 tx=0x7f92a0011aa0 comp rx=0 tx=0).stop 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 shutdown_connections 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f9298038420 0x7f929803a8e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 --2- 172.21.15.32:0/2150472309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b0106560 0x7f92b019a240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.541+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 >> 172.21.15.32:0/2150472309 conn(0x7f92b0101b40 msgr2=0x7f92b0102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.542+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 shutdown_connections 2024-06-03T01:12:22.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:21.542+0000 7f92b4ea5700 1 -- 172.21.15.32:0/2150472309 wait complete. 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f28a2281700 1 Processor -- start 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f28a2281700 1 -- start start 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f28a2281700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f28a2281700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c079e90 con 0x7f289c106fd0 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f289bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f289bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39858/0 (socket says 172.21.15.32:39858) 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.515+0000 7f289bfff700 1 -- 172.21.15.32:0/264289673 learned_addr learned my addr 172.21.15.32:0/264289673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f289bfff700 1 -- 172.21.15.32:0/264289673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c109900 con 0x7f289c106fd0 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f289bfff700 1 --2- 172.21.15.32:0/264289673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f288c0098a0 tx=0x7f288c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb449fe51fac2aa8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f289affd700 1 -- 172.21.15.32:0/264289673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f288c004180 con 0x7f289c106fd0 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f289affd700 1 -- 172.21.15.32:0/264289673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f288c0042e0 con 0x7f289c106fd0 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f289affd700 1 -- 172.21.15.32:0/264289673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f288c011af0 con 0x7f289c106fd0 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f28a2281700 1 -- 172.21.15.32:0/264289673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 msgr2=0x7f289c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.516+0000 7f28a2281700 1 --2- 172.21.15.32:0/264289673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f288c0098a0 tx=0x7f288c009bb0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 -- 172.21.15.32:0/264289673 shutdown_connections 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 --2- 172.21.15.32:0/264289673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c1093c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 -- 172.21.15.32:0/264289673 >> 172.21.15.32:0/264289673 conn(0x7f289c100a20 msgr2=0x7f289c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 -- 172.21.15.32:0/264289673 shutdown_connections 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 -- 172.21.15.32:0/264289673 wait complete. 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 Processor -- start 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.517+0000 7f28a2281700 1 -- start start 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28a2281700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28a2281700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c19a750 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f289bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f289bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39876/0 (socket says 172.21.15.32:39876) 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f289bfff700 1 -- 172.21.15.32:0/1593343466 learned_addr learned my addr 172.21.15.32:0/1593343466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f289bfff700 1 -- 172.21.15.32:0/1593343466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c009550 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f289bfff700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f288c011670 tx=0x7f288c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f288c003a40 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f288c003ba0 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c19a950 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.518+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c19adf0 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.519+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f288c024790 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.519+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f288c02d3f0 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.519+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c10a6f0 con 0x7f289c106fd0 2024-06-03T01:12:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.520+0000 7f28997fa700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 0x7f288403a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.520+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f288c04c640 con 0x7f289c106fd0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.520+0000 7f289b7fe700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 0x7f288403a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.520+0000 7f289b7fe700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 0x7f288403a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2890006fd0 tx=0x7f2890006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.524+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f288c010bd0 con 0x7f289c106fd0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.837+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f289c10a910 con 0x7f289c106fd0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.839+0000 7f28997fa700 1 -- 172.21.15.32:0/1593343466 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f288c0107f0 con 0x7f289c106fd0 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 msgr2=0x7f288403a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 0x7f288403a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2890006fd0 tx=0x7f2890006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 msgr2=0x7f289c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f288c011670 tx=0x7f288c011b50 comp rx=0 tx=0).stop 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 shutdown_connections 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2884038460 0x7f288403a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 --2- 172.21.15.32:0/1593343466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289c106fd0 0x7f289c19a210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 >> 172.21.15.32:0/1593343466 conn(0x7f289c100a20 msgr2=0x7f289c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 shutdown_connections 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-06-03T01:12:22.841+0000 7f28a2281700 1 -- 172.21.15.32:0/1593343466 wait complete. 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-03T01:12:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:23.422 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2024-06-03T01:12:23.454 INFO:tasks.cephadm:Fetching config... 2024-06-03T01:12:23.458 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:12:23.458 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-03T01:12:23.476 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-03T01:12:23.476 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:12:23.476 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-03T01:12:23.532 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-03T01:12:23.533 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:12:23.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/keyring of=/dev/stdout 2024-06-03T01:12:23.599 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-03T01:12:23.600 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:12:23.600 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-03T01:12:23.656 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-03T01:12:23.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1ha3nMV9AHJ8oh58MpwmsalUDE8BeM6zTvmui+kxSiRYtEE+pTAIAGVyz0mDKEsX2H/kAjyvLwI9twWHbCJwCmrzcCw9R7afgRD0aL+uk5bft9tY3iMJGTObXGUIOkAm0HrHu2QWvskQ1ewDdxecC+Ymgeph0Jf0vWx9V+sl453rHVIoZetmyn4WmmdM3dQxYAba5dlb9FpRpFN0kkMd8MfyE4Up3Gg56WKd0WG3EkKfV5Afb/RnilqpCLBQO2fBz46ywagOhAMSz6xo4FzS9jgB6sM5W0YW+Ov425hE0VkJklIkCzg+sgM9dRjrx9wdqfkQkILYS9kKLzxEAyOGI721ob/o1Nd37whhY2Io/MauFHw6goJLyfdmycrBVnFOAN4jkMFiqjSj+IkehjfMRnBf81LiENn+2no2ld8LC7xzmLYJ4e9/NuNhqp/5lTZReJPTUgE9/jkppQiHQcDS1V9iHf1a491DK13+vO+uyBp2+FgGH6OnvU9qHi2OZgbk= ceph-12a144f0-2146-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T01:12:23.732 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1ha3nMV9AHJ8oh58MpwmsalUDE8BeM6zTvmui+kxSiRYtEE+pTAIAGVyz0mDKEsX2H/kAjyvLwI9twWHbCJwCmrzcCw9R7afgRD0aL+uk5bft9tY3iMJGTObXGUIOkAm0HrHu2QWvskQ1ewDdxecC+Ymgeph0Jf0vWx9V+sl453rHVIoZetmyn4WmmdM3dQxYAba5dlb9FpRpFN0kkMd8MfyE4Up3Gg56WKd0WG3EkKfV5Afb/RnilqpCLBQO2fBz46ywagOhAMSz6xo4FzS9jgB6sM5W0YW+Ov425hE0VkJklIkCzg+sgM9dRjrx9wdqfkQkILYS9kKLzxEAyOGI721ob/o1Nd37whhY2Io/MauFHw6goJLyfdmycrBVnFOAN4jkMFiqjSj+IkehjfMRnBf81LiENn+2no2ld8LC7xzmLYJ4e9/NuNhqp/5lTZReJPTUgE9/jkppQiHQcDS1V9iHf1a491DK13+vO+uyBp2+FgGH6OnvU9qHi2OZgbk= ceph-12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:12:23.743 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1ha3nMV9AHJ8oh58MpwmsalUDE8BeM6zTvmui+kxSiRYtEE+pTAIAGVyz0mDKEsX2H/kAjyvLwI9twWHbCJwCmrzcCw9R7afgRD0aL+uk5bft9tY3iMJGTObXGUIOkAm0HrHu2QWvskQ1ewDdxecC+Ymgeph0Jf0vWx9V+sl453rHVIoZetmyn4WmmdM3dQxYAba5dlb9FpRpFN0kkMd8MfyE4Up3Gg56WKd0WG3EkKfV5Afb/RnilqpCLBQO2fBz46ywagOhAMSz6xo4FzS9jgB6sM5W0YW+Ov425hE0VkJklIkCzg+sgM9dRjrx9wdqfkQkILYS9kKLzxEAyOGI721ob/o1Nd37whhY2Io/MauFHw6goJLyfdmycrBVnFOAN4jkMFiqjSj+IkehjfMRnBf81LiENn+2no2ld8LC7xzmLYJ4e9/NuNhqp/5lTZReJPTUgE9/jkppQiHQcDS1V9iHf1a491DK13+vO+uyBp2+FgGH6OnvU9qHi2OZgbk= ceph-12a144f0-2146-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T01:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1ha3nMV9AHJ8oh58MpwmsalUDE8BeM6zTvmui+kxSiRYtEE+pTAIAGVyz0mDKEsX2H/kAjyvLwI9twWHbCJwCmrzcCw9R7afgRD0aL+uk5bft9tY3iMJGTObXGUIOkAm0HrHu2QWvskQ1ewDdxecC+Ymgeph0Jf0vWx9V+sl453rHVIoZetmyn4WmmdM3dQxYAba5dlb9FpRpFN0kkMd8MfyE4Up3Gg56WKd0WG3EkKfV5Afb/RnilqpCLBQO2fBz46ywagOhAMSz6xo4FzS9jgB6sM5W0YW+Ov425hE0VkJklIkCzg+sgM9dRjrx9wdqfkQkILYS9kKLzxEAyOGI721ob/o1Nd37whhY2Io/MauFHw6goJLyfdmycrBVnFOAN4jkMFiqjSj+IkehjfMRnBf81LiENn+2no2ld8LC7xzmLYJ4e9/NuNhqp/5lTZReJPTUgE9/jkppQiHQcDS1V9iHf1a491DK13+vO+uyBp2+FgGH6OnvU9qHi2OZgbk= ceph-12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:12:23.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-03T01:12:24.003 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:23 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1593343466' entity='client.admin' 2024-06-03T01:12:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:12:26.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 -- 172.21.15.32:0/1256950984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 msgr2=0x7f10500ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:26.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 --2- 172.21.15.32:0/1256950984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f10500ffb20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f104c009910 tx=0x7f104c009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 -- 172.21.15.32:0/1256950984 shutdown_connections 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 --2- 172.21.15.32:0/1256950984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f10500ffb20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 -- 172.21.15.32:0/1256950984 >> 172.21.15.32:0/1256950984 conn(0x7f10500facc0 msgr2=0x7f10500fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 -- 172.21.15.32:0/1256950984 shutdown_connections 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.666+0000 7f105764d700 1 -- 172.21.15.32:0/1256950984 wait complete. 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f105764d700 1 Processor -- start 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f105764d700 1 -- start start 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f105764d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f105764d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050195e40 con 0x7f10500ff700 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f10553e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f10553e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39896/0 (socket says 172.21.15.32:39896) 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.667+0000 7f10553e9700 1 -- 172.21.15.32:0/3586747691 learned_addr learned my addr 172.21.15.32:0/3586747691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f10553e9700 1 -- 172.21.15.32:0/3586747691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0095f0 con 0x7f10500ff700 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f10553e9700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f104c005b40 tx=0x7f104c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f104c01d070 con 0x7f10500ff700 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f104c00be10 con 0x7f10500ff700 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1050196040 con 0x7f10500ff700 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f104c00f460 con 0x7f10500ff700 2024-06-03T01:12:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.668+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10501964e0 con 0x7f10500ff700 2024-06-03T01:12:26.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.669+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f104c00f5c0 con 0x7f10500ff700 2024-06-03T01:12:26.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.669+0000 7f10467fc700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 0x7f103c03a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:26.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.669+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050062430 con 0x7f10500ff700 2024-06-03T01:12:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.669+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f104c04d410 con 0x7f10500ff700 2024-06-03T01:12:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.669+0000 7f1054be8700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 0x7f103c03a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.670+0000 7f1054be8700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 0x7f103c03a8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:26.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.674+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1050062430 con 0x7f10500ff700 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:12:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:12:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:26 smithi032 ceph-mon[29360]: Deploying daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:12:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.874+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f105002cfe0 con 0x7f10500ff700 2024-06-03T01:12:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.877+0000 7f10467fc700 1 -- 172.21.15.32:0/3586747691 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f104c027070 con 0x7f10500ff700 2024-06-03T01:12:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 msgr2=0x7f103c03a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 0x7f103c03a8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 msgr2=0x7f1050195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f104c005b40 tx=0x7f104c004740 comp rx=0 tx=0).stop 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 shutdown_connections 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f103c038420 0x7f103c03a8e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 --2- 172.21.15.32:0/3586747691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500ff700 0x7f1050195900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 >> 172.21.15.32:0/3586747691 conn(0x7f10500facc0 msgr2=0x7f10500690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 shutdown_connections 2024-06-03T01:12:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:26.879+0000 7f105764d700 1 -- 172.21.15.32:0/3586747691 wait complete. 2024-06-03T01:12:27.400 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-03T01:12:27.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-03T01:12:27.973 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:27 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3586747691' entity='client.admin' 2024-06-03T01:12:28.020 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:12:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 -- 172.21.15.32:0/2096180685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 msgr2=0x7f00fc0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 --2- 172.21.15.32:0/2096180685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc0ffba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009910 tx=0x7f00f8009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 -- 172.21.15.32:0/2096180685 shutdown_connections 2024-06-03T01:12:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 --2- 172.21.15.32:0/2096180685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc0ffba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 -- 172.21.15.32:0/2096180685 >> 172.21.15.32:0/2096180685 conn(0x7f00fc0fad20 msgr2=0x7f00fc0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.555+0000 7f01043c3700 1 -- 172.21.15.32:0/2096180685 shutdown_connections 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f01043c3700 1 -- 172.21.15.32:0/2096180685 wait complete. 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f01043c3700 1 Processor -- start 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f01043c3700 1 -- start start 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f01043c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f01043c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc1986f0 con 0x7f00fc0ff780 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.556+0000 7f010215f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f010215f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52334/0 (socket says 172.21.15.32:52334) 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f010215f700 1 -- 172.21.15.32:0/3652675957 learned_addr learned my addr 172.21.15.32:0/3652675957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f010215f700 1 -- 172.21.15.32:0/3652675957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80095f0 con 0x7f00fc0ff780 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f010215f700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005b40 tx=0x7f00f8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f801d070 con 0x7f00fc0ff780 2024-06-03T01:12:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00f800be10 con 0x7f00fc0ff780 2024-06-03T01:12:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc198830 con 0x7f00fc0ff780 2024-06-03T01:12:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f800f460 con 0x7f00fc0ff780 2024-06-03T01:12:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.557+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc198cd0 con 0x7f00fc0ff780 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.558+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f00f800f6d0 con 0x7f00fc0ff780 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.558+0000 7f00f37fe700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 0x7f00e803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.558+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00f804c670 con 0x7f00fc0ff780 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.559+0000 7f010195e700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 0x7f00e803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.559+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc18e3a0 con 0x7f00fc0ff780 2024-06-03T01:12:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.559+0000 7f010195e700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 0x7f00e803a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006fd0 tx=0x7f00ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.564+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00f8027070 con 0x7f00fc0ff780 2024-06-03T01:12:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.790+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f00fc0611d0 con 0x7f00e80383c0 2024-06-03T01:12:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.794+0000 7f00f37fe700 1 -- 172.21.15.32:0/3652675957 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00fc0611d0 con 0x7f00e80383c0 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 msgr2=0x7f00e803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 0x7f00e803a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006fd0 tx=0x7f00ec006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 msgr2=0x7f00fc068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005b40 tx=0x7f00f8004740 comp rx=0 tx=0).stop 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 shutdown_connections 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f00e80383c0 0x7f00e803a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 --2- 172.21.15.32:0/3652675957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00fc0ff780 0x7f00fc068af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 >> 172.21.15.32:0/3652675957 conn(0x7f00fc0fad20 msgr2=0x7f00fc0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 shutdown_connections 2024-06-03T01:12:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:30.796+0000 7f01043c3700 1 -- 172.21.15.32:0/3652675957 wait complete. 2024-06-03T01:12:31.354 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-06-03T01:12:31.355 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:12:31.355 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-06-03T01:12:31.375 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:12:31.375 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:12:31.434 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-06-03T01:12:31.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch host add smithi044 2024-06-03T01:12:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:12:32.084 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:31 smithi032 ceph-mon[29360]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:32.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:31 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:33.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:33.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:12:33.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:33 smithi032 ceph-mon[29360]: Deploying daemon crash.smithi032 on smithi032 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 -- 172.21.15.32:0/2557426749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 msgr2=0x7f866c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 --2- 172.21.15.32:0/2557426749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c103700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 -- 172.21.15.32:0/2557426749 shutdown_connections 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 --2- 172.21.15.32:0/2557426749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c103700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 -- 172.21.15.32:0/2557426749 >> 172.21.15.32:0/2557426749 conn(0x7f866c0fad20 msgr2=0x7f866c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 -- 172.21.15.32:0/2557426749 shutdown_connections 2024-06-03T01:12:33.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.674+0000 7f8674691700 1 -- 172.21.15.32:0/2557426749 wait complete. 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f8674691700 1 Processor -- start 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f8674691700 1 -- start start 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f8674691700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f8674691700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c195ea0 con 0x7f866c101310 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f867242d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f867242d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52354/0 (socket says 172.21.15.32:52354) 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f867242d700 1 -- 172.21.15.32:0/2961603537 learned_addr learned my addr 172.21.15.32:0/2961603537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f867242d700 1 -- 172.21.15.32:0/2961603537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680095f0 con 0x7f866c101310 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.675+0000 7f867242d700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8668005b40 tx=0x7f8668004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f866801d070 con 0x7f866c101310 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c1960a0 con 0x7f866c101310 2024-06-03T01:12:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f866800be10 con 0x7f866c101310 2024-06-03T01:12:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f866800f460 con 0x7f866c101310 2024-06-03T01:12:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c196540 con 0x7f866c101310 2024-06-03T01:12:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.676+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f866800b960 con 0x7f866c101310 2024-06-03T01:12:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.677+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c18f4a0 con 0x7f866c101310 2024-06-03T01:12:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.677+0000 7f86637fe700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 0x7f865803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.677+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f866804c3f0 con 0x7f866c101310 2024-06-03T01:12:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.677+0000 7f8671c2c700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 0x7f865803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.677+0000 7f8671c2c700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 0x7f865803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f865c006fd0 tx=0x7f865c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:33.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.681+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f866802a360 con 0x7f866c101310 2024-06-03T01:12:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:33.887+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) v1 -- 0x7f866c0611d0 con 0x7f86580383c0 2024-06-03T01:12:35.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:35 smithi032 ceph-mon[29360]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:35.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:35 smithi032 ceph-mon[29360]: Deploying cephadm binary to smithi044 2024-06-03T01:12:35.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:35 smithi032 ceph-mon[29360]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:35.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:35 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:36.465 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-06-03T01:12:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.463+0000 7f86637fe700 1 -- 172.21.15.32:0/2961603537 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f866c0611d0 con 0x7f86580383c0 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.465+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 msgr2=0x7f865803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.465+0000 7f8674691700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 0x7f865803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f865c006fd0 tx=0x7f865c006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 msgr2=0x7f866c195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8668005b40 tx=0x7f8668004740 comp rx=0 tx=0).stop 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 shutdown_connections 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f86580383c0 0x7f865803a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 --2- 172.21.15.32:0/2961603537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c101310 0x7f866c195960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 >> 172.21.15.32:0/2961603537 conn(0x7f866c0fad20 msgr2=0x7f866c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 shutdown_connections 2024-06-03T01:12:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:36.466+0000 7f8674691700 1 -- 172.21.15.32:0/2961603537 wait complete. 2024-06-03T01:12:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:37.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:37.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: Deploying daemon node-exporter.smithi032 on smithi032 2024-06-03T01:12:37.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:37.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: Added host smithi044 2024-06-03T01:12:37.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:36 smithi032 ceph-mon[29360]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:37.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T01:12:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:12:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 -- 172.21.15.32:0/1512655086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 msgr2=0x7f42780ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 --2- 172.21.15.32:0/1512655086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f42780ffba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f426800b0a0 tx=0x7f426800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:12:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 -- 172.21.15.32:0/1512655086 shutdown_connections 2024-06-03T01:12:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 --2- 172.21.15.32:0/1512655086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f42780ffba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 -- 172.21.15.32:0/1512655086 >> 172.21.15.32:0/1512655086 conn(0x7f42780fad20 msgr2=0x7f42780fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 -- 172.21.15.32:0/1512655086 shutdown_connections 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.466+0000 7f427f12d700 1 -- 172.21.15.32:0/1512655086 wait complete. 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427f12d700 1 Processor -- start 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427f12d700 1 -- start start 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427f12d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427f12d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278193cc0 con 0x7f42780ff780 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427cec9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427cec9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36640/0 (socket says 172.21.15.32:36640) 2024-06-03T01:12:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427cec9700 1 -- 172.21.15.32:0/2393355625 learned_addr learned my addr 172.21.15.32:0/2393355625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427cec9700 1 -- 172.21.15.32:0/2393355625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268009d00 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.467+0000 7f427cec9700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f426800b830 tx=0x7f426800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.468+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f426800e040 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.468+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4268012af0 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.468+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f426801d9f0 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.468+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4278193ec0 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.468+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42781942e0 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.469+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f4268012c60 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.469+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427818d290 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.469+0000 7f4275ffb700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 0x7f426403cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.469+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f426804c870 con 0x7f42780ff780 2024-06-03T01:12:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.469+0000 7f4277fff700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 0x7f426403cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:39.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.470+0000 7f4277fff700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 0x7f426403cb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:39.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.474+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f426801c5c0 con 0x7f42780ff780 2024-06-03T01:12:39.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.684+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42780611d0 con 0x7f426403a640 2024-06-03T01:12:39.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.685+0000 7f4275ffb700 1 -- 172.21.15.32:0/2393355625 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f42780611d0 con 0x7f426403a640 2024-06-03T01:12:39.685 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:12:39.685 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.687+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 msgr2=0x7f426403cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.687+0000 7f427f12d700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 0x7f426403cb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.687+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 msgr2=0x7f4278193780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.687+0000 7f427f12d700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f426800b830 tx=0x7f426800bd40 comp rx=0 tx=0).stop 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 shutdown_connections 2024-06-03T01:12:39.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f426403a640 0x7f426403cb00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 --2- 172.21.15.32:0/2393355625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42780ff780 0x7f4278193780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 >> 172.21.15.32:0/2393355625 conn(0x7f42780fad20 msgr2=0x7f42780fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 shutdown_connections 2024-06-03T01:12:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:39.688+0000 7f427f12d700 1 -- 172.21.15.32:0/2393355625 wait complete. 2024-06-03T01:12:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:39 smithi032 ceph-mon[29360]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:39 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:40.439 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-03T01:12:40.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-03T01:12:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:12:41.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:12:41.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:41.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:41.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:41.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:41.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:41 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 -- 172.21.15.32:0/2294309825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 msgr2=0x7f15540ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 --2- 172.21.15.32:0/2294309825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15540ffba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1544009910 tx=0x7f1544009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 -- 172.21.15.32:0/2294309825 shutdown_connections 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 --2- 172.21.15.32:0/2294309825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15540ffba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 -- 172.21.15.32:0/2294309825 >> 172.21.15.32:0/2294309825 conn(0x7f15540fad20 msgr2=0x7f15540fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 -- 172.21.15.32:0/2294309825 shutdown_connections 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.323+0000 7f1558f41700 1 -- 172.21.15.32:0/2294309825 wait complete. 2024-06-03T01:12:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f1558f41700 1 Processor -- start 2024-06-03T01:12:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f1558f41700 1 -- start start 2024-06-03T01:12:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f1558f41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f1558f41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554194de0 con 0x7f15540ff780 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f155259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f155259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36662/0 (socket says 172.21.15.32:36662) 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.324+0000 7f155259c700 1 -- 172.21.15.32:0/2625721295 learned_addr learned my addr 172.21.15.32:0/2625721295 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f155259c700 1 -- 172.21.15.32:0/2625721295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15440095f0 con 0x7f15540ff780 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f155259c700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1544005b40 tx=0x7f1544004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154401d070 con 0x7f15540ff780 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554194fe0 con 0x7f15540ff780 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f154400be10 con 0x7f15540ff780 2024-06-03T01:12:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154400f460 con 0x7f15540ff780 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.325+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554195480 con 0x7f15540ff780 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.326+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f154400f6d0 con 0x7f15540ff780 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.326+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155418e3a0 con 0x7f15540ff780 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.326+0000 7f15437fe700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 0x7f153c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.326+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f154404c3e0 con 0x7f15540ff780 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.326+0000 7f1551d9b700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 0x7f153c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.327+0000 7f1551d9b700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 0x7f153c03a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.331+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f154402a360 con 0x7f15540ff780 2024-06-03T01:12:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.574+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f155402cc70 con 0x7f15540ff780 2024-06-03T01:12:42.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.622+0000 7f15437fe700 1 -- 172.21.15.32:0/2625721295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1544027020 con 0x7f15540ff780 2024-06-03T01:12:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 msgr2=0x7f153c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 0x7f153c03a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 msgr2=0x7f15541948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1544005b40 tx=0x7f1544004740 comp rx=0 tx=0).stop 2024-06-03T01:12:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 shutdown_connections 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f153c0383c0 0x7f153c03a880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 --2- 172.21.15.32:0/2625721295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15540ff780 0x7f15541948a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 >> 172.21.15.32:0/2625721295 conn(0x7f15540fad20 msgr2=0x7f15540fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 shutdown_connections 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:12:42.624+0000 7f1558f41700 1 -- 172.21.15.32:0/2625721295 wait complete. 2024-06-03T01:12:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-06-03T01:12:42.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:42 smithi032 ceph-mon[29360]: Deploying daemon alertmanager.smithi032 on smithi032 2024-06-03T01:12:42.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:42 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2625721295' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T01:12:43.715 INFO:tasks.cephadm:Adding mon.smithi032 on smithi032 2024-06-03T01:12:43.715 INFO:tasks.cephadm:Adding mon.smithi044 on smithi044 2024-06-03T01:12:43.715 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch apply mon '2;smithi032:172.21.15.32=smithi032;smithi044:172.21.15.44=smithi044' 2024-06-03T01:12:43.926 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:43.975 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:43 smithi032 ceph-mon[29360]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:43 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2625721295' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T01:12:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:43 smithi032 ceph-mon[29360]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T01:12:45.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 -- 172.21.15.44:0/2378984910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 msgr2=0x7f8ae8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:45.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 --2- 172.21.15.44:0/2378984910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae8100ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009910 tx=0x7f8ad8009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:45.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 -- 172.21.15.44:0/2378984910 shutdown_connections 2024-06-03T01:12:45.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 --2- 172.21.15.44:0/2378984910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae8100ca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:45.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 -- 172.21.15.44:0/2378984910 >> 172.21.15.44:0/2378984910 conn(0x7f8ae80fbe60 msgr2=0x7f8ae80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 -- 172.21.15.44:0/2378984910 shutdown_connections 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.476+0000 7f8aedefc700 1 -- 172.21.15.44:0/2378984910 wait complete. 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8aedefc700 1 Processor -- start 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8aedefc700 1 -- start start 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8aedefc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8aedefc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8195ee0 con 0x7f8ae8100880 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:43878/0 (socket says 172.21.15.44:43878) 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.477+0000 7f8ae77fe700 1 -- 172.21.15.44:0/1802163162 learned_addr learned my addr 172.21.15.44:0/1802163162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8ae77fe700 1 -- 172.21.15.44:0/1802163162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80095f0 con 0x7f8ae8100880 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8ae77fe700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8005b40 tx=0x7f8ad8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:45.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ad801d070 con 0x7f8ae8100880 2024-06-03T01:12:45.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae81960e0 con 0x7f8ae8100880 2024-06-03T01:12:45.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ad800be10 con 0x7f8ae8100880 2024-06-03T01:12:45.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ad800f460 con 0x7f8ae8100880 2024-06-03T01:12:45.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.478+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8196580 con 0x7f8ae8100880 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.479+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f8ad800b960 con 0x7f8ae8100880 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.479+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae818f4a0 con 0x7f8ae8100880 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.479+0000 7f8ae4ff9700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 0x7f8ad403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.479+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ad804c4b0 con 0x7f8ae8100880 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.479+0000 7f8ae6ffd700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 0x7f8ad403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:45.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.480+0000 7f8ae6ffd700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 0x7f8ad403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006fd0 tx=0x7f8ad0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:45.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.486+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8ad802a360 con 0x7f8ae8100880 2024-06-03T01:12:45.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.693+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 --> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi032:172.21.15.32=smithi032;smithi044:172.21.15.44=smithi044", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ae8061340 con 0x7f8ad40383c0 2024-06-03T01:12:45.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.699+0000 7f8ae4ff9700 1 -- 172.21.15.44:0/1802163162 <== mgr.14164 v2:172.21.15.32:6800/1578498944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8ae8061340 con 0x7f8ad40383c0 2024-06-03T01:12:45.699 INFO:teuthology.orchestra.run.smithi044.stdout:Scheduled mon update... 2024-06-03T01:12:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 msgr2=0x7f8ad403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 0x7f8ad403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006fd0 tx=0x7f8ad0006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 msgr2=0x7f8ae81959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:45.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8005b40 tx=0x7f8ad8004740 comp rx=0 tx=0).stop 2024-06-03T01:12:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 shutdown_connections 2024-06-03T01:12:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ad40383c0 0x7f8ad403a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:45.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 --2- 172.21.15.44:0/1802163162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae8100880 0x7f8ae81959a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 >> 172.21.15.44:0/1802163162 conn(0x7f8ae80fbe60 msgr2=0x7f8ae80fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 shutdown_connections 2024-06-03T01:12:45.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:45.701+0000 7f8aedefc700 1 -- 172.21.15.44:0/1802163162 wait complete. 2024-06-03T01:12:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:45 smithi032 ceph-mon[29360]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:45 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:46.283 DEBUG:teuthology.orchestra.run.smithi044:mon.smithi044> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi044.service 2024-06-03T01:12:46.286 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:12:46.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:12:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:46.932 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:46 smithi032 ceph-mon[29360]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi032:172.21.15.32=smithi032;smithi044:172.21.15.44=smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:12:46.932 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:46 smithi032 ceph-mon[29360]: Saving service mon spec with placement smithi032:172.21.15.32=smithi032;smithi044:172.21.15.44=smithi044;count:2 2024-06-03T01:12:46.932 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:46 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:46.932 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:46 smithi032 ceph-mon[29360]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:48.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/2100320434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 msgr2=0x7f0b840fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:48.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/2100320434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b840fef70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009910 tx=0x7f0b80009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/2100320434 shutdown_connections 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/2100320434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b840fef70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/2100320434 >> 172.21.15.44:0/2100320434 conn(0x7f0b840fa4b0 msgr2=0x7f0b840fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/2100320434 shutdown_connections 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/2100320434 wait complete. 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 Processor -- start 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- start start 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:48.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.108+0000 7f0b8bbdb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841980b0 con 0x7f0b840feb50 2024-06-03T01:12:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.109+0000 7f0b89977700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.109+0000 7f0b89977700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54296/0 (socket says 172.21.15.44:54296) 2024-06-03T01:12:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.109+0000 7f0b89977700 1 -- 172.21.15.44:0/3659755773 learned_addr learned my addr 172.21.15.44:0/3659755773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:12:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.109+0000 7f0b89977700 1 -- 172.21.15.44:0/3659755773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b800095f0 con 0x7f0b840feb50 2024-06-03T01:12:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.109+0000 7f0b89977700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005b40 tx=0x7f0b80004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.110+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b8001d070 con 0x7f0b840feb50 2024-06-03T01:12:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.110+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b8000be10 con 0x7f0b840feb50 2024-06-03T01:12:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.110+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b841982b0 con 0x7f0b840feb50 2024-06-03T01:12:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.110+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b8000f460 con 0x7f0b840feb50 2024-06-03T01:12:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.110+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b84198750 con 0x7f0b840feb50 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.111+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f0b8000f6d0 con 0x7f0b840feb50 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.111+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b841916c0 con 0x7f0b840feb50 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.111+0000 7f0b7affd700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 0x7f0b7003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.111+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b8004c6b0 con 0x7f0b840feb50 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.111+0000 7f0b89176700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 0x7f0b7003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.112+0000 7f0b89176700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 0x7f0b7003a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:48.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.117+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0b80020730 con 0x7f0b840feb50 2024-06-03T01:12:48.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.392+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b840ff680 con 0x7f0b840feb50 2024-06-03T01:12:48.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.392+0000 7f0b7affd700 1 -- 172.21.15.44:0/3659755773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0b840ff680 con 0x7f0b840feb50 2024-06-03T01:12:48.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:12:48.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:12:48.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.394+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 msgr2=0x7f0b7003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.394+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 0x7f0b7003a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.394+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 msgr2=0x7f0b84197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.394+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005b40 tx=0x7f0b80004740 comp rx=0 tx=0).stop 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 shutdown_connections 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0b700383c0 0x7f0b7003a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 --2- 172.21.15.44:0/3659755773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b840feb50 0x7f0b84197b70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 >> 172.21.15.44:0/3659755773 conn(0x7f0b840fa4b0 msgr2=0x7f0b840fb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 shutdown_connections 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:48.395+0000 7f0b8bbdb700 1 -- 172.21.15.44:0/3659755773 wait complete. 2024-06-03T01:12:48.396 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:12:48.828 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:48 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/3659755773' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:12:49.784 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:49 smithi032 ceph-mon[29360]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:49.994 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:12:49.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:12:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:51.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- 172.21.15.44:0/4200902012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 msgr2=0x7fc784101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/4200902012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc784101230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc780009910 tx=0x7fc780009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- 172.21.15.44:0/4200902012 shutdown_connections 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/4200902012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc784101230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- 172.21.15.44:0/4200902012 >> 172.21.15.44:0/4200902012 conn(0x7fc7840fa4d0 msgr2=0x7fc7840fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- 172.21.15.44:0/4200902012 shutdown_connections 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- 172.21.15.44:0/4200902012 wait complete. 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 Processor -- start 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.551+0000 7fc78c3a3700 1 -- start start 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78c3a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78c3a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784198700 con 0x7fc784100e10 2024-06-03T01:12:51.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78a13f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78a13f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54302/0 (socket says 172.21.15.44:54302) 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78a13f700 1 -- 172.21.15.44:0/666418837 learned_addr learned my addr 172.21.15.44:0/666418837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78a13f700 1 -- 172.21.15.44:0/666418837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800095f0 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78a13f700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7800092e0 tx=0x7fc780009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc78001d070 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc784198840 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.552+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc78000be10 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.553+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc78000f460 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.553+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc784198ce0 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.553+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78418e460 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.553+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc780022470 con 0x7fc784100e10 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.554+0000 7fc7777fe700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 0x7fc77003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.554+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc78004c620 con 0x7fc784100e10 2024-06-03T01:12:51.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.554+0000 7fc78993e700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 0x7fc77003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:51.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.554+0000 7fc78993e700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 0x7fc77003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:51.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.559+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc78000b960 con 0x7fc784100e10 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: Deploying daemon grafana.smithi032 on smithi032 2024-06-03T01:12:51.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:51 smithi032 ceph-mon[29360]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.838+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc78402cc70 con 0x7fc784100e10 2024-06-03T01:12:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.838+0000 7fc7777fe700 1 -- 172.21.15.44:0/666418837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc780027020 con 0x7fc784100e10 2024-06-03T01:12:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:12:51.839 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:12:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 msgr2=0x7fc77003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 0x7fc77003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 msgr2=0x7fc7840ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7800092e0 tx=0x7fc780009e60 comp rx=0 tx=0).stop 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 shutdown_connections 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fc7700383c0 0x7fc77003a880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 --2- 172.21.15.44:0/666418837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc784100e10 0x7fc7840ff430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.840+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 >> 172.21.15.44:0/666418837 conn(0x7fc7840fa4d0 msgr2=0x7fc7840fb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.841+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 shutdown_connections 2024-06-03T01:12:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:51.841+0000 7fc78c3a3700 1 -- 172.21.15.44:0/666418837 wait complete. 2024-06-03T01:12:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:12:52.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:52 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/666418837' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:12:53.628 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:12:53.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:12:53.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:53 smithi032 ceph-mon[29360]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2420095130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 msgr2=0x7f63a4100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2420095130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a4100cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6394009910 tx=0x7f6394009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2420095130 shutdown_connections 2024-06-03T01:12:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2420095130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a4100cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:55.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2420095130 >> 172.21.15.44:0/2420095130 conn(0x7f63a40fbe20 msgr2=0x7f63a40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2420095130 shutdown_connections 2024-06-03T01:12:55.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.325+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2420095130 wait complete. 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a8b6f700 1 Processor -- start 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a8b6f700 1 -- start start 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a8b6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a8b6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4195ee0 con 0x7f63a41008a0 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54320/0 (socket says 172.21.15.44:54320) 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a259c700 1 -- 172.21.15.44:0/2914290285 learned_addr learned my addr 172.21.15.44:0/2914290285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.326+0000 7f63a259c700 1 -- 172.21.15.44:0/2914290285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940095f0 con 0x7f63a41008a0 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f63a259c700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6394005b40 tx=0x7f6394004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:55.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639401d070 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f639400be10 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a41960e0 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639400f460 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.327+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a4196580 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.328+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f639400f5c0 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.328+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a418f4c0 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.328+0000 7f639b7fe700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 0x7f639003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.328+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f639404c5f0 con 0x7f63a41008a0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.328+0000 7f63a1d9b700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 0x7f639003a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:55.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.329+0000 7f63a1d9b700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 0x7f639003a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:55.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.334+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f639402a360 con 0x7f63a41008a0 2024-06-03T01:12:55.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.617+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f63a402cc70 con 0x7f63a41008a0 2024-06-03T01:12:55.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.617+0000 7f639b7fe700 1 -- 172.21.15.44:0/2914290285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6394027070 con 0x7f63a41008a0 2024-06-03T01:12:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:12:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 msgr2=0x7f639003a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 0x7f639003a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 msgr2=0x7f63a41959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6394005b40 tx=0x7f6394004740 comp rx=0 tx=0).stop 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 shutdown_connections 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6390038410 0x7f639003a8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 --2- 172.21.15.44:0/2914290285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63a41008a0 0x7f63a41959a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.619+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 >> 172.21.15.44:0/2914290285 conn(0x7f63a40fbe20 msgr2=0x7f63a40fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.620+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 shutdown_connections 2024-06-03T01:12:55.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:55.620+0000 7f63a8b6f700 1 -- 172.21.15.44:0/2914290285 wait complete. 2024-06-03T01:12:55.621 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:12:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:55 smithi032 ceph-mon[29360]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:55 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:12:56.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:56 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2914290285' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:12:57.178 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:12:57.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:12:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:57.424 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:12:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:57 smithi032 ceph-mon[29360]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1472607710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 msgr2=0x7f6bec0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1472607710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec0ffba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009910 tx=0x7f6be8009c20 comp rx=0 tx=0).stop 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1472607710 shutdown_connections 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1472607710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec0ffba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1472607710 >> 172.21.15.44:0/1472607710 conn(0x7f6bec0fad20 msgr2=0x7f6bec0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1472607710 shutdown_connections 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.788+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1472607710 wait complete. 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf3d41700 1 Processor -- start 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf3d41700 1 -- start start 2024-06-03T01:12:58.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf3d41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:58.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf3d41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec195ee0 con 0x7f6bec0ff780 2024-06-03T01:12:58.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf1add700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:58.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf1add700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:60330/0 (socket says 172.21.15.44:60330) 2024-06-03T01:12:58.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf1add700 1 -- 172.21.15.44:0/1391157361 learned_addr learned my addr 172.21.15.44:0/1391157361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:12:58.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf1add700 1 -- 172.21.15.44:0/1391157361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be80095f0 con 0x7f6bec0ff780 2024-06-03T01:12:58.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.789+0000 7f6bf1add700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6be8005b40 tx=0x7f6be8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:58.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.790+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6be801d070 con 0x7f6bec0ff780 2024-06-03T01:12:58.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.790+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec1960e0 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.790+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6be800be10 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.790+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6be800f460 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.790+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec196580 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.791+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f6be80227e0 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.791+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec18f3b0 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.791+0000 7f6be2ffd700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 0x7f6bd803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.791+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6be804c360 con 0x7f6bec0ff780 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.791+0000 7f6bf12dc700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 0x7f6bd803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:58.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.792+0000 7f6bf12dc700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 0x7f6bd803a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:58.796+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6be802ba00 con 0x7f6bec0ff780 2024-06-03T01:12:59.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.072+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6bec02cc70 con 0x7f6bec0ff780 2024-06-03T01:12:59.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.073+0000 7f6be2ffd700 1 -- 172.21.15.44:0/1391157361 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6be8027030 con 0x7f6bec0ff780 2024-06-03T01:12:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:12:59.073 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 msgr2=0x7f6bd803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 0x7f6bd803a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc006e40 comp rx=0 tx=0).stop 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 msgr2=0x7f6bec1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6be8005b40 tx=0x7f6be8004740 comp rx=0 tx=0).stop 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 shutdown_connections 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f6bd8038410 0x7f6bd803a8d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 --2- 172.21.15.44:0/1391157361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bec0ff780 0x7f6bec1959a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 >> 172.21.15.44:0/1391157361 conn(0x7f6bec0fad20 msgr2=0x7f6bec068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 shutdown_connections 2024-06-03T01:12:59.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:12:59.075+0000 7f6bf3d41700 1 -- 172.21.15.44:0/1391157361 wait complete. 2024-06-03T01:12:59.076 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:59 smithi032 ceph-mon[29360]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:12:59 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/1391157361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:00.622 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:00.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:01.892 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:01 smithi032 ceph-mon[29360]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/2720923067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 msgr2=0x7f2b44103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/2720923067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44103780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009910 tx=0x7f2b40009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/2720923067 shutdown_connections 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/2720923067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44103780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/2720923067 >> 172.21.15.44:0/2720923067 conn(0x7f2b440fad20 msgr2=0x7f2b440fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/2720923067 shutdown_connections 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.228+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/2720923067 wait complete. 2024-06-03T01:13:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4c6fe700 1 Processor -- start 2024-06-03T01:13:02.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4c6fe700 1 -- start start 2024-06-03T01:13:02.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4c6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4c6fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b44101230 con 0x7f2b44101390 2024-06-03T01:13:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4a49a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4a49a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:60342/0 (socket says 172.21.15.44:60342) 2024-06-03T01:13:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.229+0000 7f2b4a49a700 1 -- 172.21.15.44:0/803917269 learned_addr learned my addr 172.21.15.44:0/803917269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b4a49a700 1 -- 172.21.15.44:0/803917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400095f0 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b4a49a700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005b40 tx=0x7f2b40004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b4001d070 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b4000be10 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b4000f460 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b440ff340 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.230+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b440ff7e0 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.231+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b440623c0 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.231+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f2b400227e0 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.231+0000 7f2b3b7fe700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 0x7f2b3003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.231+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2b4004c4c0 con 0x7f2b44101390 2024-06-03T01:13:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.232+0000 7f2b49c99700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 0x7f2b3003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.232+0000 7f2b49c99700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 0x7f2b3003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.237+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b4002bba0 con 0x7f2b44101390 2024-06-03T01:13:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.517+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2b4418ee80 con 0x7f2b44101390 2024-06-03T01:13:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.518+0000 7f2b3b7fe700 1 -- 172.21.15.44:0/803917269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2b40027020 con 0x7f2b44101390 2024-06-03T01:13:02.518 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:02.518 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 msgr2=0x7f2b3003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 0x7f2b3003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 msgr2=0x7f2b44100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005b40 tx=0x7f2b40004740 comp rx=0 tx=0).stop 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 shutdown_connections 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f2b300383c0 0x7f2b3003a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 --2- 172.21.15.44:0/803917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b44101390 0x7f2b44100cf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 >> 172.21.15.44:0/803917269 conn(0x7f2b440fad20 msgr2=0x7f2b440fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 shutdown_connections 2024-06-03T01:13:02.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:02.520+0000 7f2b4c6fe700 1 -- 172.21.15.44:0/803917269 wait complete. 2024-06-03T01:13:02.521 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:02.881 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:02 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/803917269' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:03.829 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:03 smithi032 ceph-mon[29360]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:04.108 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:04.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:04.894 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:04.895 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:04 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 -- 172.21.15.44:0/575891719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 msgr2=0x7f8ebc100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 --2- 172.21.15.44:0/575891719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc100ca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b0a0 tx=0x7f8eac00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:13:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 -- 172.21.15.44:0/575891719 shutdown_connections 2024-06-03T01:13:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 --2- 172.21.15.44:0/575891719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc100ca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 -- 172.21.15.44:0/575891719 >> 172.21.15.44:0/575891719 conn(0x7f8ebc0fbe60 msgr2=0x7f8ebc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 -- 172.21.15.44:0/575891719 shutdown_connections 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.819+0000 7f8ec3491700 1 -- 172.21.15.44:0/575891719 wait complete. 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec3491700 1 Processor -- start 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec3491700 1 -- start start 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec3491700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec3491700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc075070 con 0x7f8ebc100880 2024-06-03T01:13:05.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec122d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec122d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:60356/0 (socket says 172.21.15.44:60356) 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.820+0000 7f8ec122d700 1 -- 172.21.15.44:0/3473835208 learned_addr learned my addr 172.21.15.44:0/3473835208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8ec122d700 1 -- 172.21.15.44:0/3473835208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009d00 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8ec122d700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b830 tx=0x7f8eac00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8eac00e040 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8eac012af0 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc073180 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8eac01d9f0 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.821+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc0735a0 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.822+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f8eac012c60 con 0x7f8ebc100880 2024-06-03T01:13:05.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.822+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc073fb0 con 0x7f8ebc100880 2024-06-03T01:13:05.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.822+0000 7f8eba7fc700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 0x7f8ea403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:05.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.822+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8eac04c990 con 0x7f8ebc100880 2024-06-03T01:13:05.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.822+0000 7f8ec0a2c700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 0x7f8ea403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:05.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.823+0000 7f8ec0a2c700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 0x7f8ea403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:05.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:05.827+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8eac01c5c0 con 0x7f8ebc100880 2024-06-03T01:13:05.999 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:05 smithi032 ceph-mon[29360]: Deploying daemon prometheus.smithi032 on smithi032 2024-06-03T01:13:05.999 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:05 smithi032 ceph-mon[29360]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:06.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.110+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ebc18ee80 con 0x7f8ebc100880 2024-06-03T01:13:06.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.110+0000 7f8eba7fc700 1 -- 172.21.15.44:0/3473835208 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8eac01c3e0 con 0x7f8ebc100880 2024-06-03T01:13:06.110 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:06.110 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:06.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 msgr2=0x7f8ea403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:06.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 0x7f8ea403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 msgr2=0x7f8ebc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b830 tx=0x7f8eac00bd40 comp rx=0 tx=0).stop 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 shutdown_connections 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f8ea40383c0 0x7f8ea403a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 --2- 172.21.15.44:0/3473835208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ebc100880 0x7f8ebc074b30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 >> 172.21.15.44:0/3473835208 conn(0x7f8ebc0fbe60 msgr2=0x7f8ebc0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 shutdown_connections 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:06.112+0000 7f8ec3491700 1 -- 172.21.15.44:0/3473835208 wait complete. 2024-06-03T01:13:06.113 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:06.962 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:06 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/3473835208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:07.699 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:07.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:07.951 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:07 smithi032 ceph-mon[29360]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 -- 172.21.15.44:0/1142079454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 msgr2=0x7f09640ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 --2- 172.21.15.44:0/1142079454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640ff330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0954009910 tx=0x7f0954009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 -- 172.21.15.44:0/1142079454 shutdown_connections 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 --2- 172.21.15.44:0/1142079454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640ff330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 -- 172.21.15.44:0/1142079454 >> 172.21.15.44:0/1142079454 conn(0x7f09640fa4b0 msgr2=0x7f09640fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 -- 172.21.15.44:0/1142079454 shutdown_connections 2024-06-03T01:13:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 -- 172.21.15.44:0/1142079454 wait complete. 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.284+0000 7f096a0d5700 1 Processor -- start 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f096a0d5700 1 -- start start 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f096a0d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f096a0d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09641986a0 con 0x7f09640fef10 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f09637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f09637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:50000/0 (socket says 172.21.15.44:50000) 2024-06-03T01:13:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f09637fe700 1 -- 172.21.15.44:0/3318252365 learned_addr learned my addr 172.21.15.44:0/3318252365 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.285+0000 7f09637fe700 1 -- 172.21.15.44:0/3318252365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540095f0 con 0x7f09640fef10 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f09637fe700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09540092e0 tx=0x7f0954009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f095401d070 con 0x7f09640fef10 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f095400be10 con 0x7f09640fef10 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09641987e0 con 0x7f09640fef10 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f095400f460 con 0x7f09640fef10 2024-06-03T01:13:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.286+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0964198c80 con 0x7f09640fef10 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f095400f6d0 con 0x7f09640fef10 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f0960ff9700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 0x7f094c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0964104cc0 con 0x7f09640fef10 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f095404c500 con 0x7f09640fef10 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f0962ffd700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 0x7f094c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.287+0000 7f0962ffd700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 0x7f094c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0958006fd0 tx=0x7f0958006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.294+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0954020bd0 con 0x7f09640fef10 2024-06-03T01:13:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.572+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f09641992e0 con 0x7f09640fef10 2024-06-03T01:13:09.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.573+0000 7f0960ff9700 1 -- 172.21.15.44:0/3318252365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f09540207e0 con 0x7f09640fef10 2024-06-03T01:13:09.573 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:09.573 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 msgr2=0x7f094c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 0x7f094c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0958006fd0 tx=0x7f0958006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 msgr2=0x7f09640fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09540092e0 tx=0x7f0954009e60 comp rx=0 tx=0).stop 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 shutdown_connections 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f094c0383c0 0x7f094c03a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 --2- 172.21.15.44:0/3318252365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09640fef10 0x7f09640fb440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 >> 172.21.15.44:0/3318252365 conn(0x7f09640fa4b0 msgr2=0x7f09640fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 shutdown_connections 2024-06-03T01:13:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:09.575+0000 7f096a0d5700 1 -- 172.21.15.44:0/3318252365 wait complete. 2024-06-03T01:13:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:09 smithi032 ceph-mon[29360]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:09 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/3318252365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:11.262 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:11.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:11.968 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:11 smithi032 ceph-mon[29360]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 -- 172.21.15.44:0/381551138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 msgr2=0x7fba14103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 --2- 172.21.15.44:0/381551138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14103710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba10009910 tx=0x7fba10009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 -- 172.21.15.44:0/381551138 shutdown_connections 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 --2- 172.21.15.44:0/381551138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14103710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 -- 172.21.15.44:0/381551138 >> 172.21.15.44:0/381551138 conn(0x7fba140fad30 msgr2=0x7fba140fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:13.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 -- 172.21.15.44:0/381551138 shutdown_connections 2024-06-03T01:13:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.015+0000 7fba1bb31700 1 -- 172.21.15.44:0/381551138 wait complete. 2024-06-03T01:13:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba1bb31700 1 Processor -- start 2024-06-03T01:13:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba1bb31700 1 -- start start 2024-06-03T01:13:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba1bb31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba1bb31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba14195ea0 con 0x7fba14101320 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba198cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba198cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:50018/0 (socket says 172.21.15.44:50018) 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba198cd700 1 -- 172.21.15.44:0/13568593 learned_addr learned my addr 172.21.15.44:0/13568593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.016+0000 7fba198cd700 1 -- 172.21.15.44:0/13568593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100095f0 con 0x7fba14101320 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba198cd700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba10005b40 tx=0x7fba10004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba1001d070 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fba1000be10 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba1000f460 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba141960a0 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.017+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba14196540 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.018+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1418f4b0 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.018+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fba100227e0 con 0x7fba14101320 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.018+0000 7fba06ffd700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 0x7fba0003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:13.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.018+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba1004c4b0 con 0x7fba14101320 2024-06-03T01:13:13.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.018+0000 7fba190cc700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 0x7fba0003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:13.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.019+0000 7fba190cc700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 0x7fba0003a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:13.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.023+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fba1002bba0 con 0x7fba14101320 2024-06-03T01:13:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.306+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fba1402cc70 con 0x7fba14101320 2024-06-03T01:13:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.307+0000 7fba06ffd700 1 -- 172.21.15.44:0/13568593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fba10027020 con 0x7fba14101320 2024-06-03T01:13:13.307 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:13.307 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 msgr2=0x7fba0003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 0x7fba0003a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 msgr2=0x7fba14195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba10005b40 tx=0x7fba10004740 comp rx=0 tx=0).stop 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 shutdown_connections 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7fba000383c0 0x7fba0003a880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 --2- 172.21.15.44:0/13568593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba14101320 0x7fba14195960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 >> 172.21.15.44:0/13568593 conn(0x7fba140fad30 msgr2=0x7fba140fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 shutdown_connections 2024-06-03T01:13:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:13.309+0000 7fba1bb31700 1 -- 172.21.15.44:0/13568593 wait complete. 2024-06-03T01:13:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:13 smithi032 ceph-mon[29360]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:13 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/13568593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:14.858 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:14.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:16.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:15 smithi032 ceph-mon[29360]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 -- 172.21.15.44:0/2672943705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 msgr2=0x7f64740ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 --2- 172.21.15.44:0/2672943705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64740ffbb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6464009910 tx=0x7f6464009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 -- 172.21.15.44:0/2672943705 shutdown_connections 2024-06-03T01:13:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 --2- 172.21.15.44:0/2672943705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64740ffbb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 -- 172.21.15.44:0/2672943705 >> 172.21.15.44:0/2672943705 conn(0x7f64740fad30 msgr2=0x7f64740fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 -- 172.21.15.44:0/2672943705 shutdown_connections 2024-06-03T01:13:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.542+0000 7f647880b700 1 -- 172.21.15.44:0/2672943705 wait complete. 2024-06-03T01:13:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647880b700 1 Processor -- start 2024-06-03T01:13:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647880b700 1 -- start start 2024-06-03T01:13:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647880b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647880b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474194de0 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:50030/0 (socket says 172.21.15.44:50030) 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.543+0000 7f647259c700 1 -- 172.21.15.44:0/2488707819 learned_addr learned my addr 172.21.15.44:0/2488707819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f647259c700 1 -- 172.21.15.44:0/2488707819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640095f0 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f647259c700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6464005b40 tx=0x7f6464004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f646401f070 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6474194fe0 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f646400be10 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6464024620 con 0x7f64740ff790 2024-06-03T01:13:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.544+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6474195480 con 0x7f64740ff790 2024-06-03T01:13:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.545+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64741055d0 con 0x7f64740ff790 2024-06-03T01:13:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.545+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f64640199c0 con 0x7f64740ff790 2024-06-03T01:13:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.546+0000 7f64637fe700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 0x7f645c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.546+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f646404e600 con 0x7f64740ff790 2024-06-03T01:13:16.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.546+0000 7f6471d9b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 0x7f645c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.546+0000 7f6471d9b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 0x7f645c03a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.551+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f646402c540 con 0x7f64740ff790 2024-06-03T01:13:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.833+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f64741057c0 con 0x7f64740ff790 2024-06-03T01:13:16.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.834+0000 7f64637fe700 1 -- 172.21.15.44:0/2488707819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6464029070 con 0x7f64740ff790 2024-06-03T01:13:16.834 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:16.834 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 msgr2=0x7f645c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 0x7f645c03a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 msgr2=0x7f64741948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6464005b40 tx=0x7f6464004740 comp rx=0 tx=0).stop 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 shutdown_connections 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f645c0383c0 0x7f645c03a880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 --2- 172.21.15.44:0/2488707819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64740ff790 0x7f64741948a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 >> 172.21.15.44:0/2488707819 conn(0x7f64740fad30 msgr2=0x7f64740fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 shutdown_connections 2024-06-03T01:13:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:16.836+0000 7f647880b700 1 -- 172.21.15.44:0/2488707819 wait complete. 2024-06-03T01:13:16.837 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:17.943 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:17 smithi032 ceph-mon[29360]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:17.943 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:17 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2488707819' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:18.489 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:18.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:18.698 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:18.746 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:19.873 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:19 smithi032 ceph-mon[29360]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 -- 172.21.15.44:0/230365785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 msgr2=0x7f6638100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 --2- 172.21.15.44:0/230365785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638100ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6628009910 tx=0x7f6628009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 -- 172.21.15.44:0/230365785 shutdown_connections 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 --2- 172.21.15.44:0/230365785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638100ca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 -- 172.21.15.44:0/230365785 >> 172.21.15.44:0/230365785 conn(0x7f66380fbe00 msgr2=0x7f66380fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 -- 172.21.15.44:0/230365785 shutdown_connections 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.206+0000 7f663d948700 1 -- 172.21.15.44:0/230365785 wait complete. 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f663d948700 1 Processor -- start 2024-06-03T01:13:20.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f663d948700 1 -- start start 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f663d948700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f663d948700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638195ea0 con 0x7f6638100880 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f6636ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f6636ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54140/0 (socket says 172.21.15.44:54140) 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.207+0000 7f6636ffd700 1 -- 172.21.15.44:0/949255281 learned_addr learned my addr 172.21.15.44:0/949255281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f6636ffd700 1 -- 172.21.15.44:0/949255281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280095f0 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f6636ffd700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6628005b40 tx=0x7f6628004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f662801d070 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f662800be10 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66381960a0 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f662800f460 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.208+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6638196540 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.209+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f662800f6d0 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.209+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663818f4a0 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.209+0000 7f663c946700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 0x7f662403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.209+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f662804c6c0 con 0x7f6638100880 2024-06-03T01:13:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.209+0000 7f662e7fc700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 0x7f662403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:20.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.210+0000 7f662e7fc700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 0x7f662403a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6620006fd0 tx=0x7f6620006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.214+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f662802a360 con 0x7f6638100880 2024-06-03T01:13:20.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.493+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f663802cc70 con 0x7f6638100880 2024-06-03T01:13:20.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.494+0000 7f663c946700 1 -- 172.21.15.44:0/949255281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6628027030 con 0x7f6638100880 2024-06-03T01:13:20.494 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:20.494 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 msgr2=0x7f662403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 0x7f662403a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6620006fd0 tx=0x7f6620006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:20.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 msgr2=0x7f6638195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6628005b40 tx=0x7f6628004740 comp rx=0 tx=0).stop 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 shutdown_connections 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f66240383c0 0x7f662403a880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 --2- 172.21.15.44:0/949255281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6638100880 0x7f6638195960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 >> 172.21.15.44:0/949255281 conn(0x7f66380fbe00 msgr2=0x7f66380fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 shutdown_connections 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:20.496+0000 7f663d948700 1 -- 172.21.15.44:0/949255281 wait complete. 2024-06-03T01:13:20.497 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:20 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/949255281' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:21.909 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:21 smithi032 ceph-mon[29360]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:21.909 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:21 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:21.909 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:21 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:21.909 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:21 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:21.910 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:21 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-03T01:13:22.041 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:22.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:22.252 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:22 smithi032 ceph-mon[29360]: from='mgr.14164 172.21.15.32:0/3457471779' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-03T01:13:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:22 smithi032 ceph-mon[29360]: mgrmap e14: smithi032.kfcfaa(active, since 67s) 2024-06-03T01:13:23.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 -- 172.21.15.44:0/2383828022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 msgr2=0x7f07340ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:23.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 --2- 172.21.15.44:0/2383828022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07340ffba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0730009910 tx=0x7f0730009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:23.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 -- 172.21.15.44:0/2383828022 shutdown_connections 2024-06-03T01:13:23.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 --2- 172.21.15.44:0/2383828022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07340ffba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 -- 172.21.15.44:0/2383828022 >> 172.21.15.44:0/2383828022 conn(0x7f07340fad20 msgr2=0x7f07340fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 -- 172.21.15.44:0/2383828022 shutdown_connections 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.784+0000 7f073c725700 1 -- 172.21.15.44:0/2383828022 wait complete. 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073c725700 1 Processor -- start 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073c725700 1 -- start start 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073c725700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073c725700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734195ee0 con 0x7f07340ff780 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073a4c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073a4c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54162/0 (socket says 172.21.15.44:54162) 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.785+0000 7f073a4c1700 1 -- 172.21.15.44:0/139168831 learned_addr learned my addr 172.21.15.44:0/139168831 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:23.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f073a4c1700 1 -- 172.21.15.44:0/139168831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300095f0 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f073a4c1700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0730005b40 tx=0x7f0730004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f073001d070 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07341960e0 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f073000be10 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f073000f460 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.786+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0734196580 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f07300227e0 con 0x7f07340ff780 2024-06-03T01:13:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073418f3b0 con 0x7f07340ff780 2024-06-03T01:13:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f072b7fe700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 0x7f072003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f073004c410 con 0x7f07340ff780 2024-06-03T01:13:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f0739cc0700 1 -- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 msgr2=0x7f072003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1578498944 2024-06-03T01:13:23.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.787+0000 7f0739cc0700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 0x7f072003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:13:23.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.792+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f073002bba0 con 0x7f07340ff780 2024-06-03T01:13:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.988+0000 7f0739cc0700 1 -- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 msgr2=0x7f072003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1578498944 2024-06-03T01:13:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:23.988+0000 7f0739cc0700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 0x7f072003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:13:24.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.077+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f073402cc70 con 0x7f07340ff780 2024-06-03T01:13:24.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.077+0000 7f072b7fe700 1 -- 172.21.15.44:0/139168831 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0730027030 con 0x7f07340ff780 2024-06-03T01:13:24.078 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:24.078 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 msgr2=0x7f072003a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 0x7f072003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 msgr2=0x7f07341959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0730005b40 tx=0x7f0730004740 comp rx=0 tx=0).stop 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 shutdown_connections 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f0720038410 0x7f072003a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 --2- 172.21.15.44:0/139168831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07340ff780 0x7f07341959a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 >> 172.21.15.44:0/139168831 conn(0x7f07340fad20 msgr2=0x7f0734068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:24.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 shutdown_connections 2024-06-03T01:13:24.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:24.080+0000 7f073c725700 1 -- 172.21.15.44:0/139168831 wait complete. 2024-06-03T01:13:24.081 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:24 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/139168831' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:25.705 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:25.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:25.918 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:25.966 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:27.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 -- 172.21.15.44:0/312162033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 msgr2=0x7f7194100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 --2- 172.21.15.44:0/312162033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f7194100cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7190009910 tx=0x7f7190009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 -- 172.21.15.44:0/312162033 shutdown_connections 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 --2- 172.21.15.44:0/312162033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f7194100cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 -- 172.21.15.44:0/312162033 >> 172.21.15.44:0/312162033 conn(0x7f71940fbe20 msgr2=0x7f71940fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 -- 172.21.15.44:0/312162033 shutdown_connections 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.438+0000 7f719bbf3700 1 -- 172.21.15.44:0/312162033 wait complete. 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719bbf3700 1 Processor -- start 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719bbf3700 1 -- start start 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719bbf3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719bbf3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7194195ee0 con 0x7f71941008a0 2024-06-03T01:13:27.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719998f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:27.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719998f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54622/0 (socket says 172.21.15.44:54622) 2024-06-03T01:13:27.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.439+0000 7f719998f700 1 -- 172.21.15.44:0/1549567719 learned_addr learned my addr 172.21.15.44:0/1549567719 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:27.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f719998f700 1 -- 172.21.15.44:0/1549567719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900095f0 con 0x7f71941008a0 2024-06-03T01:13:27.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f719998f700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7190005b40 tx=0x7f7190004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:27.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f719001d070 con 0x7f71941008a0 2024-06-03T01:13:27.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f719000be10 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71941960e0 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f719000f460 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.440+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7194196580 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f719000f6d0 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f718affd700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 0x7f718003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f719004c4e0 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719418f4c0 con 0x7f71941008a0 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f719918e700 1 -- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 msgr2=0x7f718003a920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1578498944 2024-06-03T01:13:27.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.441+0000 7f719918e700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 0x7f718003a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:13:27.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.446+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7190027070 con 0x7f71941008a0 2024-06-03T01:13:27.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.642+0000 7f719918e700 1 -- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 msgr2=0x7f718003a920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1578498944 2024-06-03T01:13:27.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.642+0000 7f719918e700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 0x7f718003a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:13:27.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.726+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f719402cc70 con 0x7f71941008a0 2024-06-03T01:13:27.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.726+0000 7f718affd700 1 -- 172.21.15.44:0/1549567719 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f719002c440 con 0x7f71941008a0 2024-06-03T01:13:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:27.727 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:27.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 msgr2=0x7f718003a920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:13:27.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 0x7f718003a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 msgr2=0x7f71941959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7190005b40 tx=0x7f7190004740 comp rx=0 tx=0).stop 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 shutdown_connections 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:6800/1578498944,v1:172.21.15.32:6801/1578498944] conn(0x7f7180038460 0x7f718003a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 --2- 172.21.15.44:0/1549567719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71941008a0 0x7f71941959a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 >> 172.21.15.44:0/1549567719 conn(0x7f71940fbe20 msgr2=0x7f71940fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.728+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 shutdown_connections 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:27.729+0000 7f719bbf3700 1 -- 172.21.15.44:0/1549567719 wait complete. 2024-06-03T01:13:27.729 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:27 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/1549567719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:29.333 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:29.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: Active manager daemon smithi032.kfcfaa restarted 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: osdmap e5: 0 total, 0 up, 0 in 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: mgrmap e15: smithi032.kfcfaa(active, starting, since 0.00315522s) 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:13:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:13:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:13:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:13:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:29.583 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:13:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:13:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:13:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:29 smithi032 ceph-mon[29360]: mgrmap e16: smithi032.kfcfaa(active, since 1.00536s) 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.099+0000 7fc1368c4700 1 -- 172.21.15.44:0/1538054498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 msgr2=0x7fc1300ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.099+0000 7fc1368c4700 1 --2- 172.21.15.44:0/1538054498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300ffbb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc120009910 tx=0x7fc120009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 -- 172.21.15.44:0/1538054498 shutdown_connections 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 --2- 172.21.15.44:0/1538054498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300ffbb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 -- 172.21.15.44:0/1538054498 >> 172.21.15.44:0/1538054498 conn(0x7fc1300fad30 msgr2=0x7fc1300fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:31.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 -- 172.21.15.44:0/1538054498 shutdown_connections 2024-06-03T01:13:31.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 -- 172.21.15.44:0/1538054498 wait complete. 2024-06-03T01:13:31.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 Processor -- start 2024-06-03T01:13:31.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.100+0000 7fc1368c4700 1 -- start start 2024-06-03T01:13:31.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc1368c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:31.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc1368c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300fd210 con 0x7fc1300ff790 2024-06-03T01:13:31.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:31.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54636/0 (socket says 172.21.15.44:54636) 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12ffff700 1 -- 172.21.15.44:0/2680046978 learned_addr learned my addr 172.21.15.44:0/2680046978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12ffff700 1 -- 172.21.15.44:0/2680046978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200095f0 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12ffff700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc120005b40 tx=0x7fc120004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.101+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc12001d070 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc12000be10 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc12000f460 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1300fb320 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1300fb7c0 con 0x7fc1300ff790 2024-06-03T01:13:31.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7fc12000b960 con 0x7fc1300ff790 2024-06-03T01:13:31.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.102+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1300623c0 con 0x7fc1300ff790 2024-06-03T01:13:31.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.103+0000 7fc12d7fa700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 0x7fc11803a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:31.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.103+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc12004c3e0 con 0x7fc1300ff790 2024-06-03T01:13:31.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.103+0000 7fc12f7fe700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 0x7fc11803a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:31.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.103+0000 7fc12f7fe700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 0x7fc11803a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:31.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.107+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc12002bba0 con 0x7fc1300ff790 2024-06-03T01:13:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.389+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc13018eda0 con 0x7fc1300ff790 2024-06-03T01:13:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.390+0000 7fc12d7fa700 1 -- 172.21.15.44:0/2680046978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc120027030 con 0x7fc1300ff790 2024-06-03T01:13:31.390 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:31.390 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 msgr2=0x7fc11803a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 0x7fc11803a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 msgr2=0x7fc1300fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc120005b40 tx=0x7fc120004740 comp rx=0 tx=0).stop 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 shutdown_connections 2024-06-03T01:13:31.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc118038270 0x7fc11803a730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:31.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 --2- 172.21.15.44:0/2680046978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1300ff790 0x7fc1300fccd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:31.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 >> 172.21.15.44:0/2680046978 conn(0x7fc1300fad30 msgr2=0x7fc130069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:31.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 shutdown_connections 2024-06-03T01:13:31.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:31.393+0000 7fc1368c4700 1 -- 172.21.15.44:0/2680046978 wait complete. 2024-06-03T01:13:31.394 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:31.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: [03/Jun/2024:01:13:30] ENGINE Bus STARTING 2024-06-03T01:13:31.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: [03/Jun/2024:01:13:30] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:13:31.890 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: [03/Jun/2024:01:13:30] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:13:31.890 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: [03/Jun/2024:01:13:30] ENGINE Bus STARTED 2024-06-03T01:13:31.890 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:31.890 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:31 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2680046978' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:32 smithi032 ceph-mon[29360]: mgrmap e17: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:13:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:32 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:32.966 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:32.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:33.414 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:33.474 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-03T01:13:34.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:33 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:34.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:33 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:34.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:33 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:34.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:33 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:34.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:33 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:13:35.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 -- 172.21.15.44:0/2430433740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 msgr2=0x7f47e0101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:35.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 --2- 172.21.15.44:0/2430433740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0101230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b0a0 tx=0x7f47d000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:13:35.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 -- 172.21.15.44:0/2430433740 shutdown_connections 2024-06-03T01:13:35.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 --2- 172.21.15.44:0/2430433740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0101230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 -- 172.21.15.44:0/2430433740 >> 172.21.15.44:0/2430433740 conn(0x7f47e00fa490 msgr2=0x7f47e00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 -- 172.21.15.44:0/2430433740 shutdown_connections 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.519+0000 7f47e7a83700 1 -- 172.21.15.44:0/2430433740 wait complete. 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e7a83700 1 Processor -- start 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e7a83700 1 -- start start 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e7a83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e7a83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0195ed0 con 0x7f47e0100e10 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e581f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e581f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:54656/0 (socket says 172.21.15.44:54656) 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.520+0000 7f47e581f700 1 -- 172.21.15.44:0/97611693 learned_addr learned my addr 172.21.15.44:0/97611693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47e581f700 1 -- 172.21.15.44:0/97611693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0009d00 con 0x7f47e0100e10 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47e581f700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47d0015040 tx=0x7f47d000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:35.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47d000e040 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47d001d930 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47d00129d0 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e01960d0 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.521+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e01964f0 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.522+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f47d0012c40 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.522+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e018f470 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.522+0000 7f47deffd700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 0x7f47c803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.522+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f47d004cae0 con 0x7f47e0100e10 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.523+0000 7f47e501e700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 0x7f47c803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.523+0000 7f47e501e700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 0x7f47c803aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.528+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f47d001c5c0 con 0x7f47e0100e10 2024-06-03T01:13:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.812+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f47e00623c0 con 0x7f47e0100e10 2024-06-03T01:13:35.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.812+0000 7f47deffd700 1 -- 172.21.15.44:0/97611693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f47d001c3e0 con 0x7f47e0100e10 2024-06-03T01:13:35.812 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:35.812 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:35.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 msgr2=0x7f47c803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:35.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 0x7f47c803aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:35.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 msgr2=0x7f47e0195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47d0015040 tx=0x7f47d000bb10 comp rx=0 tx=0).stop 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 shutdown_connections 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f47c8038550 0x7f47c803aa10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 --2- 172.21.15.44:0/97611693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e0100e10 0x7f47e0195990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 >> 172.21.15.44:0/97611693 conn(0x7f47e00fa490 msgr2=0x7f47e00ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 shutdown_connections 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:35.814+0000 7f47e7a83700 1 -- 172.21.15.44:0/97611693 wait complete. 2024-06-03T01:13:35.815 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/97611693' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:13:36.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:36 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:13:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:13:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:37 smithi032 ceph-mon[29360]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:13:37.418 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:37.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:37.688 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:39.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 -- 172.21.15.44:0/86960022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 msgr2=0x7fcf9c106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:39.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 --2- 172.21.15.44:0/86960022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c106bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009910 tx=0x7fcf8c009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:39.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 -- 172.21.15.44:0/86960022 shutdown_connections 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 --2- 172.21.15.44:0/86960022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c106bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 -- 172.21.15.44:0/86960022 >> 172.21.15.44:0/86960022 conn(0x7fcf9c0fc320 msgr2=0x7fcf9c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 -- 172.21.15.44:0/86960022 shutdown_connections 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.444+0000 7fcfa1738700 1 -- 172.21.15.44:0/86960022 wait complete. 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.445+0000 7fcfa1738700 1 Processor -- start 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.445+0000 7fcfa1738700 1 -- start start 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.445+0000 7fcfa1738700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.445+0000 7fcfa1738700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c198460 con 0x7fcf9c106810 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.445+0000 7fcf9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:53218/0 (socket says 172.21.15.44:53218) 2024-06-03T01:13:39.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf9affd700 1 -- 172.21.15.44:0/374910310 learned_addr learned my addr 172.21.15.44:0/374910310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf9affd700 1 -- 172.21.15.44:0/374910310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0095f0 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf9affd700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0092e0 tx=0x7fcf8c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf8c01d070 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c1986c0 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcf8c00be10 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf8c022e10 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.446+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c198b40 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.447+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fcf8c022470 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.447+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c0623c0 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.447+0000 7fcf98ff9700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 0x7fcf8803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.448+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcf8c04c500 con 0x7fcf9c106810 2024-06-03T01:13:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.448+0000 7fcf927fc700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 0x7fcf8803a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:39.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.448+0000 7fcf927fc700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 0x7fcf8803a920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf84006fd0 tx=0x7fcf84006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.453+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcf8c00f460 con 0x7fcf9c106810 2024-06-03T01:13:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.737+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcf9c19cb30 con 0x7fcf9c106810 2024-06-03T01:13:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.738+0000 7fcf98ff9700 1 -- 172.21.15.44:0/374910310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fcf8c027030 con 0x7fcf9c106810 2024-06-03T01:13:39.738 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:39.738 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 msgr2=0x7fcf8803a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 0x7fcf8803a920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf84006fd0 tx=0x7fcf84006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 msgr2=0x7fcf9c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0092e0 tx=0x7fcf8c004c80 comp rx=0 tx=0).stop 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 shutdown_connections 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcf88038460 0x7fcf8803a920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 --2- 172.21.15.44:0/374910310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf9c106810 0x7fcf9c197f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.740+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 >> 172.21.15.44:0/374910310 conn(0x7fcf9c0fc320 msgr2=0x7fcf9c108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.741+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 shutdown_connections 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:39.741+0000 7fcfa1738700 1 -- 172.21.15.44:0/374910310 wait complete. 2024-06-03T01:13:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/374910310' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-06-03T01:13:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:41.414 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:41.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:41.640 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:42.011 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:41 smithi032 ceph-mon[29360]: Deploying daemon crash.smithi044 on smithi044 2024-06-03T01:13:43.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 -- 172.21.15.44:0/1067002038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 msgr2=0x7f4d14101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:43.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 --2- 172.21.15.44:0/1067002038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14101da0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009910 tx=0x7f4d04009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:43.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 -- 172.21.15.44:0/1067002038 shutdown_connections 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 --2- 172.21.15.44:0/1067002038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14101da0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 -- 172.21.15.44:0/1067002038 >> 172.21.15.44:0/1067002038 conn(0x7f4d140fb210 msgr2=0x7f4d140fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 -- 172.21.15.44:0/1067002038 shutdown_connections 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.102+0000 7f4d19295700 1 -- 172.21.15.44:0/1067002038 wait complete. 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.103+0000 7f4d19295700 1 Processor -- start 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.103+0000 7f4d19295700 1 -- start start 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.103+0000 7f4d19295700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.103+0000 7f4d19295700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14196380 con 0x7f4d141019c0 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.104+0000 7f4d12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.104+0000 7f4d12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:53238/0 (socket says 172.21.15.44:53238) 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.104+0000 7f4d12ffd700 1 -- 172.21.15.44:0/2517098128 learned_addr learned my addr 172.21.15.44:0/2517098128 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.104+0000 7f4d12ffd700 1 -- 172.21.15.44:0/2517098128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040095f0 con 0x7f4d141019c0 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.104+0000 7f4d12ffd700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d04005b40 tx=0x7f4d04004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.105+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d0401d070 con 0x7f4d141019c0 2024-06-03T01:13:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.105+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d14196580 con 0x7f4d141019c0 2024-06-03T01:13:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.105+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4d0400be10 con 0x7f4d141019c0 2024-06-03T01:13:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.105+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d14196940 con 0x7f4d141019c0 2024-06-03T01:13:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.105+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d04022d90 con 0x7f4d141019c0 2024-06-03T01:13:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.106+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f4d0400f460 con 0x7f4d141019c0 2024-06-03T01:13:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.106+0000 7f4cfbfff700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 0x7f4cfc03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.106+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4d0404c580 con 0x7f4d141019c0 2024-06-03T01:13:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.106+0000 7f4d127fc700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 0x7f4cfc03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.106+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d140623c0 con 0x7f4d141019c0 2024-06-03T01:13:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.107+0000 7f4d127fc700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 0x7f4cfc03a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006fd0 tx=0x7f4d08006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.112+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d04017440 con 0x7f4d141019c0 2024-06-03T01:13:43.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.401+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4d1410ac50 con 0x7f4d141019c0 2024-06-03T01:13:43.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.402+0000 7f4cfbfff700 1 -- 172.21.15.44:0/2517098128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4d04020370 con 0x7f4d141019c0 2024-06-03T01:13:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:43.403 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:43.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 msgr2=0x7f4cfc03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:43.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 0x7f4cfc03a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006fd0 tx=0x7f4d08006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:43.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 msgr2=0x7f4d14195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:43.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d04005b40 tx=0x7f4d04004c80 comp rx=0 tx=0).stop 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 shutdown_connections 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4cfc0384b0 0x7f4cfc03a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 --2- 172.21.15.44:0/2517098128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d141019c0 0x7f4d14195e40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 >> 172.21.15.44:0/2517098128 conn(0x7f4d140fb210 msgr2=0x7f4d14105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 shutdown_connections 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:43.404+0000 7f4d19295700 1 -- 172.21.15.44:0/2517098128 wait complete. 2024-06-03T01:13:43.405 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:43.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:43 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2517098128' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: Deploying daemon node-exporter.smithi044 on smithi044 2024-06-03T01:13:44.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:44.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:13:45.208 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:45.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:45.415 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:48.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 -- 172.21.15.44:0/2693069731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 msgr2=0x7ff770106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2693069731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff770106c00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff760009910 tx=0x7ff760009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 -- 172.21.15.44:0/2693069731 shutdown_connections 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2693069731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff770106c00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 -- 172.21.15.44:0/2693069731 >> 172.21.15.44:0/2693069731 conn(0x7ff770075290 msgr2=0x7ff7700756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 -- 172.21.15.44:0/2693069731 shutdown_connections 2024-06-03T01:13:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.276+0000 7ff775d9b700 1 -- 172.21.15.44:0/2693069731 wait complete. 2024-06-03T01:13:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.277+0000 7ff775d9b700 1 Processor -- start 2024-06-03T01:13:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.277+0000 7ff775d9b700 1 -- start start 2024-06-03T01:13:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.277+0000 7ff775d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.277+0000 7ff775d9b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7701017d0 con 0x7ff770106820 2024-06-03T01:13:48.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:37394/0 (socket says 172.21.15.44:37394) 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76f7fe700 1 -- 172.21.15.44:0/2689324985 learned_addr learned my addr 172.21.15.44:0/2689324985 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76f7fe700 1 -- 172.21.15.44:0/2689324985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600095f0 con 0x7ff770106820 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76f7fe700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff760005b40 tx=0x7ff760004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff76001d070 con 0x7ff770106820 2024-06-03T01:13:48.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7600044d0 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7700ffe00 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff760003910 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.278+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff770100280 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.279+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7ff760003b90 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.279+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff770105ad0 con 0x7ff770106820 2024-06-03T01:13:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.280+0000 7ff76d7fa700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 0x7ff75c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.280+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff76004c450 con 0x7ff770106820 2024-06-03T01:13:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.280+0000 7ff767fff700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 0x7ff75c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.280+0000 7ff767fff700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 0x7ff75c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff758006fd0 tx=0x7ff758006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:48.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.285+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff76002a950 con 0x7ff770106820 2024-06-03T01:13:48.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.574+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff770100530 con 0x7ff770106820 2024-06-03T01:13:48.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.575+0000 7ff76d7fa700 1 -- 172.21.15.44:0/2689324985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff76002a360 con 0x7ff770106820 2024-06-03T01:13:48.575 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:48.575 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:48.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 msgr2=0x7ff75c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:48.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 0x7ff75c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff758006fd0 tx=0x7ff758006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:48.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 msgr2=0x7ff7700ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:48.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff760005b40 tx=0x7ff760004c80 comp rx=0 tx=0).stop 2024-06-03T01:13:48.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 shutdown_connections 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75c0384b0 0x7ff75c03a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 --2- 172.21.15.44:0/2689324985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff770106820 0x7ff7700ff800 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 >> 172.21.15.44:0/2689324985 conn(0x7ff770075290 msgr2=0x7ff7700fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 shutdown_connections 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:48.577+0000 7ff775d9b700 1 -- 172.21.15.44:0/2689324985 wait complete. 2024-06-03T01:13:48.578 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: Deploying daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:13:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:48 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2689324985' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:49 smithi032 ceph-mon[29360]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:50.458 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:50.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:13:52.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.591+0000 7ff4ec274700 1 -- 172.21.15.44:0/1149181840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 msgr2=0x7ff4e40683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.591+0000 7ff4ec274700 1 --2- 172.21.15.44:0/1149181840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e40683a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009910 tx=0x7ff4e0009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 -- 172.21.15.44:0/1149181840 shutdown_connections 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 --2- 172.21.15.44:0/1149181840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e40683a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 -- 172.21.15.44:0/1149181840 >> 172.21.15.44:0/1149181840 conn(0x7ff4e4075290 msgr2=0x7ff4e40756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 -- 172.21.15.44:0/1149181840 shutdown_connections 2024-06-03T01:13:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 -- 172.21.15.44:0/1149181840 wait complete. 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.592+0000 7ff4ec274700 1 Processor -- start 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ec274700 1 -- start start 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ec274700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ec274700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e4198460 con 0x7ff4e4067fc0 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ea010700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ea010700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:37438/0 (socket says 172.21.15.44:37438) 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ea010700 1 -- 172.21.15.44:0/2447704646 learned_addr learned my addr 172.21.15.44:0/2447704646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.593+0000 7ff4ea010700 1 -- 172.21.15.44:0/2447704646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00095f0 con 0x7ff4e4067fc0 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4ea010700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005b40 tx=0x7ff4e0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4e001d070 con 0x7ff4e4067fc0 2024-06-03T01:13:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e4198660 con 0x7ff4e4067fc0 2024-06-03T01:13:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff4e00044d0 con 0x7ff4e4067fc0 2024-06-03T01:13:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4e0003930 con 0x7ff4e4067fc0 2024-06-03T01:13:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.594+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e4198a20 con 0x7ff4e4067fc0 2024-06-03T01:13:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.595+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e40623c0 con 0x7ff4e4067fc0 2024-06-03T01:13:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.595+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7ff4e0003bf0 con 0x7ff4e4067fc0 2024-06-03T01:13:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.595+0000 7ff4db7fe700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 0x7ff4d003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.595+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff4e004c710 con 0x7ff4e4067fc0 2024-06-03T01:13:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.596+0000 7ff4e980f700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 0x7ff4d003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.596+0000 7ff4e980f700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 0x7ff4d003a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.601+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff4e002a950 con 0x7ff4e4067fc0 2024-06-03T01:13:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.887+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff4e410a030 con 0x7ff4e4067fc0 2024-06-03T01:13:52.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.887+0000 7ff4db7fe700 1 -- 172.21.15.44:0/2447704646 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff4e002a360 con 0x7ff4e4067fc0 2024-06-03T01:13:52.887 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:52.888 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 msgr2=0x7ff4d003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 0x7ff4d003a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 msgr2=0x7ff4e4197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005b40 tx=0x7ff4e0004c80 comp rx=0 tx=0).stop 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 shutdown_connections 2024-06-03T01:13:52.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff4d00384b0 0x7ff4d003a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 --2- 172.21.15.44:0/2447704646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e4067fc0 0x7ff4e4197f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 >> 172.21.15.44:0/2447704646 conn(0x7ff4e4075290 msgr2=0x7ff4e40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 shutdown_connections 2024-06-03T01:13:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:52.890+0000 7ff4ec274700 1 -- 172.21.15.44:0/2447704646 wait complete. 2024-06-03T01:13:52.891 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:53.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:52 smithi032 ceph-mon[29360]: Deploying daemon mon.smithi044 on smithi044 2024-06-03T01:13:53.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:52 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2447704646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:13:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:53 smithi032 ceph-mon[29360]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:55.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:54 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:55.521 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:55.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:55.772 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:13:56.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:13:55 smithi032 ceph-mon[29360]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:57.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 -- 172.21.15.44:0/3876498913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 msgr2=0x7f3c70106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:57.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 --2- 172.21.15.44:0/3876498913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70106d00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009910 tx=0x7f3c60009c20 comp rx=0 tx=0).stop 2024-06-03T01:13:57.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 -- 172.21.15.44:0/3876498913 shutdown_connections 2024-06-03T01:13:57.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 --2- 172.21.15.44:0/3876498913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70106d00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:57.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 -- 172.21.15.44:0/3876498913 >> 172.21.15.44:0/3876498913 conn(0x7f3c70075340 msgr2=0x7f3c70075750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 -- 172.21.15.44:0/3876498913 shutdown_connections 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.335+0000 7f3c751ce700 1 -- 172.21.15.44:0/3876498913 wait complete. 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.336+0000 7f3c751ce700 1 Processor -- start 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.336+0000 7f3c751ce700 1 -- start start 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.336+0000 7f3c751ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.336+0000 7f3c751ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c70198590 con 0x7f3c70106920 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c6ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c6ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.44:35052/0 (socket says 172.21.15.44:35052) 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c6ed9d700 1 -- 172.21.15.44:0/2268550025 learned_addr learned my addr 172.21.15.44:0/2268550025 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c6ed9d700 1 -- 172.21.15.44:0/2268550025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600095f0 con 0x7f3c70106920 2024-06-03T01:13:57.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c6ed9d700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005b40 tx=0x7f3c60004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:57.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c6001d070 con 0x7f3c70106920 2024-06-03T01:13:57.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c70198790 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c600044d0 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.337+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c60003910 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.338+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c70198c10 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.339+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f3c60003b40 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.339+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c700623c0 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.339+0000 7f3c57fff700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 0x7f3c5803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.339+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3c6004c570 con 0x7f3c70106920 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.339+0000 7f3c6e59c700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 0x7f3c5803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.340+0000 7f3c6e59c700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 0x7f3c5803a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006fd0 tx=0x7f3c64006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:57.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.344+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3c6002a950 con 0x7f3c70106920 2024-06-03T01:13:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.627+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3c7019cb00 con 0x7f3c70106920 2024-06-03T01:13:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.628+0000 7f3c57fff700 1 -- 172.21.15.44:0/2268550025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3c6002a360 con 0x7f3c70106920 2024-06-03T01:13:57.629 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:13:57.629 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:11:10.279598Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 msgr2=0x7f3c5803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 0x7f3c5803a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006fd0 tx=0x7f3c64006e40 comp rx=0 tx=0).stop 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 msgr2=0x7f3c70198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005b40 tx=0x7f3c60004c80 comp rx=0 tx=0).stop 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 shutdown_connections 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f3c58038500 0x7f3c5803a9c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 --2- 172.21.15.44:0/2268550025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c70106920 0x7f3c70198050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 >> 172.21.15.44:0/2268550025 conn(0x7f3c70075340 msgr2=0x7f3c700fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 shutdown_connections 2024-06-03T01:13:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:13:57.630+0000 7f3c751ce700 1 -- 172.21.15.44:0/2268550025 wait complete. 2024-06-03T01:13:57.632 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-06-03T01:13:57.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: load: jerasure load: lrc 2024-06-03T01:13:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: RocksDB version: 7.9.2 2024-06-03T01:13:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Git sha 0 2024-06-03T01:13:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Compile date 2023-12-11 22:07:34 2024-06-03T01:13:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: DB SUMMARY 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: DB Session ID: ABKME137FVJIEW1UCRPY 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: CURRENT file: CURRENT 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi044/store.db dir, Total Num: 0, files: 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi044/store.db: 000004.log size: 511 ; 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.error_if_exists: 0 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.create_if_missing: 0 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.paranoid_checks: 1 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T01:13:57.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.env: 0x55e9a4249720 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.info_log: 0x55e9a63172e0 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.statistics: (nil) 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.use_fsync: 0 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_log_file_size: 0 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T01:13:57.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_fallocate: 1 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.use_direct_reads: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.db_log_dir: 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.wal_dir: 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T01:13:57.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.write_buffer_manager: 0x55e9a5588140 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T01:13:57.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.unordered_write: 0 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.row_cache: None 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.wal_filter: None 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T01:13:57.978 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.two_write_queues: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.wal_compression: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.atomic_flush: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.log_readahead_size: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T01:13:57.979 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_background_jobs: 2 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_background_compactions: -1 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_subcompactions: 1 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T01:13:57.980 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_open_files: -1 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_background_flushes: -1 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Compression algorithms supported: 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kZSTD supported: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kXpressCompression supported: 0 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kZlibCompression supported: 1 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kSnappyCompression supported: 1 2024-06-03T01:13:57.981 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kLZ4Compression supported: 1 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: kBZip2Compression supported: 0 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000005 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.merge_operator: 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_filter: None 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T01:13:57.982 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e9a63173e0) 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_type: 0 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_index_type: 0 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_shortening: 1 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: checksum: 4 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: no_block_cache: 0 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache: 0x55e9a5629350 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_options: 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: capacity : 536870912 2024-06-03T01:13:57.983 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_shard_bits : 4 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: strict_capacity_limit : 0 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_compressed: (nil) 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: persistent_cache: (nil) 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size: 4096 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size_deviation: 10 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_restart_interval: 16 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_block_restart_interval: 1 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_block_size: 4096 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: partition_filters: 0 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: use_delta_encoding: 1 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: filter_policy: bloomfilter 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: whole_key_filtering: 1 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: verify_compression: 0 2024-06-03T01:13:57.984 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: format_version: 5 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_index_compression: 1 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_align: 0 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_auto_readahead_size: 262144 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: prepopulate_block_cache: 0 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: initial_auto_readahead_size: 8192 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression: NoCompression 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T01:13:57.985 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.num_levels: 7 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T01:13:57.986 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T01:13:57.987 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T01:13:57.988 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.table_properties_collectors: 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.inplace_update_support: 0 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T01:13:57.989 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.bloom_locality: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.max_successive_merges: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.ttl: 2592000 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enable_blob_files: false 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.min_blob_size: 0 2024-06-03T01:13:57.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9ff58016-de4e-4565-a30a-178f41e925c3 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377237684883, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377237685295, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717377237, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9ff58016-de4e-4565-a30a-178f41e925c3", "db_session_id": "ABKME137FVJIEW1UCRPY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-03T01:13:57.991 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377237685380, "job": 1, "event": "recovery_finished"} 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi044/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e9a56c6000 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: DB pointer 0x55e9a56b2000 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** DB Stats ** 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:13:57.992 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T01:13:57.993 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache BinnedLRUCache@0x55e9a5629350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-03T01:13:57.994 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044 does not exist in monmap, will attempt to join an existing cluster 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: using public_addr v2:172.21.15.44:0/0 -> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: starting mon.smithi044 rank -1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi044 fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(???) e0 preinit fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).mds e1 new map 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).mds e1 print_map 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: e1 2024-06-03T01:13:57.995 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: legacy client fscid: -1 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: No filesystems configured 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T01:13:57.996 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expand map: {default=false} 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta from 'false' to 'false' 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expanded map: {default=false} 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expand map: {default=info} 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta from 'info' to 'info' 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expanded map: {default=info} 2024-06-03T01:13:57.997 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expand map: {default=daemon} 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta from 'daemon' to 'daemon' 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expanded map: {default=daemon} 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expand map: {default=debug} 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta from 'debug' to 'debug' 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: expand_channel_meta expanded map: {default=debug} 2024-06-03T01:13:57.998 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:57 smithi044 ceph-mon[36956]: mon.smithi044@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-06-03T01:13:58.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:58 smithi044 bash[36905]: b1e9d52e00f738c55ce33d9f09abb8deff6b77a9a3bde02da99b176ff72baca4 2024-06-03T01:13:58.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:13:58 smithi044 systemd[1]: Started Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:13:59.459 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-03T01:13:59.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T01:13:59.682 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: mon.smithi032 calling monitor election 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: mon.smithi044 calling monitor election 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi044=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: fsmap 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: osdmap e5: 0 total, 0 up, 0 in 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: mgrmap e17: smithi032.kfcfaa(active, since 33s) 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: overall HEALTH_OK 2024-06-03T01:14:03.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:02 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: mon.smithi032 calling monitor election 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: mon.smithi044 calling monitor election 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi044=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: fsmap 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: osdmap e5: 0 total, 0 up, 0 in 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: mgrmap e17: smithi032.kfcfaa(active, since 33s) 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: overall HEALTH_OK 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1689415121' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:14:03.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:03.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:02 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:04.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:03 smithi032 ceph-mon[29360]: mgrmap e18: smithi032.kfcfaa(active, since 33s), standbys: smithi044.wszjlw 2024-06-03T01:14:04.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:14:04.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:03 smithi032 ceph-mon[29360]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:04.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:04.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 -- 172.21.15.44:0/2435856291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 msgr2=0x7f8fa0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/2435856291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fa0005a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4076780 tx=0x7f8f9c00feb0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 -- 172.21.15.44:0/2435856291 shutdown_connections 2024-06-03T01:14:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/2435856291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fa0005a50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 -- 172.21.15.44:0/2435856291 >> 172.21.15.44:0/2435856291 conn(0x7f8fb4075340 msgr2=0x7f8fb4075750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 -- 172.21.15.44:0/2435856291 shutdown_connections 2024-06-03T01:14:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 -- 172.21.15.44:0/2435856291 wait complete. 2024-06-03T01:14:04.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:03 smithi044 ceph-mon[36956]: mgrmap e18: smithi032.kfcfaa(active, since 33s), standbys: smithi044.wszjlw 2024-06-03T01:14:04.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:14:04.099 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:03 smithi044 ceph-mon[36956]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:04.099 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:04.099 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:14:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.097+0000 7f8fbaf99700 1 Processor -- start 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fbaf99700 1 -- start start 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fbaf99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fb4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fbaf99700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fbaf99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4196a70 con 0x7f8fa00035c0 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fbaf99700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4196be0 con 0x7f8fb4196450 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42010/0 (socket says 172.21.15.44:42010) 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 -- 172.21.15.44:0/151704638 learned_addr learned my addr 172.21.15.44:0/151704638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb8d35700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fb4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 -- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 msgr2=0x7f8fb419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T01:14:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 -- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 msgr2=0x7f8fb419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb3fff700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb8d35700 1 -- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 msgr2=0x7f8fb419a8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb8d35700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.098+0000 7f8fb8d35700 1 -- 172.21.15.44:0/151704638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c00f900 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fb8d35700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fb4195f10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c010010 tx=0x7f8f9c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f9c013960 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f9c004030 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb419aec0 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f9c011c50 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.099+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb419b410 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.100+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb40623c0 con 0x7f8fa00035c0 2024-06-03T01:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.101+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f8f9c0041a0 con 0x7f8fa00035c0 2024-06-03T01:14:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.101+0000 7f8fb1ffb700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 0x7f8f9406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.101+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f8f9c08d120 con 0x7f8fa00035c0 2024-06-03T01:14:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.101+0000 7f8fb3fff700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 0x7f8f9406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.102+0000 7f8fb3fff700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 0x7f8f9406ea20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009400 tx=0x7f8fa4007040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.105+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f9c058a00 con 0x7f8fa00035c0 2024-06-03T01:14:04.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.393+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8fb419b680 con 0x7f8fa00035c0 2024-06-03T01:14:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.393+0000 7f8fb1ffb700 1 -- 172.21.15.44:0/151704638 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7f8f9c031020 con 0x7f8fa00035c0 2024-06-03T01:14:04.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-06-03T01:14:04.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","modified":"2024-06-03T01:13:57.712306Z","created":"2024-06-03T01:11:10.279598Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi044","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T01:14:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 msgr2=0x7f8f9406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 0x7f8f9406ea20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009400 tx=0x7f8fa4007040 comp rx=0 tx=0).stop 2024-06-03T01:14:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 msgr2=0x7f8fb4195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fb4195f10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c010010 tx=0x7f8f9c004f90 comp rx=0 tx=0).stop 2024-06-03T01:14:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 shutdown_connections 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa00035c0 0x7f8fb4195f10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f9406c560 0x7f8f9406ea20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 --2- 172.21.15.44:0/151704638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fb4196450 0x7f8fb419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 >> 172.21.15.44:0/151704638 conn(0x7f8fb4075340 msgr2=0x7f8fb418e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 shutdown_connections 2024-06-03T01:14:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:14:04.396+0000 7f8fbaf99700 1 -- 172.21.15.44:0/151704638 wait complete. 2024-06-03T01:14:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 2 2024-06-03T01:14:05.046 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:04 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:05.047 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:04 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/151704638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:14:05.268 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-03T01:14:05.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-03T01:14:05.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:04 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:05.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:04 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/151704638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T01:14:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:14:06.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.203 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:14:07.204 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:14:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:14:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:14:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:14:07.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 -- 172.21.15.32:0/139568483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 msgr2=0x7ff300105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 --2- 172.21.15.32:0/139568483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300105b60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b0a0 tx=0x7ff2f000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 -- 172.21.15.32:0/139568483 shutdown_connections 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 --2- 172.21.15.32:0/139568483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300105b60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 -- 172.21.15.32:0/139568483 >> 172.21.15.32:0/139568483 conn(0x7ff300075290 msgr2=0x7ff3000756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 -- 172.21.15.32:0/139568483 shutdown_connections 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.308+0000 7ff307f5d700 1 -- 172.21.15.32:0/139568483 wait complete. 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 Processor -- start 2024-06-03T01:14:07.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 -- start start 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff300197230 0x7ff3001976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30019fd60 con 0x7ff300105780 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff307f5d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30019fea0 con 0x7ff300197230 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49152/0 (socket says 172.21.15.32:49152) 2024-06-03T01:14:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 -- 172.21.15.32:0/2215891926 learned_addr learned my addr 172.21.15.32:0/2215891926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:14:07.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 -- 172.21.15.32:0/2215891926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff300197230 msgr2=0x7ff3001976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:07.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff300197230 0x7ff3001976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.309+0000 7ff305cf9700 1 -- 172.21.15.32:0/2215891926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0009d00 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff305cf9700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d010 tx=0x7ff2f00076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f000e040 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3001a00a0 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2f0007d10 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f001d840 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.310+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3001a0560 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.311+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3000623c0 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.311+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7ff2f0004810 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.312+0000 7ff2feffd700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 0x7ff2e806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.312+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff2f008bfa0 con 0x7ff300105780 2024-06-03T01:14:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.312+0000 7ff3054f8700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 0x7ff2e806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.312+0000 7ff3054f8700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 0x7ff2e806ea80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff300069070 tx=0x7ff2f4006dd0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.316+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff2f00bf0e0 con 0x7ff300105780 2024-06-03T01:14:07.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.518+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff300068580 con 0x7ff300105780 2024-06-03T01:14:07.519 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:14:07.519 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-06-03T01:14:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:14:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-06-03T01:14:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.519+0000 7ff2feffd700 1 -- 172.21.15.32:0/2215891926 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7ff2f00170f0 con 0x7ff300105780 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 msgr2=0x7ff2e806ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 0x7ff2e806ea80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff300069070 tx=0x7ff2f4006dd0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 msgr2=0x7ff300196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d010 tx=0x7ff2f00076d0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 shutdown_connections 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff300105780 0x7ff300196cf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff2e806c5c0 0x7ff2e806ea80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 --2- 172.21.15.32:0/2215891926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff300197230 0x7ff3001976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:07.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 >> 172.21.15.32:0/2215891926 conn(0x7ff300075290 msgr2=0x7ff300101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 shutdown_connections 2024-06-03T01:14:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:07.521+0000 7ff307f5d700 1 -- 172.21.15.32:0/2215891926 wait complete. 2024-06-03T01:14:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:07 smithi044 ceph-mon[36956]: Reconfiguring mon.smithi032 (unknown last config time)... 2024-06-03T01:14:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:07 smithi044 ceph-mon[36956]: Reconfiguring daemon mon.smithi032 on smithi032 2024-06-03T01:14:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:07 smithi044 ceph-mon[36956]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:07 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2215891926' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:08.234 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:07 smithi032 ceph-mon[29360]: Reconfiguring mon.smithi032 (unknown last config time)... 2024-06-03T01:14:08.234 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:07 smithi032 ceph-mon[29360]: Reconfiguring daemon mon.smithi032 on smithi032 2024-06-03T01:14:08.234 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:07 smithi032 ceph-mon[29360]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:08.234 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:07 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2215891926' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:08.255 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-03T01:14:08.255 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:14:08.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T01:14:08.328 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:14:08.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:14:08.394 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:14:08.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T01:14:08.428 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:14:08.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:14:08.498 INFO:tasks.cephadm:Deploying OSDs... 2024-06-03T01:14:08.498 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:14:08.499 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T01:14:08.515 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T01:14:08.515 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-06-03 01:13:20.214550796 +0000 2024-06-03T01:14:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-06-03 01:07:52.664030024 +0000 2024-06-03T01:14:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-06-03 01:07:52.664030024 +0000 2024-06-03T01:14:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-06-03 01:07:52.664030024 +0000 2024-06-03T01:14:08.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T01:14:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-06-03T01:14:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-06-03T01:14:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 8.998e-05 s, 5.7 MB/s 2024-06-03T01:14:08.637 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T01:14:08.695 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-06-03 01:13:20.215550777 +0000 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-06-03 01:07:52.990023262 +0000 2024-06-03T01:14:08.753 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-06-03 01:07:52.990023262 +0000 2024-06-03T01:14:08.754 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-06-03 01:07:52.990023262 +0000 2024-06-03T01:14:08.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T01:14:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-06-03T01:14:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-06-03T01:14:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000110542 s, 4.6 MB/s 2024-06-03T01:14:08.819 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T01:14:08.877 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 708 Links: 1 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:08.933 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-06-03 01:13:20.215550777 +0000 2024-06-03T01:14:08.934 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-06-03 01:07:53.314016540 +0000 2024-06-03T01:14:08.934 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-06-03 01:07:53.314016540 +0000 2024-06-03T01:14:08.934 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-06-03 01:07:53.314016540 +0000 2024-06-03T01:14:08.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T01:14:08.997 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-06-03T01:14:08.997 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-06-03T01:14:08.997 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.1687e-05 s, 5.6 MB/s 2024-06-03T01:14:08.999 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T01:14:09.055 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 727 Links: 1 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-06-03 01:13:20.215550777 +0000 2024-06-03T01:14:09.112 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-06-03 01:07:53.634009902 +0000 2024-06-03T01:14:09.113 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-06-03 01:07:53.634009902 +0000 2024-06-03T01:14:09.113 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-06-03 01:07:53.634009902 +0000 2024-06-03T01:14:09.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T01:14:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-06-03T01:14:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-06-03T01:14:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.621e-05 s, 5.3 MB/s 2024-06-03T01:14:09.179 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T01:14:09.236 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:14:09.236 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T01:14:09.256 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T01:14:09.256 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-06-03T01:14:09.316 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T01:14:09.316 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:09.316 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-03T01:14:09.316 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:09.317 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:09.317 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-06-03 01:13:55.773961866 +0000 2024-06-03T01:14:09.317 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-06-03 01:07:52.677296138 +0000 2024-06-03T01:14:09.317 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-06-03 01:07:52.677296138 +0000 2024-06-03T01:14:09.317 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-06-03 01:07:52.677296138 +0000 2024-06-03T01:14:09.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T01:14:09.386 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-06-03T01:14:09.386 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-06-03T01:14:09.386 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000233694 s, 2.2 MB/s 2024-06-03T01:14:09.392 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T01:14:09.451 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-06-03 01:13:55.774961851 +0000 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-06-03 01:07:53.020290600 +0000 2024-06-03T01:14:09.510 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-06-03 01:07:53.020290600 +0000 2024-06-03T01:14:09.511 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-06-03 01:07:53.020290600 +0000 2024-06-03T01:14:09.511 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T01:14:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-06-03T01:14:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-06-03T01:14:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000209117 s, 2.4 MB/s 2024-06-03T01:14:09.578 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T01:14:09.637 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-06-03 01:13:55.774961851 +0000 2024-06-03T01:14:09.695 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-06-03 01:07:53.362285079 +0000 2024-06-03T01:14:09.696 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-06-03 01:07:53.362285079 +0000 2024-06-03T01:14:09.696 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-06-03 01:07:53.362285079 +0000 2024-06-03T01:14:09.696 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T01:14:09.761 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-06-03T01:14:09.762 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-06-03T01:14:09.762 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000212873 s, 2.4 MB/s 2024-06-03T01:14:09.763 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T01:14:09.820 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: Reconfiguring mgr.smithi032.kfcfaa (unknown last config time)... 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:14:09.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:09.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: Reconfiguring daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:14:09.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:09 smithi032 ceph-mon[29360]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-06-03 01:13:55.774961851 +0000 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-06-03 01:07:53.706279524 +0000 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-06-03 01:07:53.706279524 +0000 2024-06-03T01:14:09.878 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-06-03 01:07:53.706279524 +0000 2024-06-03T01:14:09.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T01:14:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-06-03T01:14:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-06-03T01:14:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000195972 s, 2.6 MB/s 2024-06-03T01:14:09.944 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T01:14:10.001 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2024-06-03T01:14:10.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T01:14:10.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: Reconfiguring mgr.smithi032.kfcfaa (unknown last config time)... 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: Reconfiguring daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:14:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:09 smithi044 ceph-mon[36956]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: Reconfiguring ceph-exporter.smithi032 (monmap changed)... 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: Reconfiguring daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:14:11.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:11 smithi032 ceph-mon[29360]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:11.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:11.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: Reconfiguring ceph-exporter.smithi032 (monmap changed)... 2024-06-03T01:14:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:14:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: Reconfiguring daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:14:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:11 smithi044 ceph-mon[36956]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00844449 s, 1.2 GB/s 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 2024-06-03T01:14:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-06-03T01:14:13.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: Reconfiguring crash.smithi032 (monmap changed)... 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: Reconfiguring daemon crash.smithi032 on smithi032 2024-06-03T01:14:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:13 smithi044 ceph-mon[36956]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:13.805 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: Reconfiguring crash.smithi032 (monmap changed)... 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: Reconfiguring daemon crash.smithi032 on smithi032 2024-06-03T01:14:13.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:13 smithi032 ceph-mon[29360]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:14.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2024-06-03T01:14:14.520 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:14:14.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:14.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:15.612 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:15 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:15.612 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:15 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:15.612 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:15 smithi032 ceph-mon[29360]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-06-03T01:14:15.612 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:15 smithi032 ceph-mon[29360]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-06-03T01:14:15.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:15 smithi032 ceph-mon[29360]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:15.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:15 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:15 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:15 smithi044 ceph-mon[36956]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-06-03T01:14:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:15 smithi044 ceph-mon[36956]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-06-03T01:14:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:15 smithi044 ceph-mon[36956]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:17.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 -- 172.21.15.32:0/4124085824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 msgr2=0x7f6a24101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:17.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 --2- 172.21.15.32:0/4124085824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24101220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009910 tx=0x7f6a18009c20 comp rx=0 tx=0).stop 2024-06-03T01:14:17.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 -- 172.21.15.32:0/4124085824 shutdown_connections 2024-06-03T01:14:17.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 --2- 172.21.15.32:0/4124085824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24101220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:17.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 --2- 172.21.15.32:0/4124085824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a240fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:17.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 -- 172.21.15.32:0/4124085824 >> 172.21.15.32:0/4124085824 conn(0x7f6a240fb160 msgr2=0x7f6a240fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:17.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 -- 172.21.15.32:0/4124085824 shutdown_connections 2024-06-03T01:14:17.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 -- 172.21.15.32:0/4124085824 wait complete. 2024-06-03T01:14:17.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.214+0000 7f6a293bc700 1 Processor -- start 2024-06-03T01:14:17.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a293bc700 1 -- start start 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a293bc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a241967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a293bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a293bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24197330 con 0x7f6a24100dc0 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a293bc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24197470 con 0x7f6a240ffbc0 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51776/0 (socket says 172.21.15.32:51776) 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 -- 172.21.15.32:0/3776664649 learned_addr learned my addr 172.21.15.32:0/3776664649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a22ffd700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a241967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 -- 172.21.15.32:0/3776664649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 msgr2=0x7f6a241967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a241967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 -- 172.21.15.32:0/3776664649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180095f0 con 0x7f6a24100dc0 2024-06-03T01:14:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a22ffd700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a241967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.215+0000 7f6a227fc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b5c0 tx=0x7f6a1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.216+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a1801d070 con 0x7f6a24100dc0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.216+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a1800f460 con 0x7f6a24100dc0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.216+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2419bec0 con 0x7f6a24100dc0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.216+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a18021640 con 0x7f6a24100dc0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.216+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2419c380 con 0x7f6a24100dc0 2024-06-03T01:14:17.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.217+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24068af0 con 0x7f6a24100dc0 2024-06-03T01:14:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.217+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f6a1800f5d0 con 0x7f6a24100dc0 2024-06-03T01:14:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.218+0000 7f6a0bfff700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 0x7f6a0c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.218+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6a1808b930 con 0x7f6a24100dc0 2024-06-03T01:14:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.218+0000 7f6a22ffd700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 0x7f6a0c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.218+0000 7f6a22ffd700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 0x7f6a0c06e930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005fd0 tx=0x7f6a14005f00 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:17.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.222+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a1805ad00 con 0x7f6a24100dc0 2024-06-03T01:14:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:17.436+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a24069560 con 0x7f6a0c06c470 2024-06-03T01:14:18.027 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:17 smithi032 ceph-mon[29360]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:18.028 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:14:18.028 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:14:18.028 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:17 smithi044 ceph-mon[36956]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:14:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:14:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:19.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:18 smithi044 ceph-mon[36956]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:14:19.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:18 smithi032 ceph-mon[29360]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:14:20.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:19 smithi032 ceph-mon[29360]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:20.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:19 smithi044 ceph-mon[36956]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:21 smithi044 ceph-mon[36956]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:21 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:22.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:21 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:21 smithi032 ceph-mon[29360]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:21 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:21 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-06-03T01:14:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-06-03T01:14:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/555838907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddd78453-9794-4382-b3f3-6dbe73ec2d0f"}]: dispatch 2024-06-03T01:14:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/555838907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddd78453-9794-4382-b3f3-6dbe73ec2d0f"}]': finished 2024-06-03T01:14:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T01:14:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:22 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/555838907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddd78453-9794-4382-b3f3-6dbe73ec2d0f"}]: dispatch 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/555838907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddd78453-9794-4382-b3f3-6dbe73ec2d0f"}]': finished 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T01:14:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:22 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:24.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:23 smithi044 ceph-mon[36956]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:24.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:23 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1227538066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:14:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:23 smithi032 ceph-mon[29360]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:23 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1227538066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:14:25.967 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:25 smithi032 ceph-mon[29360]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:25.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:25 smithi044 ceph-mon[36956]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:27.952 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:27 smithi032 ceph-mon[29360]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:27 smithi044 ceph-mon[36956]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:30.130 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:29 smithi032 ceph-mon[29360]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:29 smithi044 ceph-mon[36956]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:32.191 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:31 smithi032 ceph-mon[29360]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:32.191 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:32.191 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:32.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:31 smithi044 ceph-mon[36956]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:32.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:32.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:33.048 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:32 smithi032 ceph-mon[29360]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:14:33.048 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:32 smithi032 ceph-mon[29360]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:14:33.048 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:32 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:14:33.048 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:32 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:32 smithi044 ceph-mon[36956]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:14:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:32 smithi044 ceph-mon[36956]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:14:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:32 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:14:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:32 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:34.208 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:33 smithi032 ceph-mon[29360]: Deploying daemon osd.0 on smithi032 2024-06-03T01:14:34.208 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:33 smithi032 ceph-mon[29360]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:33 smithi044 ceph-mon[36956]: Deploying daemon osd.0 on smithi032 2024-06-03T01:14:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:33 smithi044 ceph-mon[36956]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:35.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:35 smithi044 ceph-mon[36956]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:35.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:35 smithi032 ceph-mon[29360]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:14:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:37 smithi032 ceph-mon[29360]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:37.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:14:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:37 smithi044 ceph-mon[36956]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:39.190 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: Reconfiguring crash.smithi044 (monmap changed)... 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: Reconfiguring daemon crash.smithi044 on smithi044 2024-06-03T01:14:39.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:39 smithi044 ceph-mon[36956]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:39.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:39.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:39.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: Reconfiguring crash.smithi044 (monmap changed)... 2024-06-03T01:14:39.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:14:39.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:39.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: Reconfiguring daemon crash.smithi044 on smithi044 2024-06-03T01:14:39.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:39 smithi032 ceph-mon[29360]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: Reconfiguring mgr.smithi044.wszjlw (monmap changed)... 2024-06-03T01:14:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:14:40.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:14:40.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:40.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:40 smithi032 ceph-mon[29360]: Reconfiguring daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:14:40.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: Reconfiguring mgr.smithi044.wszjlw (monmap changed)... 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:40 smithi044 ceph-mon[36956]: Reconfiguring daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: Reconfiguring mon.smithi044 (monmap changed)... 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: Reconfiguring daemon mon.smithi044 on smithi044 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:41.911 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: Reconfiguring mon.smithi044 (monmap changed)... 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: Reconfiguring daemon mon.smithi044 on smithi044 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:42.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:42.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.393 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.393 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:43.395 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:43 smithi032 ceph-mon[29360]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:14:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:43.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:43 smithi044 ceph-mon[36956]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:44.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.066+0000 7f6a0bfff700 1 -- 172.21.15.32:0/3776664649 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a24069560 con 0x7f6a0c06c470 2024-06-03T01:14:45.067 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 msgr2=0x7f6a0c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 0x7f6a0c06e930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005fd0 tx=0x7f6a14005f00 comp rx=0 tx=0).stop 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 msgr2=0x7f6a24196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b5c0 tx=0x7f6a1800be20 comp rx=0 tx=0).stop 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 shutdown_connections 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a0c06c470 0x7f6a0c06e930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a240ffbc0 0x7f6a241967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 --2- 172.21.15.32:0/3776664649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a24100dc0 0x7f6a24196d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 >> 172.21.15.32:0/3776664649 conn(0x7f6a240fb160 msgr2=0x7f6a24068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 shutdown_connections 2024-06-03T01:14:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:45.069+0000 7f6a293bc700 1 -- 172.21.15.32:0/3776664649 wait complete. 2024-06-03T01:14:45.666 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.0.service 2024-06-03T01:14:45.669 INFO:tasks.cephadm:Deploying osd.1 on smithi032 with /dev/vg_nvme/lv_3... 2024-06-03T01:14:45.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T01:14:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:45 smithi044 ceph-mon[36956]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:45 smithi044 ceph-mon[36956]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T01:14:46.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:45 smithi032 ceph-mon[29360]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:46.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:46.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:46.071 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:45 smithi032 ceph-mon[29360]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T01:14:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:14:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-mon[29360]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T01:14:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-mon[29360]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T01:14:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:47.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-mon[29360]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:14:47.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-mon[29360]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:47.336 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[48123]: 2024-06-03T01:14:47.063+0000 7f5ff4ac9700 -1 osd.0 0 waiting for initial osdmap 2024-06-03T01:14:47.336 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:14:47 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[48123]: 2024-06-03T01:14:47.066+0000 7f5fee8ba700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:14:47.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:47 smithi044 ceph-mon[36956]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T01:14:47.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:47 smithi044 ceph-mon[36956]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T01:14:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:47 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:47 smithi044 ceph-mon[36956]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:14:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:47 smithi044 ceph-mon[36956]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:48.205 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:48 smithi032 ceph-mon[29360]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:14:48.205 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:48 smithi032 ceph-mon[29360]: osdmap e8: 1 total, 0 up, 1 in 2024-06-03T01:14:48.206 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:48.206 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:48 smithi044 ceph-mon[36956]: from='osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:14:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:48 smithi044 ceph-mon[36956]: osdmap e8: 1 total, 0 up, 1 in 2024-06-03T01:14:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:48 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:48 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00999432 s, 1.0 GB/s 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 2024-06-03T01:14:49.191 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-06-03T01:14:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:14:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:14:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] boot 2024-06-03T01:14:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T01:14:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:49.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:49 smithi032 ceph-mon[29360]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:49.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:14:49.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:14:49.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:49.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: osd.0 [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] boot 2024-06-03T01:14:49.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T01:14:49.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:14:49.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:49 smithi044 ceph-mon[36956]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T01:14:49.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2024-06-03T01:14:50.178 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:50 smithi032 ceph-mon[29360]: osdmap e10: 1 total, 1 up, 1 in 2024-06-03T01:14:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:50 smithi044 ceph-mon[36956]: osdmap e10: 1 total, 1 up, 1 in 2024-06-03T01:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:14:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:51 smithi032 ceph-mon[29360]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:51.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:51 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:51.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:51 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:51.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:51 smithi044 ceph-mon[36956]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 -- 172.21.15.32:0/3842235619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 msgr2=0x7f1ed81064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 --2- 172.21.15.32:0/3842235619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed81064d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009910 tx=0x7f1ecc009c20 comp rx=0 tx=0).stop 2024-06-03T01:14:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 -- 172.21.15.32:0/3842235619 shutdown_connections 2024-06-03T01:14:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 --2- 172.21.15.32:0/3842235619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed81064d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 --2- 172.21.15.32:0/3842235619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 0x7f1ed8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 -- 172.21.15.32:0/3842235619 >> 172.21.15.32:0/3842235619 conn(0x7f1ed80fb160 msgr2=0x7f1ed80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 -- 172.21.15.32:0/3842235619 shutdown_connections 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.213+0000 7f1ede03b700 1 -- 172.21.15.32:0/3842235619 wait complete. 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.214+0000 7f1ede03b700 1 Processor -- start 2024-06-03T01:14:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.214+0000 7f1ede03b700 1 -- start start 2024-06-03T01:14:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ede03b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 0x7f1ed8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ede03b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ede03b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed81994e0 con 0x7f1ed81040e0 2024-06-03T01:14:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ede03b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8199620 con 0x7f1ed81017b0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34274/0 (socket says 172.21.15.32:34274) 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 -- 172.21.15.32:0/1763742585 learned_addr learned my addr 172.21.15.32:0/1763742585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed77fe700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 0x7f1ed8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 -- 172.21.15.32:0/1763742585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 msgr2=0x7f1ed8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 0x7f1ed8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 -- 172.21.15.32:0/1763742585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc0095f0 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed6ffd700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005b40 tx=0x7f1ecc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ecc01d070 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.215+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed819e070 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.216+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc022470 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.216+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ecc00f690 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.216+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed819e560 con 0x7f1ed81040e0 2024-06-03T01:14:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.216+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8068af0 con 0x7f1ed81040e0 2024-06-03T01:14:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.217+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f1ecc022a10 con 0x7f1ed81040e0 2024-06-03T01:14:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.218+0000 7f1ed4ff9700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 0x7f1ec006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.218+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f1ecc08bea0 con 0x7f1ed81040e0 2024-06-03T01:14:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.218+0000 7f1ed77fe700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 0x7f1ec006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.218+0000 7f1ed77fe700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 0x7f1ec006e930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005950 tx=0x7f1ec800a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.223+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1ecc05b040 con 0x7f1ed81040e0 2024-06-03T01:14:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:14:52.437+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ed8100e70 con 0x7f1ec006c470 2024-06-03T01:14:52.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:52 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:14:52.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:52 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:14:52.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:52 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:52 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:14:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:52 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:14:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:52 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:53.830 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:53 smithi032 ceph-mon[29360]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:14:53.830 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:53 smithi032 ceph-mon[29360]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:53.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:53 smithi044 ceph-mon[36956]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:14:53.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:53 smithi044 ceph-mon[36956]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:55.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:55.741 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.741 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:55.864 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:55.865 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi032 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi032 to 11844M 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1886969142' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb1be659-330d-40c2-9e80-9430b3bd8a5d"}]: dispatch 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1886969142' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb1be659-330d-40c2-9e80-9430b3bd8a5d"}]': finished 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T01:14:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi032 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi032 to 11844M 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1886969142' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb1be659-330d-40c2-9e80-9430b3bd8a5d"}]: dispatch 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1886969142' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb1be659-330d-40c2-9e80-9430b3bd8a5d"}]': finished 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T01:14:56.976 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:14:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:57 smithi044 ceph-mon[36956]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:57 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:57 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:57 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1190705060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:14:57.984 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:57 smithi032 ceph-mon[29360]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:14:57.984 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:57 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:57.984 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:57 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:14:57.984 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:57 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1190705060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:14:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:58 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:14:59.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:58 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:14:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:00.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:00.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:00.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:00.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:00.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:00.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:14:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:02.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:01 smithi044 ceph-mon[36956]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:02.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:01 smithi032 ceph-mon[29360]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:04.215 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:03 smithi032 ceph-mon[29360]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:04.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:15:04.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:03 smithi044 ceph-mon[36956]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:15:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:04 smithi044 ceph-mon[36956]: Deploying daemon osd.1 on smithi032 2024-06-03T01:15:05.262 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:04 smithi032 ceph-mon[29360]: Deploying daemon osd.1 on smithi032 2024-06-03T01:15:06.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:05 smithi044 ceph-mon[36956]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:06.249 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:05 smithi032 ceph-mon[29360]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:06 smithi044 ceph-mon[36956]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:06 smithi032 ceph-mon[29360]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:09.917 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:09.917 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:09.917 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:09 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:09.917 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:09 smithi044 ceph-mon[36956]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:09.945 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:09.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:09.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:09 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:09.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:09 smithi032 ceph-mon[29360]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:11.578 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:11.578 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:11.578 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:11 smithi032 ceph-mon[29360]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:11.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:11.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:11.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:11 smithi044 ceph-mon[36956]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.911+0000 7f1ed4ff9700 1 -- 172.21.15.32:0/1763742585 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ed8100e70 con 0x7f1ec006c470 2024-06-03T01:15:11.912 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 1 on host 'smithi032' 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 msgr2=0x7f1ec006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 0x7f1ec006e930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005950 tx=0x7f1ec800a300 comp rx=0 tx=0).stop 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 msgr2=0x7f1ed8198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005b40 tx=0x7f1ecc00bd70 comp rx=0 tx=0).stop 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 shutdown_connections 2024-06-03T01:15:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f1ec006c470 0x7f1ec006e930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ed81017b0 0x7f1ed8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 --2- 172.21.15.32:0/1763742585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed81040e0 0x7f1ed8198ec0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 >> 172.21.15.32:0/1763742585 conn(0x7f1ed80fb160 msgr2=0x7f1ed8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 shutdown_connections 2024-06-03T01:15:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:11.914+0000 7f1ede03b700 1 -- 172.21.15.32:0/1763742585 wait complete. 2024-06-03T01:15:12.519 DEBUG:teuthology.orchestra.run.smithi032:osd.1> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service 2024-06-03T01:15:12.522 INFO:tasks.cephadm:Deploying osd.2 on smithi032 with /dev/vg_nvme/lv_2... 2024-06-03T01:15:12.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T01:15:12.760 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:12 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:15:12.559+0000 7f0d77112640 -1 osd.1 0 log_to_monitors true 2024-06-03T01:15:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:12 smithi032 ceph-mon[29360]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T01:15:13.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:13.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:13.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:12 smithi044 ceph-mon[36956]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T01:15:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:15:13.955 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-mon[29360]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:13.955 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-mon[29360]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T01:15:13.956 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-mon[29360]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T01:15:13.956 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:13.956 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-mon[29360]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:15:13.956 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:15:13.896+0000 7f0d6d778700 -1 osd.1 0 waiting for initial osdmap 2024-06-03T01:15:13.956 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:13 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:15:13.902+0000 7f0d64563700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:15:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:13 smithi044 ceph-mon[36956]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:13 smithi044 ceph-mon[36956]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T01:15:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:13 smithi044 ceph-mon[36956]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T01:15:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:13 smithi044 ceph-mon[36956]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:15:15.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:14 smithi044 ceph-mon[36956]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:15:15.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:14 smithi044 ceph-mon[36956]: osdmap e13: 2 total, 1 up, 2 in 2024-06-03T01:15:15.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:15.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:15.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:15.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:14 smithi032 ceph-mon[29360]: from='osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:15:15.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:14 smithi032 ceph-mon[29360]: osdmap e13: 2 total, 1 up, 2 in 2024-06-03T01:15:15.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:15.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:15.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:16.089 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T01:15:16.089 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T01:15:16.089 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-06-03T01:15:16.089 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-06-03T01:15:16.089 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0106975 s, 980 MB/s 2024-06-03T01:15:16.090 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-06-03T01:15:16.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:15:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:15:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] boot 2024-06-03T01:15:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T01:15:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:15 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: osd.1 [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] boot 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T01:15:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:15 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:15:17.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:16 smithi044 ceph-mon[36956]: osdmap e15: 2 total, 2 up, 2 in 2024-06-03T01:15:17.227 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:16 smithi032 ceph-mon[29360]: osdmap e15: 2 total, 2 up, 2 in 2024-06-03T01:15:17.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2024-06-03T01:15:17.493 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:15:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:17 smithi044 ceph-mon[36956]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:17 smithi032 ceph-mon[29360]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:18.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 -- 172.21.15.32:0/1342961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 msgr2=0x7f1038101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 --2- 172.21.15.32:0/1342961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038101270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1020009910 tx=0x7f1020009c20 comp rx=0 tx=0).stop 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 -- 172.21.15.32:0/1342961173 shutdown_connections 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 --2- 172.21.15.32:0/1342961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038101270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 --2- 172.21.15.32:0/1342961173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 0x7f1038100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 -- 172.21.15.32:0/1342961173 >> 172.21.15.32:0/1342961173 conn(0x7f10380fb190 msgr2=0x7f10380fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 -- 172.21.15.32:0/1342961173 shutdown_connections 2024-06-03T01:15:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.969+0000 7f103e58d700 1 -- 172.21.15.32:0/1342961173 wait complete. 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 Processor -- start 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 -- start start 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 0x7f1038198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1038199290 con 0x7f1038100e10 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f103e58d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10381993d0 con 0x7f10380ffc10 2024-06-03T01:15:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47586/0 (socket says 172.21.15.32:47586) 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 -- 172.21.15.32:0/2714487067 learned_addr learned my addr 172.21.15.32:0/2714487067 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f1037fff700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 0x7f1038198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 -- 172.21.15.32:0/2714487067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 msgr2=0x7f1038198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 0x7f1038198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.970+0000 7f102ffff700 1 -- 172.21.15.32:0/2714487067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200095f0 con 0x7f1038100e10 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f102ffff700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f102000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102001d070 con 0x7f1038100e10 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103819de20 con 0x7f1038100e10 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f102000f460 con 0x7f1038100e10 2024-06-03T01:15:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1020021640 con 0x7f1038100e10 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.971+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103819e2e0 con 0x7f1038100e10 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.972+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10381087a0 con 0x7f1038100e10 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.973+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f102000fab0 con 0x7f1038100e10 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.973+0000 7f1035ffb700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 0x7f102406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.973+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f102008bf70 con 0x7f1038100e10 2024-06-03T01:15:18.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.973+0000 7f1037fff700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 0x7f102406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:18.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.974+0000 7f1037fff700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 0x7f102406e920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1028006fd0 tx=0x7f1028008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:18.978+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10200bf0e0 con 0x7f1038100e10 2024-06-03T01:15:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:19.209+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1038068c90 con 0x7f102406c460 2024-06-03T01:15:20.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:19 smithi032 ceph-mon[29360]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:20.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:15:20.123 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:15:20.123 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:20.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:19 smithi044 ceph-mon[36956]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:20.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:15:20.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:15:20.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:21.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:20 smithi032 ceph-mon[29360]: from='client.14294 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:15:21.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:20 smithi044 ceph-mon[36956]: from='client.14294 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:15:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:21 smithi032 ceph-mon[29360]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:21 smithi044 ceph-mon[36956]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi032 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi032 to 5922M 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:23 smithi044 ceph-mon[36956]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi032 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi032 to 5922M 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:23.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:23.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:23.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:23.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:23 smithi032 ceph-mon[29360]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:24.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:24 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3621824378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c0dbb0a-a82a-4a4e-a268-24795b04ee7f"}]: dispatch 2024-06-03T01:15:24.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:24 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3621824378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c0dbb0a-a82a-4a4e-a268-24795b04ee7f"}]': finished 2024-06-03T01:15:24.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:24 smithi032 ceph-mon[29360]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T01:15:24.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:24 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:24 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3621824378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c0dbb0a-a82a-4a4e-a268-24795b04ee7f"}]: dispatch 2024-06-03T01:15:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:24 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3621824378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c0dbb0a-a82a-4a4e-a268-24795b04ee7f"}]': finished 2024-06-03T01:15:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:24 smithi044 ceph-mon[36956]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T01:15:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:24 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:25.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:25.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:25.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:25 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3456039364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:15:25.062 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:25 smithi032 ceph-mon[29360]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:25.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:25.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:25 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3456039364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:15:25.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:25 smithi044 ceph-mon[36956]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:27 smithi032 ceph-mon[29360]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:27 smithi044 ceph-mon[36956]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:29.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:29 smithi044 ceph-mon[36956]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:29 smithi032 ceph-mon[29360]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:31.019 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:30 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:15:31.019 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:30 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:30 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:15:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:30 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:31 smithi044 ceph-mon[36956]: Deploying daemon osd.2 on smithi032 2024-06-03T01:15:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:31 smithi044 ceph-mon[36956]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:32.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:31 smithi032 ceph-mon[29360]: Deploying daemon osd.2 on smithi032 2024-06-03T01:15:32.257 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:31 smithi032 ceph-mon[29360]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:33 smithi044 ceph-mon[36956]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:34.231 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:33 smithi032 ceph-mon[29360]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:35.866 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:35 smithi032 ceph-mon[29360]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:35.952 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:35 smithi044 ceph-mon[36956]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:37.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:37.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:37.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:37.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:37 smithi032 ceph-mon[29360]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:37.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:37.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:37.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:37.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:37 smithi044 ceph-mon[36956]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:39.996 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:39 smithi032 ceph-mon[29360]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:39.996 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:39 smithi032 ceph-mon[29360]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T01:15:40.072 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 2 on host 'smithi032' 2024-06-03T01:15:40.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.071+0000 7f1035ffb700 1 -- 172.21.15.32:0/2714487067 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1038068c90 con 0x7f102406c460 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 msgr2=0x7f102406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 0x7f102406e920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1028006fd0 tx=0x7f1028008040 comp rx=0 tx=0).stop 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 msgr2=0x7f1038198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f102000be20 comp rx=0 tx=0).stop 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 shutdown_connections 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f102406c460 0x7f102406e920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10380ffc10 0x7f1038198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 --2- 172.21.15.32:0/2714487067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1038100e10 0x7f1038198c70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 >> 172.21.15.32:0/2714487067 conn(0x7f10380fb190 msgr2=0x7f1038068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 shutdown_connections 2024-06-03T01:15:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:15:40.074+0000 7f103e58d700 1 -- 172.21.15.32:0/2714487067 wait complete. 2024-06-03T01:15:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:39 smithi044 ceph-mon[36956]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:40.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:39 smithi044 ceph-mon[36956]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T01:15:40.942 DEBUG:teuthology.orchestra.run.smithi032:osd.2> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.2.service 2024-06-03T01:15:40.945 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_4... 2024-06-03T01:15:40.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T01:15:41.168 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:15:41.262 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:41 smithi044 ceph-mon[36956]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:15:41.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-mon[29360]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:42.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[63345]: 2024-06-03T01:15:41.660+0000 7fa85d2ab700 -1 osd.2 0 waiting for initial osdmap 2024-06-03T01:15:42.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:41 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[63345]: 2024-06-03T01:15:41.665+0000 7fa85609a700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:15:42.901 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T01:15:42.901 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T01:15:42.901 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-06-03T01:15:42.902 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-06-03T01:15:42.902 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110456 s, 949 MB/s 2024-06-03T01:15:42.902 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-06-03T01:15:42.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:42 smithi044 ceph-mon[36956]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:15:42.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:42 smithi044 ceph-mon[36956]: osdmap e18: 3 total, 2 up, 3 in 2024-06-03T01:15:42.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:42 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:42.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:42 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:42 smithi032 ceph-mon[29360]: from='osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-06-03T01:15:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:42 smithi032 ceph-mon[29360]: osdmap e18: 3 total, 2 up, 3 in 2024-06-03T01:15:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:42 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:42 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:43.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-06-03T01:15:43.797 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] boot 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:43.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:43.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:43.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:43.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:15:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:15:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:15:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: osd.2 [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] boot 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T01:15:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:15:44.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T01:15:44.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:44 smithi044 ceph-mon[36956]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T01:15:44.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:15:44.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- 172.21.15.44:0/658906940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4101750 msgr2=0x7f4eb4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 --2- 172.21.15.44:0/658906940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4101750 0x7f4eb4103b40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009910 tx=0x7f4ea4009c20 comp rx=0 tx=0).stop 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- 172.21.15.44:0/658906940 shutdown_connections 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 --2- 172.21.15.44:0/658906940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4104080 0x7f4eb4106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 --2- 172.21.15.44:0/658906940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4101750 0x7f4eb4103b40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- 172.21.15.44:0/658906940 >> 172.21.15.44:0/658906940 conn(0x7f4eb40fb160 msgr2=0x7f4eb40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- 172.21.15.44:0/658906940 shutdown_connections 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- 172.21.15.44:0/658906940 wait complete. 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 Processor -- start 2024-06-03T01:15:45.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.197+0000 7f4eba571700 1 -- start start 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eba571700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4101750 0x7f4eb41967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eba571700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eba571700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4197310 con 0x7f4eb4101750 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eba571700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4197450 con 0x7f4eb4104080 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43816/0 (socket says 172.21.15.44:43816) 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 -- 172.21.15.44:0/395207401 learned_addr learned my addr 172.21.15.44:0/395207401 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb3fff700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4101750 0x7f4eb41967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 -- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4101750 msgr2=0x7f4eb41967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4101750 0x7f4eb41967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.198+0000 7f4eb37fe700 1 -- 172.21.15.44:0/395207401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40095f0 con 0x7f4eb4104080 2024-06-03T01:15:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eb37fe700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800d900 tx=0x7f4ea800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea80041f0 con 0x7f4eb4104080 2024-06-03T01:15:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ea8004ce0 con 0x7f4eb4104080 2024-06-03T01:15:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb419bf00 con 0x7f4eb4104080 2024-06-03T01:15:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea800b750 con 0x7f4eb4104080 2024-06-03T01:15:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.199+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb419c420 con 0x7f4eb4104080 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.200+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb4190630 con 0x7f4eb4104080 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.200+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f4ea8004e50 con 0x7f4eb4104080 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.201+0000 7f4eb17fa700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 0x7f4e9c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.201+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f4ea808ad00 con 0x7f4eb4104080 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.201+0000 7f4eb3fff700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 0x7f4e9c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.201+0000 7f4eb3fff700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 0x7f4e9c06e920 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400b5c0 tx=0x7f4ea4005ee0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.206+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ea8059910 con 0x7f4eb4104080 2024-06-03T01:15:45.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T01:15:45.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:44 smithi032 ceph-mon[29360]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T01:15:45.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:15:45.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:45.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:45.412+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eb402d050 con 0x7f4e9c06c460 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:45.966 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:45 smithi032 ceph-mon[29360]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:15:45.967 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:45 smithi032 sudo[67628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T01:15:45.967 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:45 smithi032 sudo[67628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:45.967 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:45 smithi032 sudo[67628]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:45.967 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:45 smithi032 sudo[67628]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:46.143 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:45 smithi044 ceph-mon[36956]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:15:46.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T01:15:46.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:46.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67663]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:46.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67663]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:46.828 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T01:15:46.828 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:46.828 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67770]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:46.828 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:46 smithi032 sudo[67770]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:47.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T01:15:47.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:47.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67836]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:47.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67836]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:15:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: osdmap e22: 3 total, 3 up, 3 in 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:46 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: osdmap e22: 3 total, 3 up, 3 in 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:46 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:47.722 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67905]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T01:15:47.722 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:47.722 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67905]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:47.722 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67905]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:47.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:15:47.887+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4ea8056250 con 0x7f4eb4104080 2024-06-03T01:15:48.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T01:15:48.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:48.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:48.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:15:47 smithi032 sudo[67963]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:47 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi032 2024-06-03T01:15:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:47 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi032 to 3948M 2024-06-03T01:15:48.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:47 smithi032 ceph-mon[29360]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:48.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:47 smithi032 ceph-mon[29360]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:15:48.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:47 smithi032 ceph-mon[29360]: Cluster is now healthy 2024-06-03T01:15:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:47 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi032 2024-06-03T01:15:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:47 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi032 to 3948M 2024-06-03T01:15:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:47 smithi044 ceph-mon[36956]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:47 smithi044 ceph-mon[36956]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:15:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:47 smithi044 ceph-mon[36956]: Cluster is now healthy 2024-06-03T01:15:48.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68010]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T01:15:48.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68010]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:48.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68010]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:48.643 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68010]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:48.986 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68050]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T01:15:48.987 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:48.987 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68050]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:48.987 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:48 smithi032 sudo[68050]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:49.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[40962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T01:15:49.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[40962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:49.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[40962]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:49.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[40962]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:50.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: mgrmap e19: smithi032.kfcfaa(active, since 2m), standbys: smithi044.wszjlw 2024-06-03T01:15:50.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2980028365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]: dispatch 2024-06-03T01:15:50.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]': finished 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/1289131637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:15:50.017 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:49 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:15:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[41337]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[41337]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[41337]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 sudo[41337]: pam_unix(sudo:session): session closed for user root 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: mgrmap e19: smithi032.kfcfaa(active, since 2m), standbys: smithi044.wszjlw 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/2980028365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]: dispatch 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]: dispatch 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60fd88fd-b2e7-41d0-8479-bfa395ef9d89"}]': finished 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/1289131637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:15:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T01:15:50.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:15:50.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:15:50.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:15:50.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:49 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:15:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T01:15:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T01:15:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:50 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T01:15:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:50 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:15:52.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:51 smithi032 ceph-mon[29360]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:52.159 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:51 smithi044 ceph-mon[36956]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:15:54.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:53 smithi044 ceph-mon[36956]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-06-03T01:15:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:53 smithi032 ceph-mon[29360]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-06-03T01:15:55.771 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:55 smithi044 ceph-mon[36956]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-06-03T01:15:55.771 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:15:55.771 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:55 smithi032 ceph-mon[29360]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-06-03T01:15:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:15:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:56 smithi044 ceph-mon[36956]: Deploying daemon osd.3 on smithi044 2024-06-03T01:15:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:56 smithi032 ceph-mon[29360]: Deploying daemon osd.3 on smithi044 2024-06-03T01:15:57.840 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:57 smithi044 ceph-mon[36956]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-06-03T01:15:57.995 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:57 smithi032 ceph-mon[29360]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-06-03T01:15:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:58 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:15:59.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:58 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:15:59 smithi044 ceph-mon[36956]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 119 KiB/s wr, 8 op/s 2024-06-03T01:16:00.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:15:59 smithi032 ceph-mon[29360]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 119 KiB/s wr, 8 op/s 2024-06-03T01:16:01.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:01 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:01.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:01 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:01.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:01 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:01.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:01 smithi032 ceph-mon[29360]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-06-03T01:16:01.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:01 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:01.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:01 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:01.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:01 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:01.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:01 smithi044 ceph-mon[36956]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-06-03T01:16:03.625 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2024-06-03T01:16:03.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.624+0000 7f4eb17fa700 1 -- 172.21.15.44:0/395207401 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4eb402d050 con 0x7f4e9c06c460 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.626+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 msgr2=0x7f4e9c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.626+0000 7f4eba571700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 0x7f4e9c06e920 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400b5c0 tx=0x7f4ea4005ee0 comp rx=0 tx=0).stop 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 msgr2=0x7f4eb4196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800d900 tx=0x7f4ea800dc10 comp rx=0 tx=0).stop 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 shutdown_connections 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb4101750 0x7f4eb41967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e9c06c460 0x7f4e9c06e920 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 --2- 172.21.15.44:0/395207401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb4104080 0x7f4eb4196cf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 >> 172.21.15.44:0/395207401 conn(0x7f4eb40fb160 msgr2=0x7f4eb40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 shutdown_connections 2024-06-03T01:16:03.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:03.627+0000 7f4eba571700 1 -- 172.21.15.44:0/395207401 wait complete. 2024-06-03T01:16:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:03 smithi044 ceph-mon[36956]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-06-03T01:16:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:03 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:04.328 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.3.service 2024-06-03T01:16:04.331 INFO:tasks.cephadm:Deploying osd.4 on smithi044 with /dev/vg_nvme/lv_3... 2024-06-03T01:16:04.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T01:16:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:03 smithi032 ceph-mon[29360]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-06-03T01:16:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:03 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:04.585 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:16:04.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:04 smithi044 ceph-mon[36956]: from='osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:16:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:04 smithi044 ceph-mon[36956]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:16:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:04 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:04 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:04 smithi032 ceph-mon[29360]: from='osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:16:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:04 smithi032 ceph-mon[29360]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:16:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:04 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:04 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: from='osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:06.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-mon[36956]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:06.093 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[42964]: 2024-06-03T01:16:05.937+0000 7fdd73300700 -1 osd.3 0 waiting for initial osdmap 2024-06-03T01:16:06.094 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:16:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[42964]: 2024-06-03T01:16:05.944+0000 7fdd6c0ef700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: from='osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:05 smithi032 ceph-mon[29360]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T01:16:07.218 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T01:16:07.218 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-06-03T01:16:07.218 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-06-03T01:16:07.218 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142947 s, 734 MB/s 2024-06-03T01:16:07.218 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-06-03T01:16:07.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:06 smithi044 ceph-mon[36956]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:07.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:06 smithi044 ceph-mon[36956]: osdmap e25: 4 total, 3 up, 4 in 2024-06-03T01:16:07.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:07.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:06 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:06 smithi032 ceph-mon[29360]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:06 smithi032 ceph-mon[29360]: osdmap e25: 4 total, 3 up, 4 in 2024-06-03T01:16:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:06 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:08.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-06-03T01:16:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:16:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:16:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:16:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] boot 2024-06-03T01:16:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T01:16:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:07 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: osd.3 [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] boot 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T01:16:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:07 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:16:08.768 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:09.177 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:09.178 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:09.178 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:08 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T01:16:09.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:09.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:09.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:08 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:10.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:09 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi044 2024-06-03T01:16:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:09 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi044 to 14916M 2024-06-03T01:16:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:09 smithi044 ceph-mon[36956]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:09 smithi044 ceph-mon[36956]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T01:16:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:09 smithi044 ceph-mon[36956]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T01:16:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:09 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi044 2024-06-03T01:16:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:09 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi044 to 14916M 2024-06-03T01:16:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:09 smithi032 ceph-mon[29360]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:09 smithi032 ceph-mon[29360]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T01:16:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:09 smithi032 ceph-mon[29360]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T01:16:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 -- 172.21.15.44:0/1133775820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 msgr2=0x7f95f4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 --2- 172.21.15.44:0/1133775820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4102310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009910 tx=0x7f95dc009c20 comp rx=0 tx=0).stop 2024-06-03T01:16:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 -- 172.21.15.44:0/1133775820 shutdown_connections 2024-06-03T01:16:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 --2- 172.21.15.44:0/1133775820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4102310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 --2- 172.21.15.44:0/1133775820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 -- 172.21.15.44:0/1133775820 >> 172.21.15.44:0/1133775820 conn(0x7f95f40fc230 msgr2=0x7f95f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 -- 172.21.15.44:0/1133775820 shutdown_connections 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.348+0000 7f95f88c4700 1 -- 172.21.15.44:0/1133775820 wait complete. 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 Processor -- start 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 -- start start 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4101eb0 0x7f95f4073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f40736c0 con 0x7f95f4101eb0 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f88c4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4073800 con 0x7f95f4100cb0 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53314/0 (socket says 172.21.15.44:53314) 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 -- 172.21.15.44:0/26698968 learned_addr learned my addr 172.21.15.44:0/26698968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 -- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4101eb0 msgr2=0x7f95f4073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f1d9b700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4101eb0 0x7f95f4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4101eb0 0x7f95f4073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 -- 172.21.15.44:0/26698968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0095f0 con 0x7f95f4100cb0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.349+0000 7f95f259c700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95e400d900 tx=0x7f95e400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.350+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e40041f0 con 0x7f95f4100cb0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.350+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f4073ae0 con 0x7f95f4100cb0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.350+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95e4004350 con 0x7f95f4100cb0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.350+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e4003da0 con 0x7f95f4100cb0 2024-06-03T01:16:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.350+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f419e540 con 0x7f95f4100cb0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.351+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f4109810 con 0x7f95f4100cb0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.351+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f95e4009730 con 0x7f95f4100cb0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.352+0000 7f95eb7fe700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 0x7f95e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.352+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f95e408b2f0 con 0x7f95f4100cb0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.352+0000 7f95f1d9b700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 0x7f95e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.353+0000 7f95f1d9b700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 0x7f95e006e9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b5c0 tx=0x7f95dc005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.356+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f95e4059c00 con 0x7f95f4100cb0 2024-06-03T01:16:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:10.564+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f95f419eaf0 con 0x7f95e006c4e0 2024-06-03T01:16:11.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:10 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:16:11.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:10 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:16:11.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:10 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:11.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:10 smithi044 ceph-mon[36956]: osdmap e30: 4 total, 4 up, 4 in 2024-06-03T01:16:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:10 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:16:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:10 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:16:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:10 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:10 smithi032 ceph-mon[29360]: osdmap e30: 4 total, 4 up, 4 in 2024-06-03T01:16:12.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:11 smithi044 ceph-mon[36956]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:16:12.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:11 smithi044 ceph-mon[36956]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:11 smithi032 ceph-mon[29360]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:16:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:11 smithi032 ceph-mon[29360]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:13.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:12 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/1664062443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]: dispatch 2024-06-03T01:16:13.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:12 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]: dispatch 2024-06-03T01:16:13.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:12 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]': finished 2024-06-03T01:16:13.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:12 smithi044 ceph-mon[36956]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T01:16:13.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:12 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/1664062443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]: dispatch 2024-06-03T01:16:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:12 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]: dispatch 2024-06-03T01:16:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:12 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a48b78c-eda0-4ff9-9646-d5df7e89b603"}]': finished 2024-06-03T01:16:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:12 smithi032 ceph-mon[29360]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T01:16:13.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:13 smithi044 ceph-mon[36956]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:16:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:13 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/3389062528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:16:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:13 smithi032 ceph-mon[29360]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:16:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:13 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/3389062528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:16:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:15 smithi044 ceph-mon[36956]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-03T01:16:16.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:15 smithi032 ceph-mon[29360]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-03T01:16:18.163 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:17 smithi044 ceph-mon[36956]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-06-03T01:16:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:17 smithi032 ceph-mon[29360]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-06-03T01:16:20.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:19 smithi044 ceph-mon[36956]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T01:16:20.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:19 smithi032 ceph-mon[29360]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T01:16:21.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:20 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:16:21.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:20 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:21.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:20 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:16:21.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:20 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:22.080 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:21 smithi044 ceph-mon[36956]: Deploying daemon osd.4 on smithi044 2024-06-03T01:16:22.080 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:21 smithi044 ceph-mon[36956]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T01:16:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:21 smithi032 ceph-mon[29360]: Deploying daemon osd.4 on smithi044 2024-06-03T01:16:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:21 smithi032 ceph-mon[29360]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T01:16:23.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:23 smithi044 ceph-mon[36956]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:23 smithi032 ceph-mon[29360]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:25.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:25 smithi044 ceph-mon[36956]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:25.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:25.754 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:25.754 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:25 smithi032 ceph-mon[29360]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:27 smithi044 ceph-mon[36956]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:28.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:27 smithi032 ceph-mon[29360]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.328+0000 7f95eb7fe700 1 -- 172.21.15.44:0/26698968 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f95f419eaf0 con 0x7f95e006c4e0 2024-06-03T01:16:28.328 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 4 on host 'smithi044' 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 msgr2=0x7f95e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 0x7f95e006e9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b5c0 tx=0x7f95dc005960 comp rx=0 tx=0).stop 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 msgr2=0x7f95f4074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95e400d900 tx=0x7f95e400dcc0 comp rx=0 tx=0).stop 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 shutdown_connections 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f95e006c4e0 0x7f95e006e9a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.330+0000 7f95f88c4700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4100cb0 0x7f95f4074b30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.331+0000 7f95f88c4700 1 --2- 172.21.15.44:0/26698968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4101eb0 0x7f95f4073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.331+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 >> 172.21.15.44:0/26698968 conn(0x7f95f40fc230 msgr2=0x7f95f41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.331+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 shutdown_connections 2024-06-03T01:16:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:28.331+0000 7f95f88c4700 1 -- 172.21.15.44:0/26698968 wait complete. 2024-06-03T01:16:28.954 DEBUG:teuthology.orchestra.run.smithi044:osd.4> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.4.service 2024-06-03T01:16:28.957 INFO:tasks.cephadm:Deploying osd.5 on smithi044 with /dev/vg_nvme/lv_2... 2024-06-03T01:16:28.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T01:16:29.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:16:29.155+0000 7fecf65a2640 -1 osd.4 0 log_to_monitors true 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: from='osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:16:29.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:29 smithi032 ceph-mon[29360]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:16:29.666 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:16:29.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:29.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:29.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:29.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:29.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: from='osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:16:29.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:29 smithi044 ceph-mon[36956]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:16:30.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-mon[36956]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:16:30.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-mon[36956]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T01:16:30.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:30.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-mon[36956]: from='osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:30.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-mon[36956]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:30.472 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:16:30.302+0000 7feceb405700 -1 osd.4 0 waiting for initial osdmap 2024-06-03T01:16:30.472 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:16:30 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:16:30.309+0000 7fece71fa700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:16:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:30 smithi032 ceph-mon[29360]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:16:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:30 smithi032 ceph-mon[29360]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T01:16:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:30 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:30 smithi032 ceph-mon[29360]: from='osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:30 smithi032 ceph-mon[29360]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:31.329 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:31.329 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: osdmap e33: 5 total, 4 up, 5 in 2024-06-03T01:16:31.330 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:31.330 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:31.330 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:31.330 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:31.330 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:31 smithi044 ceph-mon[36956]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: osdmap e33: 5 total, 4 up, 5 in 2024-06-03T01:16:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:31.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:31.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:31.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:31.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:31 smithi032 ceph-mon[29360]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T01:16:31.660 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T01:16:31.660 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T01:16:31.660 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-06-03T01:16:31.660 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-06-03T01:16:31.661 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138042 s, 760 MB/s 2024-06-03T01:16:31.661 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-06-03T01:16:32.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-06-03T01:16:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:16:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:16:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] boot 2024-06-03T01:16:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T01:16:32.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:32 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:32.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:16:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:16:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: osd.4 [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] boot 2024-06-03T01:16:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T01:16:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:32 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:16:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:16:33.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:33 smithi032 ceph-mon[29360]: osdmap e35: 5 total, 5 up, 5 in 2024-06-03T01:16:33.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:33 smithi032 ceph-mon[29360]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:33.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:33 smithi044 ceph-mon[36956]: osdmap e35: 5 total, 5 up, 5 in 2024-06-03T01:16:33.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:33 smithi044 ceph-mon[36956]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:34.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 -- 172.21.15.44:0/4183824559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 msgr2=0x7f91bc102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/4183824559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc102310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009910 tx=0x7f91a4009c20 comp rx=0 tx=0).stop 2024-06-03T01:16:34.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 -- 172.21.15.44:0/4183824559 shutdown_connections 2024-06-03T01:16:34.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/4183824559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc102310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/4183824559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 0x7f91bc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 -- 172.21.15.44:0/4183824559 >> 172.21.15.44:0/4183824559 conn(0x7f91bc0fc230 msgr2=0x7f91bc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 -- 172.21.15.44:0/4183824559 shutdown_connections 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.880+0000 7f91c1ec7700 1 -- 172.21.15.44:0/4183824559 wait complete. 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 Processor -- start 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 -- start start 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 0x7f91bc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc196fb0 con 0x7f91bc100cb0 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91c1ec7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc1970f0 con 0x7f91bc101eb0 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54134/0 (socket says 172.21.15.44:54134) 2024-06-03T01:16:34.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 -- 172.21.15.44:0/2114301249 learned_addr learned my addr 172.21.15.44:0/2114301249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:16:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91bb7fe700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 0x7f91bc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 -- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 msgr2=0x7f91bc1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 0x7f91bc1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 -- 172.21.15.44:0/2114301249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a40095f0 con 0x7f91bc101eb0 2024-06-03T01:16:34.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.881+0000 7f91b3fff700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91a400bd70 tx=0x7f91a400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.882+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a401d070 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.882+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f91a4005940 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.882+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc19bb50 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.882+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a4021430 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.882+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc19c010 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.883+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc109810 con 0x7f91bc101eb0 2024-06-03T01:16:34.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.883+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f91a400f860 con 0x7f91bc101eb0 2024-06-03T01:16:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.884+0000 7f91b97fa700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 0x7f91a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.884+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f91a408c710 con 0x7f91bc101eb0 2024-06-03T01:16:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.884+0000 7f91bb7fe700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 0x7f91a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.885+0000 7f91bb7fe700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 0x7f91a806e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91bc101d10 tx=0x7f91ac00a3b0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:34.888+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f91a405ae70 con 0x7f91bc101eb0 2024-06-03T01:16:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:35.096+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f91bc106800 con 0x7f91a806c4e0 2024-06-03T01:16:35.716 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:35.716 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:16:35.716 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:35.717 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:35 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:36.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:36.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:35 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:36.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:36 smithi044 ceph-mon[36956]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:16:36.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:36 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi044 2024-06-03T01:16:36.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:36 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi044 to 7458M 2024-06-03T01:16:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:36 smithi032 ceph-mon[29360]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:16:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:36 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi044 2024-06-03T01:16:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:36 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi044 to 7458M 2024-06-03T01:16:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:37 smithi044 ceph-mon[36956]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:38.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:37 smithi032 ceph-mon[29360]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2859692582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]: dispatch 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]: dispatch 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]': finished 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:39.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:38 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2768616256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:16:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/2859692582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]: dispatch 2024-06-03T01:16:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]: dispatch 2024-06-03T01:16:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b97503ff-62f3-4ae9-aba2-c0ad20ae00a4"}]': finished 2024-06-03T01:16:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T01:16:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:38 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/2768616256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:16:39.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:39 smithi044 ceph-mon[36956]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:39 smithi032 ceph-mon[29360]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:42.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:41 smithi044 ceph-mon[36956]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:42.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:41 smithi032 ceph-mon[29360]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:44.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:43 smithi044 ceph-mon[36956]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:44.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:43 smithi032 ceph-mon[29360]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:45.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:16:45.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:45.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:45 smithi044 ceph-mon[36956]: Deploying daemon osd.5 on smithi044 2024-06-03T01:16:45.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:45 smithi044 ceph-mon[36956]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:16:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:45 smithi032 ceph-mon[29360]: Deploying daemon osd.5 on smithi044 2024-06-03T01:16:46.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:45 smithi032 ceph-mon[29360]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:48.170 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:47 smithi044 ceph-mon[36956]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:48.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:47 smithi032 ceph-mon[29360]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:50.205 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:49 smithi044 ceph-mon[36956]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:49 smithi032 ceph-mon[29360]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:51.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:51 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:51.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:51 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:51.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:51 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:51.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:51 smithi044 ceph-mon[36956]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:51 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:51 smithi032 ceph-mon[29360]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.163+0000 7f91b97fa700 1 -- 172.21.15.44:0/2114301249 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f91bc106800 con 0x7f91a806c4e0 2024-06-03T01:16:53.163 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 5 on host 'smithi044' 2024-06-03T01:16:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 msgr2=0x7f91a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 0x7f91a806e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91bc101d10 tx=0x7f91ac00a3b0 comp rx=0 tx=0).stop 2024-06-03T01:16:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 msgr2=0x7f91bc196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91a400bd70 tx=0x7f91a400bda0 comp rx=0 tx=0).stop 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 shutdown_connections 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc100cb0 0x7f91bc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f91a806c4e0 0x7f91a806e9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 --2- 172.21.15.44:0/2114301249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc101eb0 0x7f91bc196a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 >> 172.21.15.44:0/2114301249 conn(0x7f91bc0fc230 msgr2=0x7f91bc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.165+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 shutdown_connections 2024-06-03T01:16:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:16:53.166+0000 7f91c1ec7700 1 -- 172.21.15.44:0/2114301249 wait complete. 2024-06-03T01:16:53.811 DEBUG:teuthology.orchestra.run.smithi044:osd.5> sudo journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.5.service 2024-06-03T01:16:53.814 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-06-03T01:16:53.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-03T01:16:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:16:54.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-mon[36956]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:54.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:54.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:54.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-mon[36956]: from='osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:16:54.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-mon[36956]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:16:54.063 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:16:53 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:16:53.913+0000 7fc43a7a2640 -1 osd.5 0 log_to_monitors true 2024-06-03T01:16:54.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:53 smithi032 ceph-mon[29360]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:54.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:54.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:54.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:53 smithi032 ceph-mon[29360]: from='osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:16:54.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:53 smithi032 ceph-mon[29360]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:16:55.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:16:55.129+0000 7fc42f605700 -1 osd.5 0 waiting for initial osdmap 2024-06-03T01:16:55.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:16:55.136+0000 7fc4293f6700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:55 smithi044 ceph-mon[36956]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:16:55.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:55.382 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:55.382 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:55 smithi032 ceph-mon[29360]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/2858348363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 msgr2=0x7fc7b00fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/2858348363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b00fffe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc798009910 tx=0x7fc798009c20 comp rx=0 tx=0).stop 2024-06-03T01:16:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/2858348363 shutdown_connections 2024-06-03T01:16:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/2858348363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 0x7fc7b0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/2858348363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b00fffe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/2858348363 >> 172.21.15.32:0/2858348363 conn(0x7fc7b00fb160 msgr2=0x7fc7b00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/2858348363 shutdown_connections 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.723+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/2858348363 wait complete. 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 Processor -- start 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 -- start start 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 0x7fc7b019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019baf0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b6e1a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019bc60 con 0x7fc7b0100dc0 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b4bb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b4bb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38004/0 (socket says 172.21.15.32:38004) 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7b4bb6700 1 -- 172.21.15.32:0/508156614 learned_addr learned my addr 172.21.15.32:0/508156614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.724+0000 7fc7affff700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 0x7fc7b019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b4bb6700 1 -- 172.21.15.32:0/508156614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 msgr2=0x7fc7b019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b4bb6700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 0x7fc7b019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b4bb6700 1 -- 172.21.15.32:0/508156614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980095f0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b4bb6700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc798005b40 tx=0x7fc79800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79801d070 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc798005dc0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b019bee0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc798022470 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.725+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b019c380 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.726+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b018f530 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.727+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc798003680 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.727+0000 7fc7adffb700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 0x7fc79c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.727+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4358+0+0 (secure 0 0 0) 0x7fc79808c940 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.727+0000 7fc7affff700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 0x7fc79c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.728+0000 7fc7affff700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 0x7fc79c06e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0069ef0 tx=0x7fc7a0009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.732+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc79805afe0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.930+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc7b0068af0 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.930+0000 7fc7adffb700 1 -- 172.21.15.32:0/508156614 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc79805ab70 con 0x7fc7b00ffbc0 2024-06-03T01:16:55.931 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:16:55.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 msgr2=0x7fc79c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 0x7fc79c06e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0069ef0 tx=0x7fc7a0009260 comp rx=0 tx=0).stop 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 msgr2=0x7fc7b0069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc798005b40 tx=0x7fc79800fa00 comp rx=0 tx=0).stop 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 shutdown_connections 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b00ffbc0 0x7fc7b0069990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc79c06c530 0x7fc79c06e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 --2- 172.21.15.32:0/508156614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7b0100dc0 0x7fc7b019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:55.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 >> 172.21.15.32:0/508156614 conn(0x7fc7b00fb160 msgr2=0x7fc7b01040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:55.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 shutdown_connections 2024-06-03T01:16:55.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:55.933+0000 7fc7b6e1a700 1 -- 172.21.15.32:0/508156614 wait complete. 2024-06-03T01:16:56.247 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:56 smithi032 ceph-mon[29360]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:56.247 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:56 smithi032 ceph-mon[29360]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T01:16:56.248 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:56.248 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:56.248 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:56 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/508156614' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:16:56.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:56 smithi044 ceph-mon[36956]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-06-03T01:16:56.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:56 smithi044 ceph-mon[36956]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T01:16:56.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:56.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:56.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:56 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/508156614' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:16:56.655 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1717377391,"num_in_osds":6,"osd_in_since":1717377397,"num_remapped_pgs":0} 2024-06-03T01:16:57.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: purged_snaps scrub starts 2024-06-03T01:16:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: purged_snaps scrub ok 2024-06-03T01:16:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] boot 2024-06-03T01:16:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T01:16:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:57 smithi044 ceph-mon[36956]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: purged_snaps scrub starts 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: purged_snaps scrub ok 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: osd.5 [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] boot 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:16:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:57 smithi032 ceph-mon[29360]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:16:57.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-03T01:16:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:16:58.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:58 smithi032 ceph-mon[29360]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T01:16:58.405 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:58 smithi044 ceph-mon[36956]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T01:16:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 -- 172.21.15.32:0/127398732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0ffc10 msgr2=0x7fabcc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 --2- 172.21.15.32:0/127398732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0ffc10 0x7fabcc100030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009910 tx=0x7fabc8009c20 comp rx=0 tx=0).stop 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 -- 172.21.15.32:0/127398732 shutdown_connections 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 --2- 172.21.15.32:0/127398732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc100e10 0x7fabcc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 --2- 172.21.15.32:0/127398732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0ffc10 0x7fabcc100030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 -- 172.21.15.32:0/127398732 >> 172.21.15.32:0/127398732 conn(0x7fabcc0fb190 msgr2=0x7fabcc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 -- 172.21.15.32:0/127398732 shutdown_connections 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.406+0000 7fabd3932700 1 -- 172.21.15.32:0/127398732 wait complete. 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 Processor -- start 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 -- start start 2024-06-03T01:16:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 0x7fabcc103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc102730 con 0x7fabcc100e10 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd3932700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc102870 con 0x7fabcc0ffc10 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd0ecd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd0ecd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41232/0 (socket says 172.21.15.32:41232) 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.407+0000 7fabd0ecd700 1 -- 172.21.15.32:0/824731368 learned_addr learned my addr 172.21.15.32:0/824731368 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd16ce700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 0x7fabcc103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd0ecd700 1 -- 172.21.15.32:0/824731368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 msgr2=0x7fabcc103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd0ecd700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 0x7fabcc103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd0ecd700 1 -- 172.21.15.32:0/824731368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80095f0 con 0x7fabcc100e10 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd16ce700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 0x7fabcc103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:16:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd0ecd700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00d900 tx=0x7fabbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabbc0041f0 con 0x7fabcc100e10 2024-06-03T01:16:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc102b50 con 0x7fabcc100e10 2024-06-03T01:16:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fabbc004ce0 con 0x7fabcc100e10 2024-06-03T01:16:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabbc00b750 con 0x7fabcc100e10 2024-06-03T01:16:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.408+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc19e530 con 0x7fabcc100e10 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.409+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc108700 con 0x7fabcc100e10 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.410+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fabbc004350 con 0x7fabcc100e10 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.410+0000 7fabc27fc700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 0x7fabb806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.410+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fabbc08b610 con 0x7fabcc100e10 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.410+0000 7fabd16ce700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 0x7fabb806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.410+0000 7fabd16ce700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 0x7fabb806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006010 tx=0x7fabc8005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.414+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fabbc059bc0 con 0x7fabcc100e10 2024-06-03T01:16:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.612+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fabcc19ed30 con 0x7fabcc100e10 2024-06-03T01:16:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.613+0000 7fabc27fc700 1 -- 172.21.15.32:0/824731368 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fabbc059750 con 0x7fabcc100e10 2024-06-03T01:16:59.613 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 msgr2=0x7fabb806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 0x7fabb806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006010 tx=0x7fabc8005960 comp rx=0 tx=0).stop 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 msgr2=0x7fabcc1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00d900 tx=0x7fabbc00dcc0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 shutdown_connections 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fabb806c4e0 0x7fabb806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabcc0ffc10 0x7fabcc103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 --2- 172.21.15.32:0/824731368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc100e10 0x7fabcc1021f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:59.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 >> 172.21.15.32:0/824731368 conn(0x7fabcc0fb190 msgr2=0x7fabcc068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:59.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 shutdown_connections 2024-06-03T01:16:59.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:16:59.615+0000 7fabd3932700 1 -- 172.21.15.32:0/824731368 wait complete. 2024-06-03T01:16:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: Detected new or changed devices on smithi044 2024-06-03T01:16:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi044 to 4972M 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:16:59.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:16:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi044 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi044 to 4972M 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:16:59.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:16:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:00.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1717377416,"num_in_osds":6,"osd_in_since":1717377397,"num_remapped_pgs":0} 2024-06-03T01:17:00.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:17:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:00.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:00 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/824731368' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:17:00.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:00 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/824731368' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:17:01.738 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:01 smithi032 ceph-mon[29360]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 -- 172.21.15.32:0/4141962227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 msgr2=0x7f11981022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 --2- 172.21.15.32:0/4141962227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f11981022a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f118800b0d0 tx=0x7f118800b3e0 comp rx=0 tx=0).stop 2024-06-03T01:17:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 -- 172.21.15.32:0/4141962227 shutdown_connections 2024-06-03T01:17:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 --2- 172.21.15.32:0/4141962227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119810acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 --2- 172.21.15.32:0/4141962227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f11981022a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 -- 172.21.15.32:0/4141962227 >> 172.21.15.32:0/4141962227 conn(0x7f11980fb650 msgr2=0x7f11980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 -- 172.21.15.32:0/4141962227 shutdown_connections 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.960+0000 7f119ecc9700 1 -- 172.21.15.32:0/4141962227 wait complete. 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 Processor -- start 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 -- start start 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119819d1b0 con 0x7f1198101ec0 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ecc9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11981966f0 con 0x7f11981027e0 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ca65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ca65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41254/0 (socket says 172.21.15.32:41254) 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.961+0000 7f119ca65700 1 -- 172.21.15.32:0/2066538984 learned_addr learned my addr 172.21.15.32:0/2066538984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f1197fff700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ca65700 1 -- 172.21.15.32:0/2066538984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 msgr2=0x7f119819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ca65700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ca65700 1 -- 172.21.15.32:0/2066538984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009d00 con 0x7f1198101ec0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f1197fff700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ca65700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f118800b0a0 tx=0x7f118800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118800e040 con 0x7f1198101ec0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f11880044f0 con 0x7f1198101ec0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1198196970 con 0x7f1198101ec0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118801da90 con 0x7f1198101ec0 2024-06-03T01:17:01.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.962+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1198196e60 con 0x7f1198101ec0 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.963+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1198108370 con 0x7f1198101ec0 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.964+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1188004040 con 0x7f1198101ec0 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.964+0000 7f1195ffb700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 0x7f118406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.964+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f118808cf60 con 0x7f1198101ec0 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.964+0000 7f1197fff700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 0x7f118406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:01.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.965+0000 7f1197fff700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 0x7f118406e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1180005fd0 tx=0x7f1180005e20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:01.969+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f118805b470 con 0x7f1198101ec0 2024-06-03T01:17:01.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:01 smithi044 ceph-mon[36956]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.167+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f119804f2e0 con 0x7f1198101ec0 2024-06-03T01:17:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.168+0000 7f1195ffb700 1 -- 172.21.15.32:0/2066538984 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11287 (secure 0 0 0) 0x7f118805b000 con 0x7f1198101ec0 2024-06-03T01:17:02.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:02.169 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","created":"2024-06-03T01:11:14.450938+0000","modified":"2024-06-03T01:16:57.131992+0000","last_up_change":"2024-06-03T01:16:56.126935+0000","last_in_change":"2024-06-03T01:16:37.676175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T01:15:43.366528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ddd78453-9794-4382-b3f3-6dbe73ec2d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6803","nonce":564758788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6805","nonce":564758788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6809","nonce":564758788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6807","nonce":564758788}]},"public_addr":"172.21.15.32:6803/564758788","cluster_addr":"172.21.15.32:6805/564758788","heartbeat_back_addr":"172.21.15.32:6809/564758788","heartbeat_front_addr":"172.21.15.32:6807/564758788","state":["exists","up"]},{"osd":1,"uuid":"cb1be659-330d-40c2-9e80-9430b3bd8a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6811","nonce":813215710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6813","nonce":813215710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6817","nonce":813215710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6815","nonce":813215710}]},"public_addr":"172.21.15.32:6811/813215710","cluster_addr":"172.21.15.32:6813/813215710","heartbeat_back_addr":"172.21.15.32:6817/813215710","heartbeat_front_addr":"172.21.15.32:6815/813215710","state":["exists","up"]},{"osd":2,"uuid":"4c0dbb0a-a82a-4a4e-a268-24795b04ee7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6819","nonce":3632513352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6821","nonce":3632513352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6825","nonce":3632513352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6823","nonce":3632513352}]},"public_addr":"172.21.15.32:6819/3632513352","cluster_addr":"172.21.15.32:6821/3632513352","heartbeat_back_addr":"172.21.15.32:6825/3632513352","heartbeat_front_addr":"172.21.15.32:6823/3632513352","state":["exists","up"]},{"osd":3,"uuid":"60fd88fd-b2e7-41d0-8479-bfa395ef9d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6801","nonce":1384539104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6803","nonce":1384539104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6807","nonce":1384539104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6805","nonce":1384539104}]},"public_addr":"172.21.15.44:6801/1384539104","cluster_addr":"172.21.15.44:6803/1384539104","heartbeat_back_addr":"172.21.15.44:6807/1384539104","heartbeat_front_addr":"172.21.15.44:6805/1384539104","state":["exists","up"]},{"osd":4,"uuid":"5a48b78c-eda0-4ff9-9646-d5df7e89b603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6809","nonce":2875962035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6811","nonce":2875962035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6815","nonce":2875962035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6813","nonce":2875962035}]},"public_addr":"172.21.15.44:6809/2875962035","cluster_addr":"172.21.15.44:6811/2875962035","heartbeat_back_addr":"172.21.15.44:6815/2875962035","heartbeat_front_addr":"172.21.15.44:6813/2875962035","state":["exists","up"]},{"osd":5,"uuid":"b97503ff-62f3-4ae9-aba2-c0ad20ae00a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6817","nonce":3789906029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6819","nonce":3789906029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6823","nonce":3789906029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6821","nonce":3789906029}]},"public_addr":"172.21.15.44:6817/3789906029","cluster_addr":"172.21.15.44:6819/3789906029","heartbeat_back_addr":"172.21.15.44:6823/3789906029","heartbeat_front_addr":"172.21.15.44:6821/3789906029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:14:46.681798+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:13.540229+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:40.872057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:04.951207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:30.118175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:54.924738+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1695685766":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6800/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/2872286231":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1166022518":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/84030829":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6801/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/2914664298":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1891444588":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1129696663":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1434819821":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/2007986688":"2024-06-04T01:12:14.539594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 msgr2=0x7f118406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 0x7f118406e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1180005fd0 tx=0x7f1180005e20 comp rx=0 tx=0).stop 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 msgr2=0x7f119819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f118800b0a0 tx=0x7f118800bdb0 comp rx=0 tx=0).stop 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 shutdown_connections 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1198101ec0 0x7f119819c670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f118406c4e0 0x7f118406e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 --2- 172.21.15.32:0/2066538984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11981027e0 0x7f119819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 >> 172.21.15.32:0/2066538984 conn(0x7f11980fb650 msgr2=0x7f11981054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 shutdown_connections 2024-06-03T01:17:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:02.170+0000 7f119ecc9700 1 -- 172.21.15.32:0/2066538984 wait complete. 2024-06-03T01:17:02.752 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:02 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2066538984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:02.774 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T01:15:43.366528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-03T01:17:02.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-03T01:17:02.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:02 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2066538984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:02.989 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:03.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:03 smithi032 ceph-mon[29360]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:03.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:03 smithi044 ceph-mon[36956]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.561+0000 7f3670590700 1 -- 172.21.15.32:0/3736448028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 msgr2=0x7f3668100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.561+0000 7f3670590700 1 --2- 172.21.15.32:0/3736448028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f3668100180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3664009910 tx=0x7f3664009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 -- 172.21.15.32:0/3736448028 shutdown_connections 2024-06-03T01:17:04.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 --2- 172.21.15.32:0/3736448028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 0x7f36681047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 --2- 172.21.15.32:0/3736448028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f3668100180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 -- 172.21.15.32:0/3736448028 >> 172.21.15.32:0/3736448028 conn(0x7f36680fb650 msgr2=0x7f36680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 -- 172.21.15.32:0/3736448028 shutdown_connections 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 -- 172.21.15.32:0/3736448028 wait complete. 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 Processor -- start 2024-06-03T01:17:04.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.562+0000 7f3670590700 1 -- start start 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f3670590700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f3670590700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 0x7f366819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f3670590700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366819d200 con 0x7f36680ffda0 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f3670590700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681966b0 con 0x7f3668100750 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41268/0 (socket says 172.21.15.32:41268) 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 -- 172.21.15.32:0/3330158494 learned_addr learned my addr 172.21.15.32:0/3330158494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366db2b700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 0x7f366819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 -- 172.21.15.32:0/3330158494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 msgr2=0x7f366819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:04.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 0x7f366819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 -- 172.21.15.32:0/3330158494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640095f0 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f366e32c700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3664006010 tx=0x7f366400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366401d070 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.563+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668196930 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.564+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3664004fa0 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.564+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366400f690 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.564+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668196e20 con 0x7f36680ffda0 2024-06-03T01:17:04.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.565+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c005320 con 0x7f36680ffda0 2024-06-03T01:17:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.565+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3664004af0 con 0x7f36680ffda0 2024-06-03T01:17:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.566+0000 7f365f7fe700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 0x7f365406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.566+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f366408caa0 con 0x7f36680ffda0 2024-06-03T01:17:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.566+0000 7f366db2b700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 0x7f365406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.566+0000 7f366db2b700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 0x7f365406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36681066b0 tx=0x7f365800a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:04.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.570+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f366405b0d0 con 0x7f36680ffda0 2024-06-03T01:17:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.767+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f364c005f70 con 0x7f36680ffda0 2024-06-03T01:17:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.768+0000 7f365f7fe700 1 -- 172.21.15.32:0/3330158494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3664027020 con 0x7f36680ffda0 2024-06-03T01:17:04.768 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 msgr2=0x7f365406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 0x7f365406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36681066b0 tx=0x7f365800a300 comp rx=0 tx=0).stop 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 msgr2=0x7f366819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3664006010 tx=0x7f366400bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 shutdown_connections 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680ffda0 0x7f366819c630 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f365406c4e0 0x7f365406e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 --2- 172.21.15.32:0/3330158494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3668100750 0x7f366819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 >> 172.21.15.32:0/3330158494 conn(0x7f36680fb650 msgr2=0x7f3668103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 shutdown_connections 2024-06-03T01:17:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:04.770+0000 7f3670590700 1 -- 172.21.15.32:0/3330158494 wait complete. 2024-06-03T01:17:05.417 INFO:tasks.cephadm:Setting up client nodes... 2024-06-03T01:17:05.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-03T01:17:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:05.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:05 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3330158494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:17:05.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:05 smithi032 ceph-mon[29360]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:05.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:05 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3330158494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:17:05.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:05 smithi044 ceph-mon[36956]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/1376838285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0101740 msgr2=0x7f4ca0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/1376838285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0101740 0x7f4ca0103b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009910 tx=0x7f4c90009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/1376838285 shutdown_connections 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/1376838285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0104070 0x7f4ca0106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/1376838285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0101740 0x7f4ca0103b30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/1376838285 >> 172.21.15.32:0/1376838285 conn(0x7f4ca00fb150 msgr2=0x7f4ca00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/1376838285 shutdown_connections 2024-06-03T01:17:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.229+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/1376838285 wait complete. 2024-06-03T01:17:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 Processor -- start 2024-06-03T01:17:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 -- start start 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0101740 0x7f4ca01010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00ffc30 con 0x7f4ca0104070 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4ca5cf6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00ffd70 con 0x7f4ca0101740 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4c9effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4c9effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58560/0 (socket says 172.21.15.32:58560) 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.230+0000 7f4c9effd700 1 -- 172.21.15.32:0/2940780170 learned_addr learned my addr 172.21.15.32:0/2940780170 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9f7fe700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0101740 0x7f4ca01010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9effd700 1 -- 172.21.15.32:0/2940780170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0101740 msgr2=0x7f4ca01010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9effd700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0101740 0x7f4ca01010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9effd700 1 -- 172.21.15.32:0/2940780170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900095f0 con 0x7f4ca0104070 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9effd700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400d8d0 tx=0x7f4c9400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c940098a0 con 0x7f4ca0104070 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca0100050 con 0x7f4ca0104070 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c94010460 con 0x7f4ca0104070 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9400f5f0 con 0x7f4ca0104070 2024-06-03T01:17:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.231+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca019e500 con 0x7f4ca0104070 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.232+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c005320 con 0x7f4ca0104070 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.233+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4c94009a00 con 0x7f4ca0104070 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.233+0000 7f4c9cff9700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 0x7f4c8806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.234+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4c9408b210 con 0x7f4ca0104070 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.234+0000 7f4c9f7fe700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 0x7f4c8806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.234+0000 7f4c9f7fe700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 0x7f4c8806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006010 tx=0x7f4c9000b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.238+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c94059840 con 0x7f4ca0104070 2024-06-03T01:17:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.523+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4c8c005190 con 0x7f4ca0104070 2024-06-03T01:17:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.527+0000 7f4c9cff9700 1 -- 172.21.15.32:0/2940780170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4c940593d0 con 0x7f4ca0104070 2024-06-03T01:17:07.527 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-06-03T01:17:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCTGV1m+ghGHxAAKrrudBFsMwmjTzCFiYXFTQ== 2024-06-03T01:17:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 msgr2=0x7f4c8806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 0x7f4c8806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006010 tx=0x7f4c9000b540 comp rx=0 tx=0).stop 2024-06-03T01:17:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 msgr2=0x7f4ca00ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400d8d0 tx=0x7f4c9400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 shutdown_connections 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4c8806c4e0 0x7f4c8806e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ca0101740 0x7f4ca01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 --2- 172.21.15.32:0/2940780170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca0104070 0x7f4ca00ff6f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 >> 172.21.15.32:0/2940780170 conn(0x7f4ca00fb150 msgr2=0x7f4ca00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 shutdown_connections 2024-06-03T01:17:07.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:07.529+0000 7f4ca5cf6700 1 -- 172.21.15.32:0/2940780170 wait complete. 2024-06-03T01:17:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:07 smithi032 ceph-mon[29360]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:07 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2940780170' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:07 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2940780170' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:17:08.127 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:17:08.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-03T01:17:08.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-03T01:17:08.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-03T01:17:08.195 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:07 smithi044 ceph-mon[36956]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:08.195 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:07 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2940780170' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:08.195 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:07 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2940780170' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:17:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi044/config 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2117700520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 msgr2=0x7f5bcc1064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2117700520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc1064c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009910 tx=0x7f5bc0009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2117700520 shutdown_connections 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2117700520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc1064c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2117700520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 0x7f5bcc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2117700520 >> 172.21.15.44:0/2117700520 conn(0x7f5bcc0fb150 msgr2=0x7f5bcc0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2117700520 shutdown_connections 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.002+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2117700520 wait complete. 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 Processor -- start 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 -- start start 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 0x7f5bcc101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0ffc90 con 0x7f5bcc1017a0 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd3d0f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0ffdd0 con 0x7f5bcc1040d0 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53002/0 (socket says 172.21.15.44:53002) 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 -- 172.21.15.44:0/2514995812 learned_addr learned my addr 172.21.15.44:0/2514995812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd1aab700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 0x7f5bcc101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 -- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 msgr2=0x7f5bcc101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 0x7f5bcc101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 -- 172.21.15.44:0/2514995812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00095f0 con 0x7f5bcc1040d0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.003+0000 7f5bd12aa700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000bd70 tx=0x7f5bc000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.004+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc001d070 con 0x7f5bcc1040d0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.004+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5bc0005940 con 0x7f5bcc1040d0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.004+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc100050 con 0x7f5bcc1040d0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.004+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc0021430 con 0x7f5bcc1040d0 2024-06-03T01:17:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.004+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc100540 con 0x7f5bcc1040d0 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.005+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc1906a0 con 0x7f5bcc1040d0 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.005+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5bc000f860 con 0x7f5bcc1040d0 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.006+0000 7f5bcaffd700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 0x7f5bb406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.006+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5bc008c8e0 con 0x7f5bcc1040d0 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.006+0000 7f5bd1aab700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 0x7f5bb406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.006+0000 7f5bd1aab700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 0x7f5bb406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00a400 tx=0x7f5bbc00d040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.010+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5bc005ae90 con 0x7f5bcc1040d0 2024-06-03T01:17:10.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:09 smithi044 ceph-mon[36956]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:10.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.286+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5bcc02d100 con 0x7f5bcc1040d0 2024-06-03T01:17:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.291+0000 7f5bcaffd700 1 -- 172.21.15.44:0/2514995812 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5bc005aa20 con 0x7f5bcc1040d0 2024-06-03T01:17:10.292 INFO:teuthology.orchestra.run.smithi044.stdout:[client.1] 2024-06-03T01:17:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCWGV1mbyQ/ERAAbwZfa0Exa9Z+9gZ/GEWYbg== 2024-06-03T01:17:10.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 msgr2=0x7f5bb406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 0x7f5bb406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00a400 tx=0x7f5bbc00d040 comp rx=0 tx=0).stop 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 msgr2=0x7f5bcc0ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000bd70 tx=0x7f5bc000bda0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 shutdown_connections 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bcc1017a0 0x7f5bcc101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5bb406c4e0 0x7f5bb406e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 --2- 172.21.15.44:0/2514995812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bcc1040d0 0x7f5bcc0ff750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.293+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 >> 172.21.15.44:0/2514995812 conn(0x7f5bcc0fb150 msgr2=0x7f5bcc0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.294+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 shutdown_connections 2024-06-03T01:17:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:17:10.294+0000 7f5bd3d0f700 1 -- 172.21.15.44:0/2514995812 wait complete. 2024-06-03T01:17:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:09 smithi032 ceph-mon[29360]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:10.932 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:17:10.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-03T01:17:10.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-03T01:17:11.015 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-03T01:17:11.015 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-03T01:17:11.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-03T01:17:11.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:10 smithi044 ceph-mon[36956]: from='client.? 172.21.15.44:0/2514995812' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:11.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:10 smithi044 ceph-mon[36956]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:11.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:10 smithi044 ceph-mon[36956]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:17:11.239 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:11.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:10 smithi032 ceph-mon[29360]: from='client.? 172.21.15.44:0/2514995812' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:10 smithi032 ceph-mon[29360]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:17:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:10 smithi032 ceph-mon[29360]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:17:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:11 smithi044 ceph-mon[36956]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:11 smithi032 ceph-mon[29360]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 -- 172.21.15.32:0/3278475320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 msgr2=0x7f6a4c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 --2- 172.21.15.32:0/3278475320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c10cfe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009910 tx=0x7f6a34009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 -- 172.21.15.32:0/3278475320 shutdown_connections 2024-06-03T01:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 --2- 172.21.15.32:0/3278475320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c10cfe0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 --2- 172.21.15.32:0/3278475320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 0x7f6a4c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 -- 172.21.15.32:0/3278475320 >> 172.21.15.32:0/3278475320 conn(0x7f6a4c0fb650 msgr2=0x7f6a4c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 -- 172.21.15.32:0/3278475320 shutdown_connections 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.855+0000 7f6a5269d700 1 -- 172.21.15.32:0/3278475320 wait complete. 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 Processor -- start 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 -- start start 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 0x7f6a4c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c19f3f0 con 0x7f6a4c100320 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a5269d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1988a0 con 0x7f6a4c0ff970 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58570/0 (socket says 172.21.15.32:58570) 2024-06-03T01:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a43fff700 1 -- 172.21.15.32:0/2026006360 learned_addr learned my addr 172.21.15.32:0/2026006360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.856+0000 7f6a4bfff700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 0x7f6a4c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a43fff700 1 -- 172.21.15.32:0/2026006360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 msgr2=0x7f6a4c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a43fff700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 0x7f6a4c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a43fff700 1 -- 172.21.15.32:0/2026006360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340095f0 con 0x7f6a4c100320 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a43fff700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006010 tx=0x7f6a340055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3401d070 con 0x7f6a4c100320 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a34005dc0 con 0x7f6a4c100320 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c198b20 con 0x7f6a4c100320 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a34021640 con 0x7f6a4c100320 2024-06-03T01:17:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.857+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c199070 con 0x7f6a4c100320 2024-06-03T01:17:12.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.858+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c10a750 con 0x7f6a4c100320 2024-06-03T01:17:12.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.859+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6a3400f880 con 0x7f6a4c100320 2024-06-03T01:17:12.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.859+0000 7f6a49ffb700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 0x7f6a3806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:12.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.859+0000 7f6a4bfff700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 0x7f6a3806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:12.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.859+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6a3408c720 con 0x7f6a4c100320 2024-06-03T01:17:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.860+0000 7f6a4bfff700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 0x7f6a3806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c005950 tx=0x7f6a3c00b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:12.864+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a3405ad90 con 0x7f6a4c100320 2024-06-03T01:17:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.121+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a4c04f2e0 con 0x7f6a4c100320 2024-06-03T01:17:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.124+0000 7f6a49ffb700 1 -- 172.21.15.32:0/2026006360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173039 (secure 0 0 0) 0x7f6a3405a920 con 0x7f6a4c100320 2024-06-03T01:17:13.128 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:13.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 msgr2=0x7f6a3806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:13.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 0x7f6a3806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c005950 tx=0x7f6a3c00b410 comp rx=0 tx=0).stop 2024-06-03T01:17:13.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 msgr2=0x7f6a4c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:13.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006010 tx=0x7f6a340055f0 comp rx=0 tx=0).stop 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 shutdown_connections 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6a3806c4e0 0x7f6a3806e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a4c0ff970 0x7f6a4c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 --2- 172.21.15.32:0/2026006360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a4c100320 0x7f6a4c19ed60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 >> 172.21.15.32:0/2026006360 conn(0x7f6a4c0fb650 msgr2=0x7f6a4c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 shutdown_connections 2024-06-03T01:17:13.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:13.126+0000 7f6a5269d700 1 -- 172.21.15.32:0/2026006360 wait complete. 2024-06-03T01:17:13.649 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"active_gid":14219,"active_name":"smithi032.kfcfaa","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2678186916},{"type":"v1","addr":"172.21.15.32:6801","nonce":2678186916}]},"active_addr":"172.21.15.32:6801/2678186916","active_change":"2024-06-03T01:13:28.845722+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14242,"name":"smithi044.wszjlw","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/","prometheus":"http://172.21.15.32:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2139490011}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3335786446}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4112439105}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1080069639}]}]} 2024-06-03T01:17:13.653 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-03T01:17:13.653 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-03T01:17:13.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:17:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:13 smithi044 ceph-mon[36956]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:13 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2026006360' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T01:17:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:14.231 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:13 smithi032 ceph-mon[29360]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:14.231 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:13 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2026006360' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T01:17:14.231 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:15.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 -- 172.21.15.32:0/2303781449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 msgr2=0x7f99001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 --2- 172.21.15.32:0/2303781449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99001058a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009910 tx=0x7f98e8009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 -- 172.21.15.32:0/2303781449 shutdown_connections 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 --2- 172.21.15.32:0/2303781449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99001058a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 --2- 172.21.15.32:0/2303781449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 0x7f9900101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.318+0000 7f99058b7700 1 -- 172.21.15.32:0/2303781449 >> 172.21.15.32:0/2303781449 conn(0x7f99000fc750 msgr2=0x7f99000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.319+0000 7f99058b7700 1 -- 172.21.15.32:0/2303781449 shutdown_connections 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.319+0000 7f99058b7700 1 -- 172.21.15.32:0/2303781449 wait complete. 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.319+0000 7f99058b7700 1 Processor -- start 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.319+0000 7f99058b7700 1 -- start start 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99058b7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 0x7f9900073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99058b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99058b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900073b10 con 0x7f9900101850 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99058b7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900073c50 con 0x7f9900100ea0 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58590/0 (socket says 172.21.15.32:58590) 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 -- 172.21.15.32:0/1561063783 learned_addr learned my addr 172.21.15.32:0/1561063783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98feffd700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 0x7f9900073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 -- 172.21.15.32:0/1561063783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 msgr2=0x7f9900073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 0x7f9900073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 -- 172.21.15.32:0/1561063783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80095f0 con 0x7f9900101850 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f98f67fc700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98e80092e0 tx=0x7f98e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e801d070 con 0x7f9900101850 2024-06-03T01:17:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98e8004fa0 con 0x7f9900101850 2024-06-03T01:17:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9900073ed0 con 0x7f9900101850 2024-06-03T01:17:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.320+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e800f690 con 0x7f9900101850 2024-06-03T01:17:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.321+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99001a2e30 con 0x7f9900101850 2024-06-03T01:17:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.321+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99001092a0 con 0x7f9900101850 2024-06-03T01:17:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.322+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f98e8022470 con 0x7f9900101850 2024-06-03T01:17:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.323+0000 7f99048b5700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 0x7f98ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.323+0000 7f98feffd700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 0x7f98ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.323+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f98e808cb20 con 0x7f9900101850 2024-06-03T01:17:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.323+0000 7f98feffd700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 0x7f98ec06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005950 tx=0x7f98f000a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.327+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f98e80577e0 con 0x7f9900101850 2024-06-03T01:17:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.527+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99000745f0 con 0x7f9900101850 2024-06-03T01:17:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.528+0000 7f99048b5700 1 -- 172.21.15.32:0/1561063783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11287 (secure 0 0 0) 0x7f98e8027020 con 0x7f9900101850 2024-06-03T01:17:15.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:15.528 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","created":"2024-06-03T01:11:14.450938+0000","modified":"2024-06-03T01:16:57.131992+0000","last_up_change":"2024-06-03T01:16:56.126935+0000","last_in_change":"2024-06-03T01:16:37.676175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T01:15:43.366528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ddd78453-9794-4382-b3f3-6dbe73ec2d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6803","nonce":564758788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6805","nonce":564758788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6809","nonce":564758788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6807","nonce":564758788}]},"public_addr":"172.21.15.32:6803/564758788","cluster_addr":"172.21.15.32:6805/564758788","heartbeat_back_addr":"172.21.15.32:6809/564758788","heartbeat_front_addr":"172.21.15.32:6807/564758788","state":["exists","up"]},{"osd":1,"uuid":"cb1be659-330d-40c2-9e80-9430b3bd8a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6811","nonce":813215710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6813","nonce":813215710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6817","nonce":813215710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6815","nonce":813215710}]},"public_addr":"172.21.15.32:6811/813215710","cluster_addr":"172.21.15.32:6813/813215710","heartbeat_back_addr":"172.21.15.32:6817/813215710","heartbeat_front_addr":"172.21.15.32:6815/813215710","state":["exists","up"]},{"osd":2,"uuid":"4c0dbb0a-a82a-4a4e-a268-24795b04ee7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6819","nonce":3632513352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6821","nonce":3632513352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6825","nonce":3632513352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6823","nonce":3632513352}]},"public_addr":"172.21.15.32:6819/3632513352","cluster_addr":"172.21.15.32:6821/3632513352","heartbeat_back_addr":"172.21.15.32:6825/3632513352","heartbeat_front_addr":"172.21.15.32:6823/3632513352","state":["exists","up"]},{"osd":3,"uuid":"60fd88fd-b2e7-41d0-8479-bfa395ef9d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6801","nonce":1384539104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6803","nonce":1384539104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6807","nonce":1384539104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6805","nonce":1384539104}]},"public_addr":"172.21.15.44:6801/1384539104","cluster_addr":"172.21.15.44:6803/1384539104","heartbeat_back_addr":"172.21.15.44:6807/1384539104","heartbeat_front_addr":"172.21.15.44:6805/1384539104","state":["exists","up"]},{"osd":4,"uuid":"5a48b78c-eda0-4ff9-9646-d5df7e89b603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6809","nonce":2875962035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6811","nonce":2875962035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6815","nonce":2875962035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6813","nonce":2875962035}]},"public_addr":"172.21.15.44:6809/2875962035","cluster_addr":"172.21.15.44:6811/2875962035","heartbeat_back_addr":"172.21.15.44:6815/2875962035","heartbeat_front_addr":"172.21.15.44:6813/2875962035","state":["exists","up"]},{"osd":5,"uuid":"b97503ff-62f3-4ae9-aba2-c0ad20ae00a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6817","nonce":3789906029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6819","nonce":3789906029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6823","nonce":3789906029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6821","nonce":3789906029}]},"public_addr":"172.21.15.44:6817/3789906029","cluster_addr":"172.21.15.44:6819/3789906029","heartbeat_back_addr":"172.21.15.44:6823/3789906029","heartbeat_front_addr":"172.21.15.44:6821/3789906029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:14:46.681798+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:13.540229+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:40.872057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:04.951207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:30.118175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:54.924738+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1695685766":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6800/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/2872286231":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1166022518":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/84030829":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6801/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/2914664298":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1891444588":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1129696663":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1434819821":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/2007986688":"2024-06-04T01:12:14.539594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 msgr2=0x7f98ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 0x7f98ec06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005950 tx=0x7f98f000a300 comp rx=0 tx=0).stop 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 msgr2=0x7f99000735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98e80092e0 tx=0x7f98e800bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 shutdown_connections 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f98ec06c4e0 0x7f98ec06e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9900100ea0 0x7f9900073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 --2- 172.21.15.32:0/1561063783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9900101850 0x7f99000735d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 >> 172.21.15.32:0/1561063783 conn(0x7f99000fc750 msgr2=0x7f99000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 shutdown_connections 2024-06-03T01:17:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:15.530+0000 7f99058b7700 1 -- 172.21.15.32:0/1561063783 wait complete. 2024-06-03T01:17:15.808 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:15 smithi032 ceph-mon[29360]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:15.808 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:15 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1561063783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:15.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:15 smithi044 ceph-mon[36956]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:15 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1561063783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:16.167 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-03T01:17:16.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:17:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.965+0000 7fdf192ed700 1 -- 172.21.15.32:0/3782519307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140739d0 msgr2=0x7fdf1410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.965+0000 7fdf192ed700 1 --2- 172.21.15.32:0/3782519307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140739d0 0x7fdf1410cfc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009910 tx=0x7fdf04009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 -- 172.21.15.32:0/3782519307 shutdown_connections 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 --2- 172.21.15.32:0/3782519307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140739d0 0x7fdf1410cfc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 --2- 172.21.15.32:0/3782519307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140730b0 0x7fdf14073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:17.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 -- 172.21.15.32:0/3782519307 >> 172.21.15.32:0/3782519307 conn(0x7fdf140fc750 msgr2=0x7fdf140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:17.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 -- 172.21.15.32:0/3782519307 shutdown_connections 2024-06-03T01:17:17.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 -- 172.21.15.32:0/3782519307 wait complete. 2024-06-03T01:17:17.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 Processor -- start 2024-06-03T01:17:17.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.966+0000 7fdf192ed700 1 -- start start 2024-06-03T01:17:17.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf192ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:17.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf192ed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140739d0 0x7fdf1419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf192ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1419f480 con 0x7fdf140730b0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf192ed700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14198930 con 0x7fdf140739d0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40850/0 (socket says 172.21.15.32:40850) 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 -- 172.21.15.32:0/2878892893 learned_addr learned my addr 172.21.15.32:0/2878892893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf127fc700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140739d0 0x7fdf1419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 -- 172.21.15.32:0/2878892893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140739d0 msgr2=0x7fdf1419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140739d0 0x7fdf1419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 -- 172.21.15.32:0/2878892893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040095f0 con 0x7fdf140730b0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf12ffd700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b4b0 tx=0x7fdefc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc011800 con 0x7fdf140730b0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdefc011e10 con 0x7fdf140730b0 2024-06-03T01:17:17.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.967+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf14198c10 con 0x7fdf140730b0 2024-06-03T01:17:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.968+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc00f2e0 con 0x7fdf140730b0 2024-06-03T01:17:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.968+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf14199160 con 0x7fdf140730b0 2024-06-03T01:17:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.969+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1410a6c0 con 0x7fdf140730b0 2024-06-03T01:17:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.969+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdefc010380 con 0x7fdf140730b0 2024-06-03T01:17:17.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.970+0000 7fdf0bfff700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.970+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdefc08afe0 con 0x7fdf140730b0 2024-06-03T01:17:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.970+0000 7fdf127fc700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:17.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.970+0000 7fdf127fc700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf140fde90 tx=0x7fdf0401a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:17.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:17.974+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdefc0595d0 con 0x7fdf140730b0 2024-06-03T01:17:18.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:17 smithi032 ceph-mon[29360]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:18.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.173+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf1404f2e0 con 0x7fdf140730b0 2024-06-03T01:17:18.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.174+0000 7fdf0bfff700 1 -- 172.21.15.32:0/2878892893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11287 (secure 0 0 0) 0x7fdefc059160 con 0x7fdf140730b0 2024-06-03T01:17:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:18.175 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"12a144f0-2146-11ef-bc9b-c7b262605968","created":"2024-06-03T01:11:14.450938+0000","modified":"2024-06-03T01:16:57.131992+0000","last_up_change":"2024-06-03T01:16:56.126935+0000","last_in_change":"2024-06-03T01:16:37.676175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T01:15:43.366528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ddd78453-9794-4382-b3f3-6dbe73ec2d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6803","nonce":564758788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6805","nonce":564758788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6809","nonce":564758788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":564758788},{"type":"v1","addr":"172.21.15.32:6807","nonce":564758788}]},"public_addr":"172.21.15.32:6803/564758788","cluster_addr":"172.21.15.32:6805/564758788","heartbeat_back_addr":"172.21.15.32:6809/564758788","heartbeat_front_addr":"172.21.15.32:6807/564758788","state":["exists","up"]},{"osd":1,"uuid":"cb1be659-330d-40c2-9e80-9430b3bd8a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6811","nonce":813215710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6813","nonce":813215710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6817","nonce":813215710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":813215710},{"type":"v1","addr":"172.21.15.32:6815","nonce":813215710}]},"public_addr":"172.21.15.32:6811/813215710","cluster_addr":"172.21.15.32:6813/813215710","heartbeat_back_addr":"172.21.15.32:6817/813215710","heartbeat_front_addr":"172.21.15.32:6815/813215710","state":["exists","up"]},{"osd":2,"uuid":"4c0dbb0a-a82a-4a4e-a268-24795b04ee7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6819","nonce":3632513352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6821","nonce":3632513352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6825","nonce":3632513352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3632513352},{"type":"v1","addr":"172.21.15.32:6823","nonce":3632513352}]},"public_addr":"172.21.15.32:6819/3632513352","cluster_addr":"172.21.15.32:6821/3632513352","heartbeat_back_addr":"172.21.15.32:6825/3632513352","heartbeat_front_addr":"172.21.15.32:6823/3632513352","state":["exists","up"]},{"osd":3,"uuid":"60fd88fd-b2e7-41d0-8479-bfa395ef9d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6801","nonce":1384539104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6803","nonce":1384539104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6807","nonce":1384539104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1384539104},{"type":"v1","addr":"172.21.15.44:6805","nonce":1384539104}]},"public_addr":"172.21.15.44:6801/1384539104","cluster_addr":"172.21.15.44:6803/1384539104","heartbeat_back_addr":"172.21.15.44:6807/1384539104","heartbeat_front_addr":"172.21.15.44:6805/1384539104","state":["exists","up"]},{"osd":4,"uuid":"5a48b78c-eda0-4ff9-9646-d5df7e89b603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6809","nonce":2875962035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6811","nonce":2875962035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6815","nonce":2875962035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2875962035},{"type":"v1","addr":"172.21.15.44:6813","nonce":2875962035}]},"public_addr":"172.21.15.44:6809/2875962035","cluster_addr":"172.21.15.44:6811/2875962035","heartbeat_back_addr":"172.21.15.44:6815/2875962035","heartbeat_front_addr":"172.21.15.44:6813/2875962035","state":["exists","up"]},{"osd":5,"uuid":"b97503ff-62f3-4ae9-aba2-c0ad20ae00a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6817","nonce":3789906029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6819","nonce":3789906029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6823","nonce":3789906029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3789906029},{"type":"v1","addr":"172.21.15.44:6821","nonce":3789906029}]},"public_addr":"172.21.15.44:6817/3789906029","cluster_addr":"172.21.15.44:6819/3789906029","heartbeat_back_addr":"172.21.15.44:6823/3789906029","heartbeat_front_addr":"172.21.15.44:6821/3789906029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:14:46.681798+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:13.540229+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:15:40.872057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:04.951207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:30.118175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:16:54.924738+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1695685766":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6800/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/2872286231":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1166022518":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/84030829":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/1578498944":"2024-06-04T01:13:28.845604+0000","172.21.15.32:6801/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:0/2914664298":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1891444588":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/1129696663":"2024-06-04T01:13:28.845604+0000","172.21.15.32:0/1434819821":"2024-06-04T01:11:41.610662+0000","172.21.15.32:6800/3147998267":"2024-06-04T01:12:14.539594+0000","172.21.15.32:6801/458796220":"2024-06-04T01:11:41.610662+0000","172.21.15.32:0/2007986688":"2024-06-04T01:12:14.539594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:17:18.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 msgr2=0x7fdf0006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf140fde90 tx=0x7fdf0401a040 comp rx=0 tx=0).stop 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 msgr2=0x7fdf1419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b4b0 tx=0x7fdefc00b7c0 comp rx=0 tx=0).stop 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 shutdown_connections 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf140730b0 0x7fdf1419e8b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 --2- 172.21.15.32:0/2878892893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf140739d0 0x7fdf1419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.176+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 >> 172.21.15.32:0/2878892893 conn(0x7fdf140fc750 msgr2=0x7fdf14107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.177+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 shutdown_connections 2024-06-03T01:17:18.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:18.177+0000 7fdf192ed700 1 -- 172.21.15.32:0/2878892893 wait complete. 2024-06-03T01:17:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:17 smithi044 ceph-mon[36956]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:18.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-03T01:17:18.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-03T01:17:18.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-03T01:17:18.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-03T01:17:18.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-03T01:17:18.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-03T01:17:19.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:18 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2878892893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:19.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:18 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2878892893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:17:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:20.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:19 smithi032 ceph-mon[29360]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:20.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:19 smithi044 ceph-mon[36956]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:22.220 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:21 smithi032 ceph-mon[29360]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:21 smithi044 ceph-mon[36956]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:23 smithi044 ceph-mon[36956]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:24.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:23 smithi032 ceph-mon[29360]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 -- 172.21.15.32:0/269405532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 msgr2=0x7fd65c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 --2- 172.21.15.32:0/269405532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c1058a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd650009910 tx=0x7fd650009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 -- 172.21.15.32:0/269405532 shutdown_connections 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 --2- 172.21.15.32:0/269405532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c1058a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 --2- 172.21.15.32:0/269405532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 0x7fd65c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 -- 172.21.15.32:0/269405532 >> 172.21.15.32:0/269405532 conn(0x7fd65c0fc750 msgr2=0x7fd65c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 -- 172.21.15.32:0/269405532 shutdown_connections 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.886+0000 7fd662737700 1 -- 172.21.15.32:0/269405532 wait complete. 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 Processor -- start 2024-06-03T01:17:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 -- start start 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 0x7fd65c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c19f430 con 0x7fd65c101850 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.887+0000 7fd662737700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1988e0 con 0x7fd65c100ea0 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40864/0 (socket says 172.21.15.32:40864) 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 -- 172.21.15.32:0/541017953 learned_addr learned my addr 172.21.15.32:0/541017953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65bfff700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 0x7fd65c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 -- 172.21.15.32:0/541017953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 msgr2=0x7fd65c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 0x7fd65c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500095f0 con 0x7fd65c101850 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd65b7fe700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6500092e0 tx=0x7fd65000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65001d070 con 0x7fd65c101850 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd650004fa0 con 0x7fd65c101850 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.888+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c198b60 con 0x7fd65c101850 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.889+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65000f690 con 0x7fd65c101850 2024-06-03T01:17:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.889+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c199050 con 0x7fd65c101850 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.889+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd65c10e0c0 con 0x7fd65c101850 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.890+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd650022470 con 0x7fd65c101850 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd6597fa700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 0x7fd64406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd65bfff700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 0x7fd64406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd65008d680 con 0x7fd65c101850 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd6597fa700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 0x7fd6440745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd644074c90 con 0x7fd6440721c0 2024-06-03T01:17:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd65008dae0 con 0x7fd65c101850 2024-06-03T01:17:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd660cd4700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 0x7fd6440745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.891+0000 7fd65bfff700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 0x7fd64406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005950 tx=0x7fd64c00a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.892+0000 7fd660cd4700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 0x7fd6440745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.892+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== osd.1 v2:172.21.15.32:6810/813215710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd644074c90 con 0x7fd6440721c0 2024-06-03T01:17:24.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.908+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 --> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd65c199500 con 0x7fd6440721c0 2024-06-03T01:17:24.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.908+0000 7fd6597fa700 1 -- 172.21.15.32:0/541017953 <== osd.1 v2:172.21.15.32:6810/813215710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd65c199500 con 0x7fd6440721c0 2024-06-03T01:17:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.908+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 msgr2=0x7fd6440745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.908+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 0x7fd6440745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 msgr2=0x7fd64406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 0x7fd64406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005950 tx=0x7fd64c00a300 comp rx=0 tx=0).stop 2024-06-03T01:17:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 msgr2=0x7fd65c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6500092e0 tx=0x7fd65000bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 shutdown_connections 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd64406c4e0 0x7fd64406e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd65c100ea0 0x7fd65c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:6810/813215710,v1:172.21.15.32:6811/813215710] conn(0x7fd6440721c0 0x7fd6440745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 --2- 172.21.15.32:0/541017953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65c101850 0x7fd65c19eda0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 >> 172.21.15.32:0/541017953 conn(0x7fd65c0fc750 msgr2=0x7fd65c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 shutdown_connections 2024-06-03T01:17:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:24.909+0000 7fd662737700 1 -- 172.21.15.32:0/541017953 wait complete. 2024-06-03T01:17:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 -- 172.21.15.32:0/1103613243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c0ff340 msgr2=0x7f334c0ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1103613243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c0ff340 0x7f334c0ff7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3340009910 tx=0x7f3340009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 -- 172.21.15.32:0/1103613243 shutdown_connections 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1103613243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c0ff340 0x7f334c0ff7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1103613243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c105350 0x7f334c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 -- 172.21.15.32:0/1103613243 >> 172.21.15.32:0/1103613243 conn(0x7f334c074e30 msgr2=0x7f334c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 -- 172.21.15.32:0/1103613243 shutdown_connections 2024-06-03T01:17:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.089+0000 7f3351c8d700 1 -- 172.21.15.32:0/1103613243 wait complete. 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.090+0000 7f3351c8d700 1 Processor -- start 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.090+0000 7f3351c8d700 1 -- start start 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f3351c8d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c0ff340 0x7f334c19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f3351c8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f3351c8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c19c1a0 con 0x7f334c105350 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f3351c8d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c195600 con 0x7f334c0ff340 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40878/0 (socket says 172.21.15.32:40878) 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 -- 172.21.15.32:0/1073027320 learned_addr learned my addr 172.21.15.32:0/1073027320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334b7fe700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c0ff340 0x7f334c19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 -- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c0ff340 msgr2=0x7f334c19b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c0ff340 0x7f334c19b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400095f0 con 0x7f334c105350 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.091+0000 7f334affd700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3340006010 tx=0x7f334000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334001f070 con 0x7f334c105350 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3340003680 con 0x7f334c105350 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c195880 con 0x7f334c105350 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334000f690 con 0x7f334c105350 2024-06-03T01:17:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c195d70 con 0x7f334c105350 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.092+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f334c10c6e0 con 0x7f334c105350 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.093+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f33400037f0 con 0x7f334c105350 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f3348ff9700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 0x7f333406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f334008f340 con 0x7f334c105350 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f334b7fe700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 0x7f333406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f3348ff9700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 0x7f33340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3334074c90 con 0x7f33340721c0 2024-06-03T01:17:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334008f7a0 con 0x7f334c105350 2024-06-03T01:17:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f334b7fe700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 0x7f333406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f333c00ac50 tx=0x7f333c00a380 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.094+0000 7f334bfff700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 0x7f33340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.095+0000 7f334bfff700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 0x7f33340745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.096+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== osd.4 v2:172.21.15.44:6808/2875962035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3334074c90 con 0x7f33340721c0 2024-06-03T01:17:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.111+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 --> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f334c04f2e0 con 0x7f33340721c0 2024-06-03T01:17:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.111+0000 7f3348ff9700 1 -- 172.21.15.32:0/1073027320 <== osd.4 v2:172.21.15.44:6808/2875962035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f334c04f2e0 con 0x7f33340721c0 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 msgr2=0x7f33340745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 0x7f33340745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 msgr2=0x7f333406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 0x7f333406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f333c00ac50 tx=0x7f333c00a380 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 msgr2=0x7f334c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3340006010 tx=0x7f334000bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 shutdown_connections 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:6808/2875962035,v1:172.21.15.44:6809/2875962035] conn(0x7f33340721c0 0x7f33340745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f333406c4e0 0x7f333406e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f334c0ff340 0x7f334c19b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 --2- 172.21.15.32:0/1073027320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f334c105350 0x7f334c19bb50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 >> 172.21.15.32:0/1073027320 conn(0x7f334c074e30 msgr2=0x7f334c103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 shutdown_connections 2024-06-03T01:17:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.112+0000 7f3351c8d700 1 -- 172.21.15.32:0/1073027320 wait complete. 2024-06-03T01:17:25.764 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:25 smithi032 ceph-mon[29360]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 -- 172.21.15.32:0/572725770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c100ea0 msgr2=0x7ff76c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 --2- 172.21.15.32:0/572725770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c100ea0 0x7ff76c101280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff768009910 tx=0x7ff768009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 -- 172.21.15.32:0/572725770 shutdown_connections 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 --2- 172.21.15.32:0/572725770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c101850 0x7ff76c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 --2- 172.21.15.32:0/572725770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c100ea0 0x7ff76c101280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 -- 172.21.15.32:0/572725770 >> 172.21.15.32:0/572725770 conn(0x7ff76c0fc750 msgr2=0x7ff76c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 -- 172.21.15.32:0/572725770 shutdown_connections 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.869+0000 7ff773b28700 1 -- 172.21.15.32:0/572725770 wait complete. 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 Processor -- start 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 -- start start 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c100ea0 0x7ff76c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c19f4b0 con 0x7ff76c101850 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.870+0000 7ff773b28700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c198960 con 0x7ff76c100ea0 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40906/0 (socket says 172.21.15.32:40906) 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 -- 172.21.15.32:0/3043452080 learned_addr learned my addr 172.21.15.32:0/3043452080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7718c4700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c100ea0 0x7ff76c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 -- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c100ea0 msgr2=0x7ff76c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c100ea0 0x7ff76c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680095f0 con 0x7ff76c101850 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff7710c3700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d900 tx=0x7ff75c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75c009900 con 0x7ff76c101850 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff75c010460 con 0x7ff76c101850 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff773b28700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c198c40 con 0x7ff76c101850 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75c00f5f0 con 0x7ff76c101850 2024-06-03T01:17:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.871+0000 7ff773b28700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c199190 con 0x7ff76c101850 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.872+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff754000fc0 con 0x7ff76c101850 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.873+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff75c0105d0 con 0x7ff76c101850 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.873+0000 7ff762ffd700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 0x7ff75806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff75c08b610 con 0x7ff76c101850 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff7718c4700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 0x7ff75806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff762ffd700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 0x7ff7580745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff758074c90 con 0x7ff7580721c0 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c08ba90 con 0x7ff76c101850 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff7718c4700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 0x7ff75806e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff768006010 tx=0x7ff76801a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.874+0000 7ff7720c5700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 0x7ff7580745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.875+0000 7ff7720c5700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 0x7ff7580745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.875+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== osd.5 v2:172.21.15.44:6816/3789906029 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff758074c90 con 0x7ff7580721c0 2024-06-03T01:17:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.891+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 --> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff754002cb0 con 0x7ff7580721c0 2024-06-03T01:17:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff762ffd700 1 -- 172.21.15.32:0/3043452080 <== osd.5 v2:172.21.15.44:6816/3789906029 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff754002cb0 con 0x7ff7580721c0 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 msgr2=0x7ff7580745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 0x7ff7580745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 msgr2=0x7ff75806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 0x7ff75806e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff768006010 tx=0x7ff76801a040 comp rx=0 tx=0).stop 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 msgr2=0x7ff76c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d900 tx=0x7ff75c00dcc0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 shutdown_connections 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:6816/3789906029,v1:172.21.15.44:6817/3789906029] conn(0x7ff7580721c0 0x7ff7580745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff75806c4e0 0x7ff75806e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76c100ea0 0x7ff76c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 --2- 172.21.15.32:0/3043452080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76c101850 0x7ff76c19ee20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 >> 172.21.15.32:0/3043452080 conn(0x7ff76c0fc750 msgr2=0x7ff76c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 shutdown_connections 2024-06-03T01:17:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:25.892+0000 7ff760ff9700 1 -- 172.21.15.32:0/3043452080 wait complete. 2024-06-03T01:17:25.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:25 smithi044 ceph-mon[36956]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:27.039 INFO:teuthology.orchestra.run.smithi032.stdout:146028888078 2024-06-03T01:17:27.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-03T01:17:28.044 INFO:teuthology.orchestra.run.smithi032.stdout:60129542172 2024-06-03T01:17:28.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-03T01:17:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:27 smithi044 ceph-mon[36956]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:28.317 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:27 smithi032 ceph-mon[29360]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 -- 172.21.15.32:0/626059963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c101840 msgr2=0x7f282c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 --2- 172.21.15.32:0/626059963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c101840 0x7f282c105890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2814009910 tx=0x7f2814009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 -- 172.21.15.32:0/626059963 shutdown_connections 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 --2- 172.21.15.32:0/626059963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c101840 0x7f282c105890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 --2- 172.21.15.32:0/626059963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c100e90 0x7f282c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 -- 172.21.15.32:0/626059963 >> 172.21.15.32:0/626059963 conn(0x7f282c0fc740 msgr2=0x7f282c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 -- 172.21.15.32:0/626059963 shutdown_connections 2024-06-03T01:17:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.730+0000 7f2832489700 1 -- 172.21.15.32:0/626059963 wait complete. 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 Processor -- start 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 -- start start 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c101840 0x7f282c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c19d240 con 0x7f282c100e90 2024-06-03T01:17:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f2832489700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1966f0 con 0x7f282c101840 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37956/0 (socket says 172.21.15.32:37956) 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 -- 172.21.15.32:0/1073682470 learned_addr learned my addr 172.21.15.32:0/1073682470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f28237fe700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c101840 0x7f282c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 -- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c101840 msgr2=0x7f282c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c101840 0x7f282c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.731+0000 7f282bfff700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140095f0 con 0x7f282c100e90 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f282bfff700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f281c00ada0 tx=0x7f281c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f281c00f070 con 0x7f282c100e90 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c1969d0 con 0x7f282c100e90 2024-06-03T01:17:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f281c00ae90 con 0x7f282c100e90 2024-06-03T01:17:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f281c005670 con 0x7f282c100e90 2024-06-03T01:17:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.732+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c196ef0 con 0x7f282c100e90 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.733+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f282c10e0b0 con 0x7f282c100e90 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.733+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f281c0058e0 con 0x7f282c100e90 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2829ffb700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 0x7f281806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f281c08ae10 con 0x7f282c100e90 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f28237fe700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 0x7f281806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2829ffb700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 0x7f28180745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2818074c90 con 0x7f28180721c0 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f281c0904f0 con 0x7f282c100e90 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f2830a26700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 0x7f28180745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.734+0000 7f28237fe700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 0x7f281806e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2814006010 tx=0x7f281401a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.735+0000 7f2830a26700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 0x7f28180745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.736+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== osd.3 v2:172.21.15.44:6800/1384539104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2818074c90 con 0x7f28180721c0 2024-06-03T01:17:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.751+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 --> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f282c04f2e0 con 0x7f28180721c0 2024-06-03T01:17:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2829ffb700 1 -- 172.21.15.32:0/1073682470 <== osd.3 v2:172.21.15.44:6800/1384539104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f282c04f2e0 con 0x7f28180721c0 2024-06-03T01:17:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 msgr2=0x7f28180745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 0x7f28180745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 msgr2=0x7f281806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 0x7f281806e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2814006010 tx=0x7f281401a040 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 msgr2=0x7f282c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f281c00ada0 tx=0x7f281c00c330 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 shutdown_connections 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c100e90 0x7f282c19c670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:6800/1384539104,v1:172.21.15.44:6801/1384539104] conn(0x7f28180721c0 0x7f28180745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f281806c4e0 0x7f281806e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 --2- 172.21.15.32:0/1073682470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282c101840 0x7f282c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 >> 172.21.15.32:0/1073682470 conn(0x7f282c0fc740 msgr2=0x7f282c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 shutdown_connections 2024-06-03T01:17:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:28.752+0000 7f2832489700 1 -- 172.21.15.32:0/1073682470 wait complete. 2024-06-03T01:17:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 -- 172.21.15.32:0/1954006628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 msgr2=0x7f8f501021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 --2- 172.21.15.32:0/1954006628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f501021e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009910 tx=0x7f8f44009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 -- 172.21.15.32:0/1954006628 shutdown_connections 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 --2- 172.21.15.32:0/1954006628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 0x7f8f5010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 --2- 172.21.15.32:0/1954006628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f501021e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 -- 172.21.15.32:0/1954006628 >> 172.21.15.32:0/1954006628 conn(0x7f8f500fb650 msgr2=0x7f8f500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 -- 172.21.15.32:0/1954006628 shutdown_connections 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.485+0000 7f8f56a94700 1 -- 172.21.15.32:0/1954006628 wait complete. 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 Processor -- start 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 -- start start 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 0x7f8f500fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501004e0 con 0x7f8f50101e00 2024-06-03T01:17:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f56a94700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50100620 con 0x7f8f50102720 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.486+0000 7f8f54830700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37936/0 (socket says 172.21.15.32:37936) 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 -- 172.21.15.32:0/354586516 learned_addr learned my addr 172.21.15.32:0/354586516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f4ffff700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 0x7f8f500fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 -- 172.21.15.32:0/354586516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 msgr2=0x7f8f500fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 0x7f8f500fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440095f0 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f54830700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006010 tx=0x7f8f4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4401d070 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f501008a0 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f4400f460 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.487+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f44017630 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.488+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501a2e50 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.488+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f5010e100 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.489+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8f4400f5d0 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.489+0000 7f8f4dffb700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 0x7f8f4006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8f4408c930 con 0x7f8f50101e00 2024-06-03T01:17:28.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4ffff700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 0x7f8f4006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4dffb700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 0x7f8f400745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f40074c90 con 0x7f8f400721c0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f440912a0 con 0x7f8f50101e00 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f55031700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 0x7f8f400745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f4ffff700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 0x7f8f4006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f50101250 tx=0x7f8f3c00b360 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.490+0000 7f8f55031700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 0x7f8f400745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.491+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== osd.2 v2:172.21.15.32:6818/3632513352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8f40074c90 con 0x7f8f400721c0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.509+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 --> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f5004f2e0 con 0x7f8f400721c0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.509+0000 7f8f4dffb700 1 -- 172.21.15.32:0/354586516 <== osd.2 v2:172.21.15.32:6818/3632513352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f5004f2e0 con 0x7f8f400721c0 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 msgr2=0x7f8f400745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 0x7f8f400745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 msgr2=0x7f8f4006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 0x7f8f4006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f50101250 tx=0x7f8f3c00b360 comp rx=0 tx=0).stop 2024-06-03T01:17:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 msgr2=0x7f8f500ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006010 tx=0x7f8f4400bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 shutdown_connections 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6818/3632513352,v1:172.21.15.32:6819/3632513352] conn(0x7f8f400721c0 0x7f8f400745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f50101e00 0x7f8f500ffa60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8f4006c4e0 0x7f8f4006e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 --2- 172.21.15.32:0/354586516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f50102720 0x7f8f500fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 >> 172.21.15.32:0/354586516 conn(0x7f8f500fb650 msgr2=0x7f8f50105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 shutdown_connections 2024-06-03T01:17:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:27.510+0000 7f8f56a94700 1 -- 172.21.15.32:0/354586516 wait complete. 2024-06-03T01:17:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:29.698 INFO:teuthology.orchestra.run.smithi032.stdout:167503724553 2024-06-03T01:17:29.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-03T01:17:30.146 INFO:teuthology.orchestra.run.smithi032.stdout:111669149716 2024-06-03T01:17:30.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-03T01:17:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:29 smithi044 ceph-mon[36956]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:30.244 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:29 smithi032 ceph-mon[29360]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:30.889 INFO:teuthology.orchestra.run.smithi032.stdout:81604378647 2024-06-03T01:17:30.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-03T01:17:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2495317038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 msgr2=0x7f5f60105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2495317038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f60105890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009910 tx=0x7f5f54009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:31.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2495317038 shutdown_connections 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2495317038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f60105890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2495317038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 0x7f5f60101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2495317038 >> 172.21.15.32:0/2495317038 conn(0x7f5f600fc740 msgr2=0x7f5f600feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.218+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2495317038 shutdown_connections 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.219+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2495317038 wait complete. 2024-06-03T01:17:31.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.219+0000 7f5f65f5b700 1 Processor -- start 2024-06-03T01:17:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f65f5b700 1 -- start start 2024-06-03T01:17:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f65f5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 0x7f5f60074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f65f5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f65f5b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f60075060 con 0x7f5f60101840 2024-06-03T01:17:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f65f5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6006df10 con 0x7f5f60100e90 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37986/0 (socket says 172.21.15.32:37986) 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f5effd700 1 -- 172.21.15.32:0/2453856859 learned_addr learned my addr 172.21.15.32:0/2453856859 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.220+0000 7f5f5f7fe700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 0x7f5f60074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f5effd700 1 -- 172.21.15.32:0/2453856859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 msgr2=0x7f5f60074940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f5effd700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 0x7f5f60074940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f5effd700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540095f0 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f5effd700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f540092e0 tx=0x7f5f5400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5401d070 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6006e190 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.221+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6006e680 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.222+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5f54005dc0 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.222+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f54017400 con 0x7f5f60101840 2024-06-03T01:17:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.222+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f4c000ff0 con 0x7f5f60101840 2024-06-03T01:17:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.224+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5f54003a60 con 0x7f5f60101840 2024-06-03T01:17:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.224+0000 7f5f5cff9700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.224+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5f5408cb80 con 0x7f5f60101840 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5cff9700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 0x7f5f480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f48074c90 con 0x7f5f480721c0 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5408cfe0 con 0x7f5f60101840 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5f7fe700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5ffff700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 0x7f5f480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5f7fe700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005950 tx=0x7f5f5000b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.225+0000 7f5f5ffff700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 0x7f5f480745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.226+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== osd.0 v2:172.21.15.32:6802/564758788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5f48074c90 con 0x7f5f480721c0 2024-06-03T01:17:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.246+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 --> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f4c002d70 con 0x7f5f480721c0 2024-06-03T01:17:31.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.246+0000 7f5f5cff9700 1 -- 172.21.15.32:0/2453856859 <== osd.0 v2:172.21.15.32:6802/564758788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5f4c002d70 con 0x7f5f480721c0 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.248+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 msgr2=0x7f5f480745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.248+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 0x7f5f480745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.248+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 msgr2=0x7f5f4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.248+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005950 tx=0x7f5f5000b410 comp rx=0 tx=0).stop 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 msgr2=0x7f5f6006d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f540092e0 tx=0x7f5f5400bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 shutdown_connections 2024-06-03T01:17:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6802/564758788,v1:172.21.15.32:6803/564758788] conn(0x7f5f480721c0 0x7f5f480745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f60100e90 0x7f5f60074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 --2- 172.21.15.32:0/2453856859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f60101840 0x7f5f6006d9d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 >> 172.21.15.32:0/2453856859 conn(0x7f5f600fc740 msgr2=0x7f5f600fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 shutdown_connections 2024-06-03T01:17:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:31.249+0000 7f5f65f5b700 1 -- 172.21.15.32:0/2453856859 wait complete. 2024-06-03T01:17:31.706 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:31.707 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:31 smithi044 ceph-mon[36956]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:32.225 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:31 smithi032 ceph-mon[29360]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:34.003 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:33 smithi032 ceph-mon[29360]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:33 smithi044 ceph-mon[36956]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:35.916 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:35 smithi032 ceph-mon[29360]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:35.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:35 smithi044 ceph-mon[36956]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- 172.21.15.32:0/1733423393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750101850 msgr2=0x7fb7501058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 --2- 172.21.15.32:0/1733423393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750101850 0x7fb7501058a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- 172.21.15.32:0/1733423393 shutdown_connections 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 --2- 172.21.15.32:0/1733423393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750101850 0x7fb7501058a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 --2- 172.21.15.32:0/1733423393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750100ea0 0x7fb750101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- 172.21.15.32:0/1733423393 >> 172.21.15.32:0/1733423393 conn(0x7fb7500fc750 msgr2=0x7fb7500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- 172.21.15.32:0/1733423393 shutdown_connections 2024-06-03T01:17:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- 172.21.15.32:0/1733423393 wait complete. 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 Processor -- start 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.092+0000 7fb7572e5700 1 -- start start 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750101850 0x7fb750196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750197540 con 0x7fb750100ea0 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75019b350 con 0x7fb750101850 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52250/0 (socket says 172.21.15.32:52250) 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 -- 172.21.15.32:0/3843347612 learned_addr learned my addr 172.21.15.32:0/3843347612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb754880700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750101850 0x7fb750196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 -- 172.21.15.32:0/3843347612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750101850 msgr2=0x7fb750196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750101850 0x7fb750196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 -- 172.21.15.32:0/3843347612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400095f0 con 0x7fb750100ea0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb755081700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00b4b0 tx=0x7fb74c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c011800 con 0x7fb750100ea0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb74c011e10 con 0x7fb750100ea0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75019b630 con 0x7fb750100ea0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c00f2e0 con 0x7fb750100ea0 2024-06-03T01:17:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.093+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75019bb50 con 0x7fb750100ea0 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.094+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7501091d0 con 0x7fb750100ea0 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.095+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb74c010380 con 0x7fb750100ea0 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.095+0000 7fb7467fc700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 0x7fb73c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.095+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb74c08aff0 con 0x7fb750100ea0 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.095+0000 7fb754880700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 0x7fb73c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.096+0000 7fb754880700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 0x7fb73c06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb740006010 tx=0x7fb74001a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:37.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.100+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb74c0596d0 con 0x7fb750100ea0 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 -- 172.21.15.32:0/2829093880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb90100750 msgr2=0x7fcb901047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 --2- 172.21.15.32:0/2829093880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb90100750 0x7fcb901047a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009910 tx=0x7fcb84009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 -- 172.21.15.32:0/2829093880 shutdown_connections 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 --2- 172.21.15.32:0/2829093880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb90100750 0x7fcb901047a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 --2- 172.21.15.32:0/2829093880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb900ffda0 0x7fcb90100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.290+0000 7fcb96896700 1 -- 172.21.15.32:0/2829093880 >> 172.21.15.32:0/2829093880 conn(0x7fcb900fb650 msgr2=0x7fcb900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.291+0000 7fcb96896700 1 -- 172.21.15.32:0/2829093880 shutdown_connections 2024-06-03T01:17:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.291+0000 7fcb96896700 1 -- 172.21.15.32:0/2829093880 wait complete. 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.291+0000 7fcb96896700 1 Processor -- start 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.291+0000 7fcb96896700 1 -- start start 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.291+0000 7fcb96896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb96896700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb90100750 0x7fcb90101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb96896700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90102350 con 0x7fcb900ffda0 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb96896700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901024c0 con 0x7fcb90100750 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52272/0 (socket says 172.21.15.32:52272) 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 -- 172.21.15.32:0/2970650293 learned_addr learned my addr 172.21.15.32:0/2970650293 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8f7fe700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb90100750 0x7fcb90101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 -- 172.21.15.32:0/2970650293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb90100750 msgr2=0x7fcb90101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb90100750 0x7fcb90101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 -- 172.21.15.32:0/2970650293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840095f0 con 0x7fcb900ffda0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8ffff700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000b4b0 tx=0x7fcb8000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb80011800 con 0x7fcb900ffda0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb901a2b90 con 0x7fcb900ffda0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb80011e10 con 0x7fcb900ffda0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb8000f2e0 con 0x7fcb900ffda0 2024-06-03T01:17:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.292+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb901a2fa0 con 0x7fcb900ffda0 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.293+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9004f2e0 con 0x7fcb900ffda0 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.294+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fcb80010380 con 0x7fcb900ffda0 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.294+0000 7fcb8d7fa700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 0x7fcb7806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.295+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcb8008afd0 con 0x7fcb900ffda0 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.295+0000 7fcb8f7fe700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 0x7fcb7806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.295+0000 7fcb8f7fe700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 0x7fcb7806e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006010 tx=0x7fcb8401a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.299+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcb80055c20 con 0x7fcb900ffda0 2024-06-03T01:17:37.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.302+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb750197cd0 con 0x7fb750100ea0 2024-06-03T01:17:37.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.302+0000 7fb7467fc700 1 -- 172.21.15.32:0/3843347612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb74c059260 con 0x7fb750100ea0 2024-06-03T01:17:37.302 INFO:teuthology.orchestra.run.smithi032.stdout:60129542175 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.304+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 msgr2=0x7fb73c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.304+0000 7fb7572e5700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 0x7fb73c06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb740006010 tx=0x7fb74001a040 comp rx=0 tx=0).stop 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.304+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 msgr2=0x7fb750196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.304+0000 7fb7572e5700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00b4b0 tx=0x7fb74c00b7c0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 shutdown_connections 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb750100ea0 0x7fb750196970 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb73c06c4e0 0x7fb73c06e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 --2- 172.21.15.32:0/3843347612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb750101850 0x7fb750196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 >> 172.21.15.32:0/3843347612 conn(0x7fb7500fc750 msgr2=0x7fb7500fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:37.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 shutdown_connections 2024-06-03T01:17:37.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.305+0000 7fb7572e5700 1 -- 172.21.15.32:0/3843347612 wait complete. 2024-06-03T01:17:37.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.503+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb901a32a0 con 0x7fcb900ffda0 2024-06-03T01:17:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.503+0000 7fcb8d7fa700 1 -- 172.21.15.32:0/2970650293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb80059240 con 0x7fcb900ffda0 2024-06-03T01:17:37.503 INFO:teuthology.orchestra.run.smithi032.stdout:111669149717 2024-06-03T01:17:37.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 msgr2=0x7fcb7806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 0x7fcb7806e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006010 tx=0x7fcb8401a040 comp rx=0 tx=0).stop 2024-06-03T01:17:37.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 msgr2=0x7fcb90101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000b4b0 tx=0x7fcb8000b7c0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 shutdown_connections 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb900ffda0 0x7fcb90101840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fcb7806c4e0 0x7fcb7806e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 --2- 172.21.15.32:0/2970650293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb90100750 0x7fcb90101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 >> 172.21.15.32:0/2970650293 conn(0x7fcb900fb650 msgr2=0x7fcb90103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.505+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 shutdown_connections 2024-06-03T01:17:37.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:37.506+0000 7fcb96896700 1 -- 172.21.15.32:0/2970650293 wait complete. 2024-06-03T01:17:37.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:37 smithi044 ceph-mon[36956]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:37.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:37 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3843347612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T01:17:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:37 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2970650293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T01:17:38.005 INFO:teuthology.orchestra.run.smithi032.stdout:38654705700 2024-06-03T01:17:38.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-03T01:17:38.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:37 smithi032 ceph-mon[29360]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:38.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:37 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3843347612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T01:17:38.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:37 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2970650293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 -- 172.21.15.32:0/2363514238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068ea0 msgr2=0x7fdabc10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2363514238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068ea0 0x7fdabc10d290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009910 tx=0x7fdaa4009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 -- 172.21.15.32:0/2363514238 shutdown_connections 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2363514238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068ea0 0x7fdabc10d290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2363514238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068580 0x7fdabc068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 -- 172.21.15.32:0/2363514238 >> 172.21.15.32:0/2363514238 conn(0x7fdabc075670 msgr2=0x7fdabc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 -- 172.21.15.32:0/2363514238 shutdown_connections 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.617+0000 7fdac26e0700 1 -- 172.21.15.32:0/2363514238 wait complete. 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 Processor -- start 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 -- start start 2024-06-03T01:17:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068ea0 0x7fdabc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc19f540 con 0x7fdabc068580 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdac26e0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1989f0 con 0x7fdabc068ea0 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52290/0 (socket says 172.21.15.32:52290) 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 -- 172.21.15.32:0/2109813806 learned_addr learned my addr 172.21.15.32:0/2109813806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdab3fff700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068ea0 0x7fdabc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 -- 172.21.15.32:0/2109813806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068ea0 msgr2=0x7fdabc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068ea0 0x7fdabc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.618+0000 7fdabbfff700 1 -- 172.21.15.32:0/2109813806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa40095f0 con 0x7fdabc068580 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdabbfff700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b4b0 tx=0x7fdaac00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac011800 con 0x7fdabc068580 2024-06-03T01:17:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc198cd0 con 0x7fdabc068580 2024-06-03T01:17:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdaac011e10 con 0x7fdabc068580 2024-06-03T01:17:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac00f2e0 con 0x7fdabc068580 2024-06-03T01:17:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.619+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc199220 con 0x7fdabc068580 2024-06-03T01:17:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.620+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc10aa00 con 0x7fdabc068580 2024-06-03T01:17:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.620+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdaac010380 con 0x7fdabc068580 2024-06-03T01:17:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.621+0000 7fdab9ffb700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 0x7fdaa806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.621+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdaac08b090 con 0x7fdabc068580 2024-06-03T01:17:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.621+0000 7fdab3fff700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 0x7fdaa806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.621+0000 7fdab3fff700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 0x7fdaa806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdabc101400 tx=0x7fdaa401a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:38.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.625+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdaac059640 con 0x7fdabc068580 2024-06-03T01:17:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.832+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdabc04f2e0 con 0x7fdabc068580 2024-06-03T01:17:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.832+0000 7fdab9ffb700 1 -- 172.21.15.32:0/2109813806 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaac0591d0 con 0x7fdabc068580 2024-06-03T01:17:38.833 INFO:teuthology.orchestra.run.smithi032.stdout:146028888080 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.834+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 msgr2=0x7fdaa806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.834+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 0x7fdaa806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdabc101400 tx=0x7fdaa401a040 comp rx=0 tx=0).stop 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.834+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 msgr2=0x7fdabc19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.834+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b4b0 tx=0x7fdaac00b7c0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 shutdown_connections 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc068580 0x7fdabc19e970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fdaa806c4e0 0x7fdaa806e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 --2- 172.21.15.32:0/2109813806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdabc068ea0 0x7fdabc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 >> 172.21.15.32:0/2109813806 conn(0x7fdabc075670 msgr2=0x7fdabc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 shutdown_connections 2024-06-03T01:17:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:38.835+0000 7fdac26e0700 1 -- 172.21.15.32:0/2109813806 wait complete. 2024-06-03T01:17:39.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:38 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2109813806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T01:17:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:38 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2109813806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T01:17:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 -- 172.21.15.32:0/1368624143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 msgr2=0x7f0fb01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/1368624143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb01058a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009910 tx=0x7f0fa4009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 -- 172.21.15.32:0/1368624143 shutdown_connections 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/1368624143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb01058a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/1368624143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 -- 172.21.15.32:0/1368624143 >> 172.21.15.32:0/1368624143 conn(0x7f0fb00fc750 msgr2=0x7f0fb00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 -- 172.21.15.32:0/1368624143 shutdown_connections 2024-06-03T01:17:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.227+0000 7f0fb52fa700 1 -- 172.21.15.32:0/1368624143 wait complete. 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 Processor -- start 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 -- start start 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb019d2d0 con 0x7f0fb0101850 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fb52fa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0196780 con 0x7f0fb0100ea0 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fae7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.228+0000 7f0fae7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52308/0 (socket says 172.21.15.32:52308) 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0fae7fc700 1 -- 172.21.15.32:0/2627999627 learned_addr learned my addr 172.21.15.32:0/2627999627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0faeffd700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0fae7fc700 1 -- 172.21.15.32:0/2627999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 msgr2=0x7f0fb019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0fae7fc700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0fae7fc700 1 -- 172.21.15.32:0/2627999627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40095f0 con 0x7f0fb0101850 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0faeffd700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.229+0000 7f0fae7fc700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400bda0 tx=0x7f0fa400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.230+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa401d070 con 0x7f0fb0101850 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.230+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fa400f460 con 0x7f0fb0101850 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.230+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa4017630 con 0x7f0fb0101850 2024-06-03T01:17:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.230+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb0196a00 con 0x7f0fb0101850 2024-06-03T01:17:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.230+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb0196ef0 con 0x7f0fb0101850 2024-06-03T01:17:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.231+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb01092a0 con 0x7f0fb0101850 2024-06-03T01:17:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.231+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0fa4003a60 con 0x7f0fb0101850 2024-06-03T01:17:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.232+0000 7f0f97fff700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.232+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0fa408c850 con 0x7f0fb0101850 2024-06-03T01:17:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.232+0000 7f0faeffd700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.233+0000 7f0faeffd700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000bc10 tx=0x7f0fa000b5f0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:39.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.238+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0fa405ae80 con 0x7f0fb0101850 2024-06-03T01:17:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.438+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0fb004f2e0 con 0x7f0fb0101850 2024-06-03T01:17:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.438+0000 7f0f97fff700 1 -- 172.21.15.32:0/2627999627 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fa405aa10 con 0x7f0fb0101850 2024-06-03T01:17:39.438 INFO:teuthology.orchestra.run.smithi032.stdout:81604378650 2024-06-03T01:17:39.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 msgr2=0x7f0f9806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:39.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000bc10 tx=0x7f0fa000b5f0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 msgr2=0x7f0fb019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400bda0 tx=0x7f0fa400be80 comp rx=0 tx=0).stop 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 shutdown_connections 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0100ea0 0x7f0fb019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 --2- 172.21.15.32:0/2627999627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0101850 0x7f0fb019cc40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.440+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 >> 172.21.15.32:0/2627999627 conn(0x7f0fb00fc750 msgr2=0x7f0fb00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.441+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 shutdown_connections 2024-06-03T01:17:39.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:39.441+0000 7f0fb52fa700 1 -- 172.21.15.32:0/2627999627 wait complete. 2024-06-03T01:17:39.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888080 for osd.4 2024-06-03T01:17:39.708 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:39 smithi032 ceph-mon[29360]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:39 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2627999627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T01:17:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:39 smithi044 ceph-mon[36956]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:39 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2627999627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T01:17:40.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378647 got 81604378650 for osd.2 2024-06-03T01:17:40.718 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:41.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149717 for osd.3 2024-06-03T01:17:41.105 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 -- 172.21.15.32:0/3162254574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 msgr2=0x7f04ac1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 --2- 172.21.15.32:0/3162254574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac1047a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0494009910 tx=0x7f0494009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 -- 172.21.15.32:0/3162254574 shutdown_connections 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 --2- 172.21.15.32:0/3162254574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac1047a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 --2- 172.21.15.32:0/3162254574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 0x7f04ac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 -- 172.21.15.32:0/3162254574 >> 172.21.15.32:0/3162254574 conn(0x7f04ac0fb650 msgr2=0x7f04ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 -- 172.21.15.32:0/3162254574 shutdown_connections 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 -- 172.21.15.32:0/3162254574 wait complete. 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.543+0000 7f04b0d83700 1 Processor -- start 2024-06-03T01:17:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04b0d83700 1 -- start start 2024-06-03T01:17:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04b0d83700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 0x7f04ac101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04b0d83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04b0d83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac1022c0 con 0x7f04ac100750 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04b0d83700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac102400 con 0x7f04ac0ffda0 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52326/0 (socket says 172.21.15.32:52326) 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 -- 172.21.15.32:0/2111344893 learned_addr learned my addr 172.21.15.32:0/2111344893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04aa59c700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 0x7f04ac101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 -- 172.21.15.32:0/2111344893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 msgr2=0x7f04ac101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 0x7f04ac101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 -- 172.21.15.32:0/2111344893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940095f0 con 0x7f04ac100750 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.544+0000 7f04a9d9b700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04940092e0 tx=0x7f049400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.545+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049401d070 con 0x7f04ac100750 2024-06-03T01:17:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.545+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac102680 con 0x7f04ac100750 2024-06-03T01:17:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.545+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0494022470 con 0x7f04ac100750 2024-06-03T01:17:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.545+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049400f690 con 0x7f04ac100750 2024-06-03T01:17:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.545+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac1a2e50 con 0x7f04ac100750 2024-06-03T01:17:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.546+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac04f2e0 con 0x7f04ac100750 2024-06-03T01:17:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.546+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f04940225e0 con 0x7f04ac100750 2024-06-03T01:17:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.547+0000 7f04a37fe700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 0x7f049806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.547+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f049408cb00 con 0x7f04ac100750 2024-06-03T01:17:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.547+0000 7f04aa59c700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 0x7f049806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.547+0000 7f04aa59c700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 0x7f049806e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f049c006fd0 tx=0x7f049c008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.552+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f04940576d0 con 0x7f04ac100750 2024-06-03T01:17:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.752+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f04ac1a30a0 con 0x7f04ac100750 2024-06-03T01:17:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.752+0000 7f04a37fe700 1 -- 172.21.15.32:0/2111344893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f049405acf0 con 0x7f04ac100750 2024-06-03T01:17:41.753 INFO:teuthology.orchestra.run.smithi032.stdout:167503724556 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.754+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 msgr2=0x7f049806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.754+0000 7f04b0d83700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 0x7f049806e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f049c006fd0 tx=0x7f049c008040 comp rx=0 tx=0).stop 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 msgr2=0x7f04ac101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04940092e0 tx=0x7f049400be20 comp rx=0 tx=0).stop 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 shutdown_connections 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f049806c4e0 0x7f049806e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0ffda0 0x7f04ac101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 --2- 172.21.15.32:0/2111344893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04ac100750 0x7f04ac101d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 >> 172.21.15.32:0/2111344893 conn(0x7f04ac0fb650 msgr2=0x7f04ac103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 shutdown_connections 2024-06-03T01:17:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:41.755+0000 7f04b0d83700 1 -- 172.21.15.32:0/2111344893 wait complete. 2024-06-03T01:17:42.044 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:41 smithi032 ceph-mon[29360]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:42.044 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:41 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2111344893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T01:17:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:42.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542175 for osd.1 2024-06-03T01:17:42.070 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:41 smithi044 ceph-mon[36956]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:42.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:41 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2111344893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T01:17:43.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724556 for osd.5 2024-06-03T01:17:43.274 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:44.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:43 smithi032 ceph-mon[29360]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:44.043 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 -- 172.21.15.32:0/61390114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 msgr2=0x7f2e780ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/61390114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e780ff980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009910 tx=0x7f2e74009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 -- 172.21.15.32:0/61390114 shutdown_connections 2024-06-03T01:17:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/61390114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 0x7f2e78105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/61390114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e780ff980 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 -- 172.21.15.32:0/61390114 >> 172.21.15.32:0/61390114 conn(0x7f2e78074e30 msgr2=0x7f2e78075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 -- 172.21.15.32:0/61390114 shutdown_connections 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.149+0000 7f2e7f45a700 1 -- 172.21.15.32:0/61390114 wait complete. 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 Processor -- start 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 -- start start 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 0x7f2e7819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7819d250 con 0x7f2e780ff5a0 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7f45a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78196700 con 0x7f2e780fff50 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7d1f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7d1f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52342/0 (socket says 172.21.15.32:52342) 2024-06-03T01:17:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.150+0000 7f2e7d1f6700 1 -- 172.21.15.32:0/2914885164 learned_addr learned my addr 172.21.15.32:0/2914885164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7c9f5700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 0x7f2e7819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7d1f6700 1 -- 172.21.15.32:0/2914885164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 msgr2=0x7f2e7819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7d1f6700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 0x7f2e7819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7d1f6700 1 -- 172.21.15.32:0/2914885164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740095f0 con 0x7f2e780ff5a0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7d1f6700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006010 tx=0x7f2e7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7401d070 con 0x7f2e780ff5a0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e7400f460 con 0x7f2e780ff5a0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78196980 con 0x7f2e780ff5a0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e74017630 con 0x7f2e780ff5a0 2024-06-03T01:17:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.151+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e78196e70 con 0x7f2e780ff5a0 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.152+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e78109270 con 0x7f2e780ff5a0 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.153+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2e7400f5d0 con 0x7f2e780ff5a0 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.153+0000 7f2e6e7fc700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 0x7f2e6406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.153+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2e7408c950 con 0x7f2e780ff5a0 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.153+0000 7f2e7c9f5700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 0x7f2e6406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.154+0000 7f2e7c9f5700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 0x7f2e6406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e78197b70 tx=0x7f2e6800b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.158+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2e7405af00 con 0x7f2e780ff5a0 2024-06-03T01:17:44.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:43 smithi044 ceph-mon[36956]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:44.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.355+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2e78197c40 con 0x7f2e780ff5a0 2024-06-03T01:17:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.355+0000 7f2e6e7fc700 1 -- 172.21.15.32:0/2914885164 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e7405aa90 con 0x7f2e780ff5a0 2024-06-03T01:17:44.358 INFO:teuthology.orchestra.run.smithi032.stdout:38654705702 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 msgr2=0x7f2e6406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 0x7f2e6406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e78197b70 tx=0x7f2e6800b410 comp rx=0 tx=0).stop 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 msgr2=0x7f2e7819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006010 tx=0x7f2e7400bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 shutdown_connections 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780ff5a0 0x7f2e7819c680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2e6406c4e0 0x7f2e6406e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 --2- 172.21.15.32:0/2914885164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e780fff50 0x7f2e7819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 >> 172.21.15.32:0/2914885164 conn(0x7f2e78074e30 msgr2=0x7f2e780fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 shutdown_connections 2024-06-03T01:17:44.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:44.358+0000 7f2e7f45a700 1 -- 172.21.15.32:0/2914885164 wait complete. 2024-06-03T01:17:45.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-06-03T01:17:45.014 DEBUG:teuthology.parallel:result is None 2024-06-03T01:17:45.014 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T01:17:45.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:17:45.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:44 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2914885164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T01:17:45.229 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:45.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:44 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2914885164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T01:17:46.220 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:45 smithi032 ceph-mon[29360]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:45 smithi044 ceph-mon[36956]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 -- 172.21.15.32:0/389562122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04101850 msgr2=0x7eff041058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 --2- 172.21.15.32:0/389562122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04101850 0x7eff041058a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff00009910 tx=0x7eff00009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 -- 172.21.15.32:0/389562122 shutdown_connections 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 --2- 172.21.15.32:0/389562122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04101850 0x7eff041058a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 --2- 172.21.15.32:0/389562122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff04100ea0 0x7eff04101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 -- 172.21.15.32:0/389562122 >> 172.21.15.32:0/389562122 conn(0x7eff040fc750 msgr2=0x7eff040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 -- 172.21.15.32:0/389562122 shutdown_connections 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.745+0000 7eff0c533700 1 -- 172.21.15.32:0/389562122 wait complete. 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 Processor -- start 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 -- start start 2024-06-03T01:17:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff04101850 0x7eff0419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419d240 con 0x7eff04100ea0 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0c533700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04196780 con 0x7eff04101850 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52364/0 (socket says 172.21.15.32:52364) 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 -- 172.21.15.32:0/12015591 learned_addr learned my addr 172.21.15.32:0/12015591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 -- 172.21.15.32:0/12015591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff04101850 msgr2=0x7eff0419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff04101850 0x7eff0419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.746+0000 7eff0a2cf700 1 -- 172.21.15.32:0/12015591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000095f0 con 0x7eff04100ea0 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7eff0a2cf700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efef800c930 tx=0x7efef800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef8007ad0 con 0x7eff04100ea0 2024-06-03T01:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efef8007c30 con 0x7eff04100ea0 2024-06-03T01:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04196a60 con 0x7eff04100ea0 2024-06-03T01:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef8018630 con 0x7eff04100ea0 2024-06-03T01:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.747+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff04196fb0 con 0x7eff04100ea0 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.748+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff041092a0 con 0x7eff04100ea0 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.748+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7efef800f450 con 0x7eff04100ea0 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.749+0000 7efef77fe700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 0x7efef006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.749+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efef808b5e0 con 0x7eff04100ea0 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.749+0000 7eff09ace700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 0x7efef006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.749+0000 7eff09ace700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 0x7efef006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff0000b5c0 tx=0x7eff00005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.753+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efef8059c10 con 0x7eff04100ea0 2024-06-03T01:17:46.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.948+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff04197230 con 0x7efef006c4e0 2024-06-03T01:17:46.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.949+0000 7efef77fe700 1 -- 172.21.15.32:0/12015591 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28125 (secure 0 0 0) 0x7eff04197230 con 0x7efef006c4e0 2024-06-03T01:17:46.950 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:46.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 msgr2=0x7efef006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:46.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 0x7efef006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff0000b5c0 tx=0x7eff00005960 comp rx=0 tx=0).stop 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 msgr2=0x7eff0419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efef800c930 tx=0x7efef800ccf0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 shutdown_connections 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff04100ea0 0x7eff0419c700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7efef006c4e0 0x7efef006e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 --2- 172.21.15.32:0/12015591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff04101850 0x7eff0419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 >> 172.21.15.32:0/12015591 conn(0x7eff040fc750 msgr2=0x7eff040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 shutdown_connections 2024-06-03T01:17:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:46.951+0000 7eff0c533700 1 -- 172.21.15.32:0/12015591 wait complete. 2024-06-03T01:17:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-06-03T01:17:47.640 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-06-03T01:17:46.893340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002143"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-06-03T01:16:56.130120+0000","last_change":"2024-06-03T01:16:31.309020+0000","last_active":"2024-06-03T01:16:56.130120+0000","last_peered":"2024-06-03T01:16:56.130120+0000","last_clean":"2024-06-03T01:16:56.130120+0000","last_became_active":"2024-06-03T01:16:31.308670+0000","last_became_peered":"2024-06-03T01:16:31.308670+0000","last_unstale":"2024-06-03T01:16:56.130120+0000","last_undegraded":"2024-06-03T01:16:56.130120+0000","last_fullsized":"2024-06-03T01:16:56.130120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:15:43.862068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:15:43.862068+0000","last_clean_scrub_stamp":"2024-06-03T01:15:43.862068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:48.601043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.71999999999999997}]},{"osd":1,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.69699999999999995}]},{"osd":2,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.748},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.68400000000000005}]},{"osd":3,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.436},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.33300000000000002}]},{"osd":1,"last update":"Mon Jun 3 01:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36499999999999999}]},{"osd":3,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.51700000000000002}]},{"osd":4,"last update":"Mon Jun 3 01:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jun 3 01:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34399999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34300000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Jun 3 01:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jun 3 01:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.374}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23599999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.252,"5min":0.22,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Jun 3 01:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Jun 3 01:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.62},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Jun 3 01:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:17:47.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:17:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:47 smithi044 ceph-mon[36956]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:47 smithi044 ceph-mon[36956]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:17:48.248 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:47 smithi032 ceph-mon[29360]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:48.248 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:47 smithi032 ceph-mon[29360]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 -- 172.21.15.32:0/2380383520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 msgr2=0x7fbde8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 --2- 172.21.15.32:0/2380383520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8100080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009910 tx=0x7fbddc009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 -- 172.21.15.32:0/2380383520 shutdown_connections 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 --2- 172.21.15.32:0/2380383520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8100080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 --2- 172.21.15.32:0/2380383520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 0x7fbde8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 -- 172.21.15.32:0/2380383520 >> 172.21.15.32:0/2380383520 conn(0x7fbde8075670 msgr2=0x7fbde8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 -- 172.21.15.32:0/2380383520 shutdown_connections 2024-06-03T01:17:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.446+0000 7fbded798700 1 -- 172.21.15.32:0/2380383520 wait complete. 2024-06-03T01:17:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 Processor -- start 2024-06-03T01:17:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 -- start start 2024-06-03T01:17:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 0x7fbde8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81022c0 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbded798700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8102400 con 0x7fbde8105bd0 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42854/0 (socket says 172.21.15.32:42854) 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 -- 172.21.15.32:0/2669896612 learned_addr learned my addr 172.21.15.32:0/2669896612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde67fc700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 0x7fbde8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 -- 172.21.15.32:0/2669896612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 msgr2=0x7fbde8101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 0x7fbde8101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.447+0000 7fbde6ffd700 1 -- 172.21.15.32:0/2669896612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbde6ffd700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c930 tx=0x7fbdd800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8007ad0 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbdd8007c30 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde81a2a60 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8018630 con 0x7fbde80ffc20 2024-06-03T01:17:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.448+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde81a2dc0 con 0x7fbde80ffc20 2024-06-03T01:17:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.449+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde804f2e0 con 0x7fbde80ffc20 2024-06-03T01:17:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.449+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbdd800f450 con 0x7fbde80ffc20 2024-06-03T01:17:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.450+0000 7fbdcffff700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 0x7fbdd006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.450+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbdd808b6c0 con 0x7fbde80ffc20 2024-06-03T01:17:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.450+0000 7fbde67fc700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 0x7fbdd006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.450+0000 7fbde67fc700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 0x7fbdd006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b5c0 tx=0x7fbddc005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.454+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbdd80561e0 con 0x7fbde80ffc20 2024-06-03T01:17:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.646+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbde8068db0 con 0x7fbdd006c4e0 2024-06-03T01:17:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.648+0000 7fbdcffff700 1 -- 172.21.15.32:0/2669896612 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28253 (secure 0 0 0) 0x7fbde8068db0 con 0x7fbdd006c4e0 2024-06-03T01:17:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 msgr2=0x7fbdd006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 0x7fbdd006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b5c0 tx=0x7fbddc005960 comp rx=0 tx=0).stop 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 msgr2=0x7fbde8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c930 tx=0x7fbdd800ccf0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 shutdown_connections 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbde80ffc20 0x7fbde8101840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbdd006c4e0 0x7fbdd006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 --2- 172.21.15.32:0/2669896612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbde8105bd0 0x7fbde8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 >> 172.21.15.32:0/2669896612 conn(0x7fbde8075670 msgr2=0x7fbde80686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 shutdown_connections 2024-06-03T01:17:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:49.650+0000 7fbded798700 1 -- 172.21.15.32:0/2669896612 wait complete. 2024-06-03T01:17:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-06-03T01:17:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:49 smithi044 ceph-mon[36956]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:50.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:49 smithi032 ceph-mon[29360]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-06-03T01:17:48.893602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002109"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-06-03T01:16:56.130120+0000","last_change":"2024-06-03T01:16:31.309020+0000","last_active":"2024-06-03T01:16:56.130120+0000","last_peered":"2024-06-03T01:16:56.130120+0000","last_clean":"2024-06-03T01:16:56.130120+0000","last_became_active":"2024-06-03T01:16:31.308670+0000","last_became_peered":"2024-06-03T01:16:31.308670+0000","last_unstale":"2024-06-03T01:16:56.130120+0000","last_undegraded":"2024-06-03T01:16:56.130120+0000","last_fullsized":"2024-06-03T01:16:56.130120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:15:43.862068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:15:43.862068+0000","last_clean_scrub_stamp":"2024-06-03T01:15:43.862068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:48.601043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.437},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.45100000000000001}]},{"osd":1,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.376},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.34699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Mon Jun 3 01:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46500000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29599999999999999}]},{"osd":1,"last update":"Mon Jun 3 01:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.371,"15min":0.374},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33500000000000002}]},{"osd":3,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Jun 3 01:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.621},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jun 3 01:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34399999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34300000000000003},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Jun 3 01:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Jun 3 01:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jun 3 01:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.374}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jun 3 01:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23599999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.252,"5min":0.22,"15min":0.22},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Jun 3 01:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Jun 3 01:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.62},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Jun 3 01:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:17:50.252 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T01:17:50.252 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-03T01:17:50.252 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-03T01:17:50.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-03T01:17:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:50 smithi044 ceph-mon[36956]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:17:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:50 smithi032 ceph-mon[29360]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:17:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- 172.21.15.32:0/1516120706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e4105bd0 msgr2=0x7f86e4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/1516120706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e4105bd0 0x7f86e4105fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009910 tx=0x7f86e0009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- 172.21.15.32:0/1516120706 shutdown_connections 2024-06-03T01:17:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/1516120706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e40ffc20 0x7f86e4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/1516120706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e4105bd0 0x7f86e4105fb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- 172.21.15.32:0/1516120706 >> 172.21.15.32:0/1516120706 conn(0x7f86e4075670 msgr2=0x7f86e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- 172.21.15.32:0/1516120706 shutdown_connections 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- 172.21.15.32:0/1516120706 wait complete. 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 Processor -- start 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.007+0000 7f86ebf6b700 1 -- start start 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86ebf6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86ebf6b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 0x7f86e4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86ebf6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e41982d0 con 0x7f86e40ffc20 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86ebf6b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e4198440 con 0x7f86e4105bd0 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42880/0 (socket says 172.21.15.32:42880) 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 -- 172.21.15.32:0/546435786 learned_addr learned my addr 172.21.15.32:0/546435786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9506700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 0x7f86e4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 -- 172.21.15.32:0/546435786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 msgr2=0x7f86e4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 0x7f86e4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 -- 172.21.15.32:0/546435786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e00095f0 con 0x7f86e40ffc20 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9506700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 0x7f86e4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86e9d07700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86e0006010 tx=0x7f86e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e001d070 con 0x7f86e40ffc20 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f86e000f460 con 0x7f86e40ffc20 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e41a0720 con 0x7f86e40ffc20 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.008+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e0021640 con 0x7f86e40ffc20 2024-06-03T01:17:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.009+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e41a0bc0 con 0x7f86e40ffc20 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.009+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e41080f0 con 0x7f86e40ffc20 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.010+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f86e000faf0 con 0x7f86e40ffc20 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.010+0000 7f86daffd700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 0x7f86d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.010+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f86e008c710 con 0x7f86e40ffc20 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.011+0000 7f86e9506700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 0x7f86d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.011+0000 7f86e9506700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 0x7f86d006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86e40691d0 tx=0x7f86d4005f50 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.015+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f86e00572b0 con 0x7f86e40ffc20 2024-06-03T01:17:52.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:51 smithi032 ceph-mon[29360]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:51 smithi044 ceph-mon[36956]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.256+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f86e4068bc0 con 0x7f86e40ffc20 2024-06-03T01:17:52.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.257+0000 7f86daffd700 1 -- 172.21.15.32:0/546435786 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f86e005a8d0 con 0x7f86e40ffc20 2024-06-03T01:17:52.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:17:52.257 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.258+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 msgr2=0x7f86d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 0x7f86d006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86e40691d0 tx=0x7f86d4005f50 comp rx=0 tx=0).stop 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 msgr2=0x7f86e41976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86e0006010 tx=0x7f86e000be20 comp rx=0 tx=0).stop 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 shutdown_connections 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e40ffc20 0x7f86e41976b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f86d006c4e0 0x7f86d006e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 --2- 172.21.15.32:0/546435786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86e4105bd0 0x7f86e4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 >> 172.21.15.32:0/546435786 conn(0x7f86e4075670 msgr2=0x7f86e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 shutdown_connections 2024-06-03T01:17:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:52.259+0000 7f86ebf6b700 1 -- 172.21.15.32:0/546435786 wait complete. 2024-06-03T01:17:52.820 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-03T01:17:52.821 INFO:tasks.cephadm:Setup complete, yielding 2024-06-03T01:17:52.821 INFO:teuthology.run_tasks:Running task print... 2024-06-03T01:17:52.834 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-06-03T01:17:52.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:17:52.850 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:17:52.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-06-03T01:17:53.061 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:53.184 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:52 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/546435786' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T01:17:53.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:52 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/546435786' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T01:17:54.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:53 smithi032 ceph-mon[29360]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:53 smithi044 ceph-mon[36956]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:54.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 -- 172.21.15.32:0/674049991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54104080 msgr2=0x7f4e54106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/674049991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54104080 0x7f4e54106470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009910 tx=0x7f4e3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 -- 172.21.15.32:0/674049991 shutdown_connections 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/674049991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54104080 0x7f4e54106470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/674049991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54101750 0x7f4e54103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 -- 172.21.15.32:0/674049991 >> 172.21.15.32:0/674049991 conn(0x7f4e540fb160 msgr2=0x7f4e540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 -- 172.21.15.32:0/674049991 shutdown_connections 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 -- 172.21.15.32:0/674049991 wait complete. 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.644+0000 7f4e59a1c700 1 Processor -- start 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e59a1c700 1 -- start start 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e59a1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e59a1c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54104080 0x7f4e540ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e59a1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540ffc40 con 0x7f4e54101750 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e59a1c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540ffd80 con 0x7f4e54104080 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42900/0 (socket says 172.21.15.32:42900) 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 -- 172.21.15.32:0/266344243 learned_addr learned my addr 172.21.15.32:0/266344243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e4bfff700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54104080 0x7f4e540ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 -- 172.21.15.32:0/266344243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54104080 msgr2=0x7f4e540ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54104080 0x7f4e540ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 -- 172.21.15.32:0/266344243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c0095f0 con 0x7f4e54101750 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.645+0000 7f4e52ffd700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400d8d0 tx=0x7f4e4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.646+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e440098a0 con 0x7f4e54101750 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.646+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e54100060 con 0x7f4e54101750 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.646+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e44010460 con 0x7f4e54101750 2024-06-03T01:17:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.646+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4400f5f0 con 0x7f4e54101750 2024-06-03T01:17:54.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.646+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5419e800 con 0x7f4e54101750 2024-06-03T01:17:54.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.647+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e54190630 con 0x7f4e54101750 2024-06-03T01:17:54.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.647+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4e44009a00 con 0x7f4e54101750 2024-06-03T01:17:54.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.648+0000 7f4e58a1a700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 0x7f4e4006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.648+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4e4408b470 con 0x7f4e54101750 2024-06-03T01:17:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.648+0000 7f4e4bfff700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 0x7f4e4006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.648+0000 7f4e4bfff700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 0x7f4e4006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006010 tx=0x7f4e3c00b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:54.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.652+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4e44059980 con 0x7f4e54101750 2024-06-03T01:17:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.855+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f4e54068af0 con 0x7f4e54101750 2024-06-03T01:17:54.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.859+0000 7f4e58a1a700 1 -- 172.21.15.32:0/266344243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4e44059510 con 0x7f4e54101750 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 msgr2=0x7f4e4006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 0x7f4e4006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006010 tx=0x7f4e3c00b540 comp rx=0 tx=0).stop 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 msgr2=0x7f4e541010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400d8d0 tx=0x7f4e4400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 shutdown_connections 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54101750 0x7f4e541010b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4e4006c4e0 0x7f4e4006e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 --2- 172.21.15.32:0/266344243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e54104080 0x7f4e540ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:54.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 >> 172.21.15.32:0/266344243 conn(0x7f4e540fb160 msgr2=0x7f4e540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:54.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 shutdown_connections 2024-06-03T01:17:54.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:54.861+0000 7f4e59a1c700 1 -- 172.21.15.32:0/266344243 wait complete. 2024-06-03T01:17:55.426 INFO:teuthology.run_tasks:Running task print... 2024-06-03T01:17:55.435 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-06-03T01:17:55.435 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:17:55.450 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:17:55.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-03T01:17:55.664 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/266344243' entity='client.admin' 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:17:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:17:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/266344243' entity='client.admin' 2024-06-03T01:17:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:17:56.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:17:56.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:17:56.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:17:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 -- 172.21.15.32:0/2078896453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 msgr2=0x7fa264101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 --2- 172.21.15.32:0/2078896453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa264101200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa254009910 tx=0x7fa254009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 -- 172.21.15.32:0/2078896453 shutdown_connections 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 --2- 172.21.15.32:0/2078896453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa264101200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 --2- 172.21.15.32:0/2078896453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 0x7fa2640fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 -- 172.21.15.32:0/2078896453 >> 172.21.15.32:0/2078896453 conn(0x7fa2640fb120 msgr2=0x7fa2640fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 -- 172.21.15.32:0/2078896453 shutdown_connections 2024-06-03T01:17:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.109+0000 7fa26c63b700 1 -- 172.21.15.32:0/2078896453 wait complete. 2024-06-03T01:17:57.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 Processor -- start 2024-06-03T01:17:57.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 -- start start 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 0x7fa264195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264195fd0 con 0x7fa264100da0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa26c63b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264196110 con 0x7fa2640ffba0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa269bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa269bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54136/0 (socket says 172.21.15.32:54136) 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.110+0000 7fa269bd6700 1 -- 172.21.15.32:0/1143461561 learned_addr learned my addr 172.21.15.32:0/1143461561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa26a3d7700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 0x7fa264195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa269bd6700 1 -- 172.21.15.32:0/1143461561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 msgr2=0x7fa264195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa269bd6700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 0x7fa264195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa269bd6700 1 -- 172.21.15.32:0/1143461561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540095f0 con 0x7fa264100da0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa269bd6700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa254006010 tx=0x7fa25400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa25401f070 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa254019610 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa264068e30 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa25400f760 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.111+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa264069320 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.113+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2641087a0 con 0x7fa264100da0 2024-06-03T01:17:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.113+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa254019c60 con 0x7fa264100da0 2024-06-03T01:17:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.114+0000 7fa25b7fe700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 0x7fa25006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.114+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa25408e9f0 con 0x7fa264100da0 2024-06-03T01:17:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.114+0000 7fa26a3d7700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 0x7fa25006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.114+0000 7fa26a3d7700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 0x7fa25006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa264100c00 tx=0x7fa260009190 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.119+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa254090e60 con 0x7fa264100da0 2024-06-03T01:17:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.336+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2641056f0 con 0x7fa25006c4e0 2024-06-03T01:17:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.337+0000 7fa25b7fe700 1 -- 172.21.15.32:0/1143461561 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa2641056f0 con 0x7fa25006c4e0 2024-06-03T01:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2024-06-03T01:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2024-06-03T01:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2024-06-03T01:17:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 msgr2=0x7fa25006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 0x7fa25006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa264100c00 tx=0x7fa260009190 comp rx=0 tx=0).stop 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 msgr2=0x7fa2641959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa254006010 tx=0x7fa25400bd70 comp rx=0 tx=0).stop 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 shutdown_connections 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa25006c4e0 0x7fa25006e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2640ffba0 0x7fa264195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 --2- 172.21.15.32:0/1143461561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264100da0 0x7fa2641959b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.339+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 >> 172.21.15.32:0/1143461561 conn(0x7fa2640fb120 msgr2=0x7fa264103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.340+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 shutdown_connections 2024-06-03T01:17:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:57.340+0000 7fa26c63b700 1 -- 172.21.15.32:0/1143461561 wait complete. 2024-06-03T01:17:57.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-03T01:17:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:57 smithi032 ceph-mon[29360]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:17:58.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:57 smithi044 ceph-mon[36956]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:17:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:58 smithi032 ceph-mon[29360]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:17:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:58 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:59.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:58 smithi044 ceph-mon[36956]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:17:59.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:58 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 -- 172.21.15.32:0/3045498291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70100cb0 msgr2=0x7f6e701010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 --2- 172.21.15.32:0/3045498291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70100cb0 0x7f6e701010d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009910 tx=0x7f6e6c009c20 comp rx=0 tx=0).stop 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 -- 172.21.15.32:0/3045498291 shutdown_connections 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 --2- 172.21.15.32:0/3045498291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70101eb0 0x7f6e70102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 --2- 172.21.15.32:0/3045498291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70100cb0 0x7f6e701010d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 -- 172.21.15.32:0/3045498291 >> 172.21.15.32:0/3045498291 conn(0x7f6e700fc230 msgr2=0x7f6e700fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 -- 172.21.15.32:0/3045498291 shutdown_connections 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.645+0000 7f6e77303700 1 -- 172.21.15.32:0/3045498291 wait complete. 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 Processor -- start 2024-06-03T01:17:59.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 -- start start 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100cb0 0x7f6e701964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70197020 con 0x7f6e70101eb0 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e77303700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70197160 con 0x7f6e70100cb0 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e7489e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e7489e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54144/0 (socket says 172.21.15.32:54144) 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.646+0000 7f6e7489e700 1 -- 172.21.15.32:0/3993084444 learned_addr learned my addr 172.21.15.32:0/3993084444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e7509f700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100cb0 0x7f6e701964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e7489e700 1 -- 172.21.15.32:0/3993084444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100cb0 msgr2=0x7f6e701964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e7489e700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100cb0 0x7f6e701964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e7489e700 1 -- 172.21.15.32:0/3993084444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0095f0 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e7489e700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000d900 tx=0x7f6e6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e600041f0 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e60004ce0 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7019bc10 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6000b750 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.647+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7019c160 con 0x7f6e70101eb0 2024-06-03T01:17:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.648+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70109810 con 0x7f6e70101eb0 2024-06-03T01:17:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.648+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6e60004350 con 0x7f6e70101eb0 2024-06-03T01:17:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.649+0000 7f6e667fc700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 0x7f6e5c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.649+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6e6008b540 con 0x7f6e70101eb0 2024-06-03T01:17:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.649+0000 7f6e7509f700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 0x7f6e5c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.650+0000 7f6e7509f700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 0x7f6e5c06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c006010 tx=0x7f6e6c005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.654+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6e60059b70 con 0x7f6e70101eb0 2024-06-03T01:17:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.860+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e70106800 con 0x7f6e5c06c4e0 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.869+0000 7f6e667fc700 1 -- 172.21.15.32:0/3993084444 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2840 (secure 0 0 0) 0x7f6e70106800 con 0x7f6e5c06c4e0 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (3m) 2m ago 5m 26.2M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (5m) 2m ago 5m 6995k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (4m) 65s ago 4m 6991k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (5m) 2m ago 5m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (4m) 65s ago 4m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (3m) 2m ago 4m 78.4M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:17:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 5be31c24972a 5db22291b59c 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (4m) 65s ago 4m 442M - 18.2.1 5be31c24972a 5fbe526898cf 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (6m) 2m ago 6m 43.0M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (4m) 65s ago 4m 40.9M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 2m ago 5m 14.8M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 65s ago 4m 17.0M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (3m) 2m ago 3m 40.0M 3948M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (2m) 2m ago 2m 42.0M 3948M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (2m) 2m ago 2m 42.7M 3948M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (2m) 65s ago 119s 42.5M 4972M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (95s) 65s ago 94s 41.0M 4972M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:17:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (70s) 65s ago 69s 14.2M 4972M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:17:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 2m ago 4m 39.8M - 2.43.0 a07b618ecd1d 55ba7950dc8e 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 msgr2=0x7f6e5c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 0x7f6e5c06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c006010 tx=0x7f6e6c005960 comp rx=0 tx=0).stop 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 msgr2=0x7f6e70196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000d900 tx=0x7f6e6000dcc0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 shutdown_connections 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f6e5c06c4e0 0x7f6e5c06e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100cb0 0x7f6e701964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 --2- 172.21.15.32:0/3993084444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e70101eb0 0x7f6e70196a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 >> 172.21.15.32:0/3993084444 conn(0x7f6e700fc230 msgr2=0x7f6e701050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 shutdown_connections 2024-06-03T01:17:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:17:59.871+0000 7f6e77303700 1 -- 172.21.15.32:0/3993084444 wait complete. 2024-06-03T01:18:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:17:59 smithi044 ceph-mon[36956]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:00.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:17:59 smithi032 ceph-mon[29360]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:00.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-03T01:18:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:01.019 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:00 smithi032 ceph-mon[29360]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:00 smithi044 ceph-mon[36956]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:01.989 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:01 smithi032 ceph-mon[29360]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.211+0000 7fa339424700 1 -- 172.21.15.32:0/3507507066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 msgr2=0x7fa334101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.211+0000 7fa339424700 1 --2- 172.21.15.32:0/3507507066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334101220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa328009910 tx=0x7fa328009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 -- 172.21.15.32:0/3507507066 shutdown_connections 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 --2- 172.21.15.32:0/3507507066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334101220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 --2- 172.21.15.32:0/3507507066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 0x7fa3340fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 -- 172.21.15.32:0/3507507066 >> 172.21.15.32:0/3507507066 conn(0x7fa3340fb160 msgr2=0x7fa3340fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 -- 172.21.15.32:0/3507507066 shutdown_connections 2024-06-03T01:18:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 -- 172.21.15.32:0/3507507066 wait complete. 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 Processor -- start 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.212+0000 7fa339424700 1 -- start start 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 0x7fa334198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341995c0 con 0x7fa334100dc0 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334199700 con 0x7fa3340ffbc0 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54154/0 (socket says 172.21.15.32:54154) 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 -- 172.21.15.32:0/2994938960 learned_addr learned my addr 172.21.15.32:0/2994938960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa332ffd700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 0x7fa334198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 -- 172.21.15.32:0/2994938960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 msgr2=0x7fa334198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 0x7fa334198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 -- 172.21.15.32:0/2994938960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3280095f0 con 0x7fa334100dc0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa3327fc700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa328006010 tx=0x7fa32800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa32801d070 con 0x7fa334100dc0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa32800f460 con 0x7fa334100dc0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33419e150 con 0x7fa334100dc0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa328021640 con 0x7fa334100dc0 2024-06-03T01:18:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.213+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33419e640 con 0x7fa334100dc0 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.214+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa334068af0 con 0x7fa334100dc0 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.215+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa32800fab0 con 0x7fa334100dc0 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.216+0000 7fa31bfff700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 0x7fa31c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.216+0000 7fa332ffd700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 0x7fa31c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.216+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa32808c7d0 con 0x7fa334100dc0 2024-06-03T01:18:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.216+0000 7fa332ffd700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 0x7fa31c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa334100c20 tx=0x7fa324009190 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.220+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa32805ae00 con 0x7fa334100dc0 2024-06-03T01:18:02.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:01 smithi044 ceph-mon[36956]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.435+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa334069560 con 0x7fa31c06c4e0 2024-06-03T01:18:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.439+0000 7fa31bfff700 1 -- 172.21.15.32:0/2994938960 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7fa334069560 con 0x7fa31c06c4e0 2024-06-03T01:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter 2/2 2m ago 6m * 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:crash 2/2 2m ago 6m * 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 2/2 2m ago 6m count:2 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:mon 2/2 2m ago 5m smithi032:172.21.15.32=smithi032;smithi044:172.21.15.44=smithi044;count:2 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:osd 6 2m ago - 2024-06-03T01:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-06-03T01:18:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 msgr2=0x7fa31c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 0x7fa31c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa334100c20 tx=0x7fa324009190 comp rx=0 tx=0).stop 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 msgr2=0x7fa334198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa328006010 tx=0x7fa32800bd70 comp rx=0 tx=0).stop 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 shutdown_connections 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa31c06c4e0 0x7fa31c06e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3340ffbc0 0x7fa334198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 --2- 172.21.15.32:0/2994938960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa334100dc0 0x7fa334198fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.441+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 >> 172.21.15.32:0/2994938960 conn(0x7fa3340fb160 msgr2=0x7fa334068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.442+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 shutdown_connections 2024-06-03T01:18:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:02.442+0000 7fa339424700 1 -- 172.21.15.32:0/2994938960 wait complete. 2024-06-03T01:18:03.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-03T01:18:03.282 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:03 smithi044 ceph-mon[36956]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:04.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:03 smithi044 ceph-mon[36956]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:03 smithi032 ceph-mon[29360]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:03 smithi032 ceph-mon[29360]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 -- 172.21.15.32:0/1985198752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 msgr2=0x7feca8106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 --2- 172.21.15.32:0/1985198752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca8106480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009910 tx=0x7fec9c009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 -- 172.21.15.32:0/1985198752 shutdown_connections 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 --2- 172.21.15.32:0/1985198752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca8106480 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 --2- 172.21.15.32:0/1985198752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 0x7feca8103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 -- 172.21.15.32:0/1985198752 >> 172.21.15.32:0/1985198752 conn(0x7feca80fb150 msgr2=0x7feca80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 -- 172.21.15.32:0/1985198752 shutdown_connections 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 -- 172.21.15.32:0/1985198752 wait complete. 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.878+0000 7fecadd75700 1 Processor -- start 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7fecadd75700 1 -- start start 2024-06-03T01:18:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7fecadd75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 0x7feca81010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7fecadd75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7fecadd75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80ffc50 con 0x7feca8104090 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7fecadd75700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80ffd90 con 0x7feca8101760 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54182/0 (socket says 172.21.15.32:54182) 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 -- 172.21.15.32:0/2774673503 learned_addr learned my addr 172.21.15.32:0/2774673503 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca77fe700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 0x7feca81010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 -- 172.21.15.32:0/2774673503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 msgr2=0x7feca81010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 0x7feca81010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 -- 172.21.15.32:0/2774673503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0095f0 con 0x7feca8104090 2024-06-03T01:18:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca6ffd700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0092e0 tx=0x7fec9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.879+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c01d070 con 0x7feca8104090 2024-06-03T01:18:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.880+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca8100010 con 0x7feca8104090 2024-06-03T01:18:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.880+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec9c00f460 con 0x7feca8104090 2024-06-03T01:18:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.880+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c017630 con 0x7feca8104090 2024-06-03T01:18:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.880+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca8100500 con 0x7feca8104090 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.881+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca8190640 con 0x7feca8104090 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.881+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fec9c00fa00 con 0x7feca8104090 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.882+0000 7feca4ff9700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 0x7fec9006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.882+0000 7feca77fe700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 0x7fec9006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.882+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fec9c08c970 con 0x7feca8104090 2024-06-03T01:18:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.882+0000 7feca77fe700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 0x7fec9006e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec98005950 tx=0x7fec9800b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:04.886+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fec9c05af20 con 0x7feca8104090 2024-06-03T01:18:05.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.094+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feca80611d0 con 0x7fec9006c4e0 2024-06-03T01:18:05.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.095+0000 7feca4ff9700 1 -- 172.21.15.32:0/2774673503 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7feca80611d0 con 0x7fec9006c4e0 2024-06-03T01:18:05.095 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2024-06-03T01:18:05.095 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2024-06-03T01:18:05.095 INFO:teuthology.orchestra.run.smithi032.stdout:smithi044 172.21.15.44 2024-06-03T01:18:05.095 INFO:teuthology.orchestra.run.smithi032.stdout:2 hosts in cluster 2024-06-03T01:18:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 msgr2=0x7fec9006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 0x7fec9006e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec98005950 tx=0x7fec9800b410 comp rx=0 tx=0).stop 2024-06-03T01:18:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 msgr2=0x7feca80ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0092e0 tx=0x7fec9c00bd70 comp rx=0 tx=0).stop 2024-06-03T01:18:05.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 shutdown_connections 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fec9006c4e0 0x7fec9006e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feca8101760 0x7feca81010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 --2- 172.21.15.32:0/2774673503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feca8104090 0x7feca80ff710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 >> 172.21.15.32:0/2774673503 conn(0x7feca80fb150 msgr2=0x7feca80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 shutdown_connections 2024-06-03T01:18:05.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:05.097+0000 7fecadd75700 1 -- 172.21.15.32:0/2774673503 wait complete. 2024-06-03T01:18:05.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-03T01:18:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:05.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:05 smithi044 ceph-mon[36956]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:06.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:05 smithi032 ceph-mon[29360]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:06.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:06 smithi044 ceph-mon[36956]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:07.030 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:06 smithi032 ceph-mon[29360]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 -- 172.21.15.32:0/1433592782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40ffbc0 msgr2=0x7fbea40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/1433592782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40ffbc0 0x7fbea40fffe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009910 tx=0x7fbea0009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 -- 172.21.15.32:0/1433592782 shutdown_connections 2024-06-03T01:18:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/1433592782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea4100dc0 0x7fbea4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/1433592782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40ffbc0 0x7fbea40fffe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 -- 172.21.15.32:0/1433592782 >> 172.21.15.32:0/1433592782 conn(0x7fbea40fb160 msgr2=0x7fbea40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 -- 172.21.15.32:0/1433592782 shutdown_connections 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.371+0000 7fbeab6fe700 1 -- 172.21.15.32:0/1433592782 wait complete. 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 Processor -- start 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 -- start start 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 0x7fbea41967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41972c0 con 0x7fbea4100dc0 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbeab6fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4197400 con 0x7fbea40ffbc0 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbea8c99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbea8c99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57746/0 (socket says 172.21.15.32:57746) 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.372+0000 7fbea8c99700 1 -- 172.21.15.32:0/2542574926 learned_addr learned my addr 172.21.15.32:0/2542574926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea949a700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 0x7fbea41967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea8c99700 1 -- 172.21.15.32:0/2542574926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 msgr2=0x7fbea41967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea8c99700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 0x7fbea41967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea8c99700 1 -- 172.21.15.32:0/2542574926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00095f0 con 0x7fbea4100dc0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea949a700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 0x7fbea41967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbea8c99700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c930 tx=0x7fbe9400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe94007ad0 con 0x7fbea4100dc0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe94007c30 con 0x7fbea4100dc0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea419bec0 con 0x7fbea4100dc0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe94018630 con 0x7fbea4100dc0 2024-06-03T01:18:07.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.373+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea419c3e0 con 0x7fbea4100dc0 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.374+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4068af0 con 0x7fbea4100dc0 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.375+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbe9400f450 con 0x7fbea4100dc0 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.375+0000 7fbe9a7fc700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 0x7fbe9006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.375+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbe9408b5c0 con 0x7fbea4100dc0 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.375+0000 7fbea949a700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 0x7fbe9006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.376+0000 7fbea949a700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 0x7fbe9006e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b5c0 tx=0x7fbea0005e20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:07.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.379+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbe94059bf0 con 0x7fbea4100dc0 2024-06-03T01:18:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.584+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbea40695e0 con 0x7fbe9006c4e0 2024-06-03T01:18:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.586+0000 7fbe9a7fc700 1 -- 172.21.15.32:0/2542574926 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fbea40695e0 con 0x7fbe9006c4e0 2024-06-03T01:18:07.586 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-03T01:18:07.586 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-06-03T01:18:07.586 INFO:teuthology.orchestra.run.smithi032.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 69s ago Has a FileSystem, LVM detected 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 msgr2=0x7fbe9006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 0x7fbe9006e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b5c0 tx=0x7fbea0005e20 comp rx=0 tx=0).stop 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 msgr2=0x7fbea4196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c930 tx=0x7fbe9400ccf0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 shutdown_connections 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbe9006c4e0 0x7fbe9006e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea40ffbc0 0x7fbea41967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 --2- 172.21.15.32:0/2542574926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4100dc0 0x7fbea4196d30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 >> 172.21.15.32:0/2542574926 conn(0x7fbea40fb160 msgr2=0x7fbea4068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 shutdown_connections 2024-06-03T01:18:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:07.588+0000 7fbeab6fe700 1 -- 172.21.15.32:0/2542574926 wait complete. 2024-06-03T01:18:07.894 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:07 smithi032 ceph-mon[29360]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:07.968 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:07 smithi044 ceph-mon[36956]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:08.178 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:18:08.188 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:08.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-06-03T01:18:08.406 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:08.828 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:08 smithi032 ceph-mon[29360]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:08 smithi044 ceph-mon[36956]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:09.797 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:09 smithi032 ceph-mon[29360]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:09 smithi044 ceph-mon[36956]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 -- 172.21.15.32:0/154733605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 msgr2=0x7f01180fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 --2- 172.21.15.32:0/154733605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f01180fffe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0108009910 tx=0x7f0108009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 -- 172.21.15.32:0/154733605 shutdown_connections 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 --2- 172.21.15.32:0/154733605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f0118101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 --2- 172.21.15.32:0/154733605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f01180fffe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 -- 172.21.15.32:0/154733605 >> 172.21.15.32:0/154733605 conn(0x7f01180fb160 msgr2=0x7f01180fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 -- 172.21.15.32:0/154733605 shutdown_connections 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.040+0000 7f011dcd4700 1 -- 172.21.15.32:0/154733605 wait complete. 2024-06-03T01:18:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 Processor -- start 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 -- start start 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f01181021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01181026e0 con 0x7f01180ffbc0 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f011dcd4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118102820 con 0x7f0118100dc0 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f01177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f01177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57764/0 (socket says 172.21.15.32:57764) 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.041+0000 7f01177fe700 1 -- 172.21.15.32:0/2588021325 learned_addr learned my addr 172.21.15.32:0/2588021325 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f0116ffd700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f01181021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f01177fe700 1 -- 172.21.15.32:0/2588021325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 msgr2=0x7f01181021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f01177fe700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f01181021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f01177fe700 1 -- 172.21.15.32:0/2588021325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080095f0 con 0x7f01180ffbc0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f0116ffd700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f01181021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f01177fe700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f010800bda0 tx=0x7f010800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010801d070 con 0x7f01180ffbc0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0108004c50 con 0x7f01180ffbc0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0118102aa0 con 0x7f01180ffbc0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010800f690 con 0x7f01180ffbc0 2024-06-03T01:18:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.042+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0118102f90 con 0x7f01180ffbc0 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.043+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118068af0 con 0x7f01180ffbc0 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.044+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f01080229b0 con 0x7f01180ffbc0 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.044+0000 7f0114ff9700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 0x7f010006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.044+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f010808cc10 con 0x7f01180ffbc0 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.044+0000 7f0116ffd700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 0x7f010006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.045+0000 7f0116ffd700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 0x7f010006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f010c007900 tx=0x7f010c008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.049+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f010805b240 con 0x7f01180ffbc0 2024-06-03T01:18:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:10.277+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0118069580 con 0x7f010006c4e0 2024-06-03T01:18:11.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:11 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-06-03T01:18:11.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:11 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-06-03T01:18:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.791+0000 7f0114ff9700 1 -- 172.21.15.32:0/2588021325 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0118069580 con 0x7f010006c4e0 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.794+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 msgr2=0x7f010006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.794+0000 7f011dcd4700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 0x7f010006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f010c007900 tx=0x7f010c008040 comp rx=0 tx=0).stop 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.794+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 msgr2=0x7f0118103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.794+0000 7f011dcd4700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f010800bda0 tx=0x7f010800be80 comp rx=0 tx=0).stop 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 shutdown_connections 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01180ffbc0 0x7f0118103b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f010006c4e0 0x7f010006e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 --2- 172.21.15.32:0/2588021325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0118100dc0 0x7f01181021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 >> 172.21.15.32:0/2588021325 conn(0x7f01180fb160 msgr2=0x7f0118068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 shutdown_connections 2024-06-03T01:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:11.795+0000 7f011dcd4700 1 -- 172.21.15.32:0/2588021325 wait complete. 2024-06-03T01:18:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:11 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[29356]: 2024-06-03T01:18:11.767+0000 7f5845053700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: pgmap v170: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-06-03T01:18:12.377 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: fsmap cephfs:0 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: Saving service mds.cephfs spec with placement count:4 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:12.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:12 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:12.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: pgmap v170: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-06-03T01:18:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: fsmap cephfs:0 2024-06-03T01:18:12.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: Saving service mds.cephfs spec with placement count:4 2024-06-03T01:18:12.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:12.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:12 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:12.617 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:13.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:18:13.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:13.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:13.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:13.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:13.285 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:13.285 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:13.285 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: Deploying daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:18:13.285 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: osdmap e44: 6 total, 6 up, 6 in 2024-06-03T01:18:13.285 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:13 smithi032 ceph-mon[29360]: pgmap v174: 65 pgs: 18 creating+peering, 9 active+clean, 38 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:13.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: Deploying daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: osdmap e44: 6 total, 6 up, 6 in 2024-06-03T01:18:13.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:13 smithi044 ceph-mon[36956]: pgmap v174: 65 pgs: 18 creating+peering, 9 active+clean, 38 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 -- 172.21.15.32:0/2427571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788076990 msgr2=0x7ff788076e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/2427571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788076990 0x7ff788076e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff778009910 tx=0x7ff778009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 -- 172.21.15.32:0/2427571595 shutdown_connections 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/2427571595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788076990 0x7ff788076e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/2427571595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788075740 0x7ff788075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 -- 172.21.15.32:0/2427571595 >> 172.21.15.32:0/2427571595 conn(0x7ff7880fe470 msgr2=0x7ff788100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 -- 172.21.15.32:0/2427571595 shutdown_connections 2024-06-03T01:18:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.121+0000 7ff78f3ee700 1 -- 172.21.15.32:0/2427571595 wait complete. 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 Processor -- start 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 -- start start 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788076990 0x7ff78819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78819d630 con 0x7ff788075740 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78f3ee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78819d770 con 0x7ff788076990 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57772/0 (socket says 172.21.15.32:57772) 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 -- 172.21.15.32:0/3629465540 learned_addr learned my addr 172.21.15.32:0/3629465540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78c989700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788076990 0x7ff78819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 -- 172.21.15.32:0/3629465540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788076990 msgr2=0x7ff78819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788076990 0x7ff78819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 -- 172.21.15.32:0/3629465540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780095f0 con 0x7ff788075740 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.122+0000 7ff78d18a700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d8d0 tx=0x7ff78400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.123+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7840098a0 con 0x7ff788075740 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.123+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff784010460 con 0x7ff788075740 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.123+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78400f5f0 con 0x7ff788075740 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.123+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7881a2220 con 0x7ff788075740 2024-06-03T01:18:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.123+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7881a2770 con 0x7ff788075740 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.124+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78810fe00 con 0x7ff788075740 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.124+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff784009a00 con 0x7ff788075740 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.125+0000 7ff77e7fc700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 0x7ff77406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.125+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff78408b630 con 0x7ff788075740 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.125+0000 7ff78c989700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 0x7ff77406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.125+0000 7ff78c989700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 0x7ff77406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff778006010 tx=0x7ff77800b560 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.129+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff784059920 con 0x7ff788075740 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.377+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff7881a2e00 con 0x7ff788075740 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.377+0000 7ff77e7fc700 1 -- 172.21.15.32:0/3629465540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7ff784016460 con 0x7ff788075740 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout:e2 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:14.378 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 2 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:18:11.768224+0000 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:18:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:in 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:up {} 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:18:14.380 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2024-06-03T01:18:14.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:14.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 msgr2=0x7ff77406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 0x7ff77406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff778006010 tx=0x7ff77800b560 comp rx=0 tx=0).stop 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 msgr2=0x7ff78819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d8d0 tx=0x7ff78400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 shutdown_connections 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff788075740 0x7ff78819cad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff77406c4e0 0x7ff77406e9a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 --2- 172.21.15.32:0/3629465540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff788076990 0x7ff78819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.379+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 >> 172.21.15.32:0/3629465540 conn(0x7ff7880fe470 msgr2=0x7ff78810d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.380+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 shutdown_connections 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:14.380+0000 7ff78f3ee700 1 -- 172.21.15.32:0/3629465540 wait complete. 2024-06-03T01:18:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-06-03T01:18:14.457 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:14 smithi032 ceph-mon[29360]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:18:14.457 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:14 smithi032 ceph-mon[29360]: osdmap e45: 6 total, 6 up, 6 in 2024-06-03T01:18:14.457 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:14.458 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:14 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:14 smithi044 ceph-mon[36956]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:18:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:14 smithi044 ceph-mon[36956]: osdmap e45: 6 total, 6 up, 6 in 2024-06-03T01:18:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:14 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:15.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:15 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3629465540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:18:15.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:15 smithi032 ceph-mon[29360]: pgmap v176: 65 pgs: 18 creating+peering, 42 active+clean, 5 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:15 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3629465540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:18:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:15 smithi044 ceph-mon[36956]: pgmap v176: 65 pgs: 18 creating+peering, 42 active+clean, 5 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:15.927 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:18:15.936 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:15.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-06-03T01:18:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:17.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: Deploying daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:boot 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: daemon mds.cephfs.smithi032.itqzrm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: Cluster is now healthy 2024-06-03T01:18:17.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: fsmap cephfs:0 1 up:standby 2024-06-03T01:18:17.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:18:17.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:creating} 2024-06-03T01:18:17.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: daemon mds.cephfs.smithi032.itqzrm is now active in filesystem cephfs as rank 0 2024-06-03T01:18:17.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:17 smithi044 ceph-mon[36956]: pgmap v177: 65 pgs: 11 creating+peering, 54 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:17.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:17.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:17.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: Deploying daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:boot 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: daemon mds.cephfs.smithi032.itqzrm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: Cluster is now healthy 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: fsmap cephfs:0 1 up:standby 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:creating} 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: daemon mds.cephfs.smithi032.itqzrm is now active in filesystem cephfs as rank 0 2024-06-03T01:18:17.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:17 smithi032 ceph-mon[29360]: pgmap v177: 65 pgs: 11 creating+peering, 54 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 -- 172.21.15.32:0/2316682258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c8074dc0 msgr2=0x7f10c8073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/2316682258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c8074dc0 0x7f10c8073220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009910 tx=0x7f10b8009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 -- 172.21.15.32:0/2316682258 shutdown_connections 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/2316682258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c80737f0 0x7f10c8073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/2316682258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c8074dc0 0x7f10c8073220 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 -- 172.21.15.32:0/2316682258 >> 172.21.15.32:0/2316682258 conn(0x7f10c80fc230 msgr2=0x7f10c80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 -- 172.21.15.32:0/2316682258 shutdown_connections 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.194+0000 7f10cc9f8700 1 -- 172.21.15.32:0/2316682258 wait complete. 2024-06-03T01:18:18.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 Processor -- start 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 -- start start 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 0x7f10c819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c819d580 con 0x7f10c80737f0 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10cc9f8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c819d6c0 con 0x7f10c8074dc0 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51168/0 (socket says 172.21.15.32:51168) 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 -- 172.21.15.32:0/3630041592 learned_addr learned my addr 172.21.15.32:0/3630041592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c5d9b700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 0x7f10c819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 -- 172.21.15.32:0/3630041592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 msgr2=0x7f10c819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 0x7f10c819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c659c700 1 -- 172.21.15.32:0/3630041592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80095f0 con 0x7f10c80737f0 2024-06-03T01:18:18.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.195+0000 7f10c5d9b700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 0x7f10c819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10c659c700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10b800bda0 tx=0x7f10b800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b801d070 con 0x7f10c80737f0 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c81a2120 con 0x7f10c80737f0 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f10b8004c50 con 0x7f10c80737f0 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b800f690 con 0x7f10c80737f0 2024-06-03T01:18:18.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.196+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c81a25e0 con 0x7f10c80737f0 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.197+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c810fe20 con 0x7f10c80737f0 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.197+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f10b8022a20 con 0x7f10c80737f0 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.198+0000 7f10b77fe700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 0x7f10b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.198+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f10b808cec0 con 0x7f10c80737f0 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.198+0000 7f10c5d9b700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 0x7f10b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.198+0000 7f10c5d9b700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 0x7f10b006e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10c8074af0 tx=0x7f10bc00a680 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:18.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.203+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10b805b300 con 0x7f10c80737f0 2024-06-03T01:18:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.443+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f10c81a2c20 con 0x7f10c80737f0 2024-06-03T01:18:18.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.698+0000 7f10b77fe700 1 -- 172.21.15.32:0/3630041592 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f10b80270b0 con 0x7f10c80737f0 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 msgr2=0x7f10b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 0x7f10b006e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10c8074af0 tx=0x7f10bc00a680 comp rx=0 tx=0).stop 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 msgr2=0x7f10c819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10b800bda0 tx=0x7f10b800be80 comp rx=0 tx=0).stop 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 shutdown_connections 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10c80737f0 0x7f10c819cab0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f10b006c4e0 0x7f10b006e9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 --2- 172.21.15.32:0/3630041592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c8074dc0 0x7f10c819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 >> 172.21.15.32:0/3630041592 conn(0x7f10c80fc230 msgr2=0x7f10c8102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:18.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 shutdown_connections 2024-06-03T01:18:18.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:18.701+0000 7f10cc9f8700 1 -- 172.21.15.32:0/3630041592 wait complete. 2024-06-03T01:18:18.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:18 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:active 2024-06-03T01:18:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:18 smithi044 ceph-mon[36956]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:active} 2024-06-03T01:18:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:18 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3630041592' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-06-03T01:18:19.007 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:18 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:active 2024-06-03T01:18:19.007 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:18 smithi032 ceph-mon[29360]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:active} 2024-06-03T01:18:19.007 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:18 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3630041592' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-06-03T01:18:19.323 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:18:19.333 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:19.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-06-03T01:18:19.560 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:19.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/3630041592' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:active} 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 574 B/s wr, 4 op/s 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: daemon mds.cephfs.smithi044.gkcyfr assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T01:18:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-06-03T01:18:19.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:18:19.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.44:6824/372165216,v1:172.21.15.44:6825/372165216] up:boot 2024-06-03T01:18:19.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:creating} 2024-06-03T01:18:19.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:19 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:19.993 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:18:19.993 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/3630041592' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-06-03T01:18:19.993 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: fsmap cephfs:1 {0=cephfs.smithi032.itqzrm=up:active} 2024-06-03T01:18:19.993 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 574 B/s wr, 4 op/s 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: daemon mds.cephfs.smithi044.gkcyfr assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.44:6824/372165216,v1:172.21.15.44:6825/372165216] up:boot 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:creating} 2024-06-03T01:18:19.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:19 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:20.916 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:20 smithi044 ceph-mon[36956]: Deploying daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:18:20.916 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:20 smithi044 ceph-mon[36956]: daemon mds.cephfs.smithi044.gkcyfr is now active in filesystem cephfs as rank 1 2024-06-03T01:18:21.015 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:20 smithi032 ceph-mon[29360]: Deploying daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:18:21.015 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:20 smithi032 ceph-mon[29360]: daemon mds.cephfs.smithi044.gkcyfr is now active in filesystem cephfs as rank 1 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 -- 172.21.15.32:0/2991129600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 msgr2=0x7fc6180ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/2991129600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6180ff960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc614009910 tx=0x7fc614009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 -- 172.21.15.32:0/2991129600 shutdown_connections 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/2991129600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 0x7fc618100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/2991129600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6180ff960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 -- 172.21.15.32:0/2991129600 >> 172.21.15.32:0/2991129600 conn(0x7fc6180fb160 msgr2=0x7fc6180fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 -- 172.21.15.32:0/2991129600 shutdown_connections 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.235+0000 7fc61ff1c700 1 -- 172.21.15.32:0/2991129600 wait complete. 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 Processor -- start 2024-06-03T01:18:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 -- start start 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 0x7fc618198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc618199520 con 0x7fc6180ff540 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61ff1c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc618199660 con 0x7fc6180ffea0 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51182/0 (socket says 172.21.15.32:51182) 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 -- 172.21.15.32:0/4012111262 learned_addr learned my addr 172.21.15.32:0/4012111262 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61d4b7700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 0x7fc618198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 -- 172.21.15.32:0/4012111262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 msgr2=0x7fc618198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 0x7fc618198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 -- 172.21.15.32:0/4012111262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6140095f0 con 0x7fc6180ff540 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.236+0000 7fc61dcb8700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6140092e0 tx=0x7fc61400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.237+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61401d070 con 0x7fc6180ff540 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.237+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc614005dc0 con 0x7fc6180ff540 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.237+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61819e0b0 con 0x7fc6180ff540 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.237+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc614017630 con 0x7fc6180ff540 2024-06-03T01:18:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.237+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61819e5a0 con 0x7fc6180ff540 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.238+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc618192820 con 0x7fc6180ff540 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.238+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc61400faf0 con 0x7fc6180ff540 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.239+0000 7fc60effd700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 0x7fc60406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.239+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc61408cbd0 con 0x7fc6180ff540 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.239+0000 7fc61d4b7700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 0x7fc60406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.239+0000 7fc61d4b7700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 0x7fc60406e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c60 tx=0x7fc608009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.243+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc61405ae40 con 0x7fc6180ff540 2024-06-03T01:18:21.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.489+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc618068af0 con 0x7fc6180ff540 2024-06-03T01:18:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.712+0000 7fc60effd700 1 -- 172.21.15.32:0/4012111262 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc61405a9d0 con 0x7fc6180ff540 2024-06-03T01:18:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 msgr2=0x7fc60406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 0x7fc60406e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c60 tx=0x7fc608009260 comp rx=0 tx=0).stop 2024-06-03T01:18:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 msgr2=0x7fc6181989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6140092e0 tx=0x7fc61400f690 comp rx=0 tx=0).stop 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 shutdown_connections 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6180ff540 0x7fc6181989c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc60406c4e0 0x7fc60406e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 --2- 172.21.15.32:0/4012111262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6180ffea0 0x7fc618198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 >> 172.21.15.32:0/4012111262 conn(0x7fc6180fb160 msgr2=0x7fc618107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 shutdown_connections 2024-06-03T01:18:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:21.714+0000 7fc61ff1c700 1 -- 172.21.15.32:0/4012111262 wait complete. 2024-06-03T01:18:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:21 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.44:6824/372165216,v1:172.21.15.44:6825/372165216] up:active 2024-06-03T01:18:21.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:21 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2024-06-03T01:18:21.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:21 smithi044 ceph-mon[36956]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-03T01:18:21.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:21 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/4012111262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-06-03T01:18:22.050 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:21 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.44:6824/372165216,v1:172.21.15.44:6825/372165216] up:active 2024-06-03T01:18:22.051 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:21 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2024-06-03T01:18:22.051 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:21 smithi032 ceph-mon[29360]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-03T01:18:22.051 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:21 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/4012111262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-06-03T01:18:22.339 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:18:22.348 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:22.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-06-03T01:18:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:22.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:22 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/4012111262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-06-03T01:18:22.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:22 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2024-06-03T01:18:23.029 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:22 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/4012111262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-06-03T01:18:23.030 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:22 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2024-06-03T01:18:23.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:23 smithi044 ceph-mon[36956]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s wr, 6 op/s 2024-06-03T01:18:24.029 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:23 smithi032 ceph-mon[29360]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s wr, 6 op/s 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.559+0000 7fd88c54f700 1 -- 172.21.15.32:0/3536978736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 msgr2=0x7fd8840fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.559+0000 7fd88c54f700 1 --2- 172.21.15.32:0/3536978736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840fffe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd880009910 tx=0x7fd880009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- 172.21.15.32:0/3536978736 shutdown_connections 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 --2- 172.21.15.32:0/3536978736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 0x7fd884101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 --2- 172.21.15.32:0/3536978736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840fffe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- 172.21.15.32:0/3536978736 >> 172.21.15.32:0/3536978736 conn(0x7fd8840fb160 msgr2=0x7fd8840fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- 172.21.15.32:0/3536978736 shutdown_connections 2024-06-03T01:18:24.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- 172.21.15.32:0/3536978736 wait complete. 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 Processor -- start 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- start start 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 0x7fd88419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88419bb80 con 0x7fd8840ffbc0 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.560+0000 7fd88c54f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88419bcc0 con 0x7fd884100dc0 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51196/0 (socket says 172.21.15.32:51196) 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 -- 172.21.15.32:0/4034326000 learned_addr learned my addr 172.21.15.32:0/4034326000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd889aea700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 0x7fd88419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 -- 172.21.15.32:0/4034326000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 msgr2=0x7fd88419b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 0x7fd88419b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 -- 172.21.15.32:0/4034326000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800095f0 con 0x7fd8840ffbc0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88a2eb700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bda0 tx=0x7fd88000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88001d070 con 0x7fd8840ffbc0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88419bf40 con 0x7fd8840ffbc0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd880004c50 con 0x7fd8840ffbc0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88000f690 con 0x7fd8840ffbc0 2024-06-03T01:18:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.561+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd884195330 con 0x7fd8840ffbc0 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.562+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88418f5b0 con 0x7fd8840ffbc0 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.563+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd880022a20 con 0x7fd8840ffbc0 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.563+0000 7fd87b7fe700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 0x7fd870072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.563+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd88008cfe0 con 0x7fd8840ffbc0 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.563+0000 7fd889aea700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 0x7fd870072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.564+0000 7fd889aea700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 0x7fd870072db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.568+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd8800c00e0 con 0x7fd8840ffbc0 2024-06-03T01:18:24.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:24.821+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd884068af0 con 0x7fd8840ffbc0 2024-06-03T01:18:25.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:24 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:25.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:24 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/4034326000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:18:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:24 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:24 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/4034326000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:18:25.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.695+0000 7fd87b7fe700 1 -- 172.21.15.32:0/4034326000 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7fd88005b260 con 0x7fd8840ffbc0 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 msgr2=0x7fd870072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 0x7fd870072db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).stop 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 msgr2=0x7fd8840699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd88000bda0 tx=0x7fd88000be80 comp rx=0 tx=0).stop 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 shutdown_connections 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8840ffbc0 0x7fd8840699d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fd8700708f0 0x7fd870072db0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 --2- 172.21.15.32:0/4034326000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd884100dc0 0x7fd88419b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 >> 172.21.15.32:0/4034326000 conn(0x7fd8840fb160 msgr2=0x7fd8841040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 shutdown_connections 2024-06-03T01:18:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:25.699+0000 7fd88c54f700 1 -- 172.21.15.32:0/4034326000 wait complete. 2024-06-03T01:18:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:inline data enabled 2024-06-03T01:18:25.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/4034326000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:boot 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 1 up:standby 2024-06-03T01:18:25.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:25 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:18:26.000 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-03T01:18:26.000 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:26.000 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:26.000 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/4034326000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:boot 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 1 up:standby 2024-06-03T01:18:26.001 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:25 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:18:26.328 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:18:26.338 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:26.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T01:18:26.561 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:27.073 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:26 smithi044 ceph-mon[36956]: Deploying daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:18:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:26 smithi032 ceph-mon[29360]: Deploying daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:18:28.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 -- 172.21.15.32:0/1985402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 msgr2=0x7fbc3c106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1985402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c106460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009910 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 -- 172.21.15.32:0/1985402483 shutdown_connections 2024-06-03T01:18:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1985402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c106460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1985402483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 -- 172.21.15.32:0/1985402483 >> 172.21.15.32:0/1985402483 conn(0x7fbc3c0fb150 msgr2=0x7fbc3c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 -- 172.21.15.32:0/1985402483 shutdown_connections 2024-06-03T01:18:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.128+0000 7fbc4307f700 1 -- 172.21.15.32:0/1985402483 wait complete. 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 Processor -- start 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 -- start start 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c196fc0 con 0x7fbc3c104070 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc4307f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c197100 con 0x7fbc3c101740 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc40e1b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42612/0 (socket says 172.21.15.32:42612) 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 -- 172.21.15.32:0/1164742381 learned_addr learned my addr 172.21.15.32:0/1164742381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 -- 172.21.15.32:0/1164742381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 msgr2=0x7fbc3c196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc3bfff700 1 -- 172.21.15.32:0/1164742381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280095f0 con 0x7fbc3c104070 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.129+0000 7fbc40e1b700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc3bfff700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc28000c00 tx=0x7fbc2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2801d070 con 0x7fbc3c104070 2024-06-03T01:18:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc28005dc0 con 0x7fbc3c104070 2024-06-03T01:18:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc280226a0 con 0x7fbc3c104070 2024-06-03T01:18:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c19bb50 con 0x7fbc3c104070 2024-06-03T01:18:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.130+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c19c040 con 0x7fbc3c104070 2024-06-03T01:18:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.131+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c190620 con 0x7fbc3c104070 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.132+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbc280038f0 con 0x7fbc3c104070 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.133+0000 7fbc39ffb700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 0x7fbc2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.133+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbc2808cc50 con 0x7fbc3c104070 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.133+0000 7fbc40e1b700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 0x7fbc2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.133+0000 7fbc40e1b700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 0x7fbc2c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc30011490 tx=0x7fbc30009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.138+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc2805af40 con 0x7fbc3c104070 2024-06-03T01:18:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-03T01:18:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.383+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc3c02cc70 con 0x7fbc3c104070 2024-06-03T01:18:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.383+0000 7fbc39ffb700 1 -- 172.21.15.32:0/1164742381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1777 (secure 0 0 0) 0x7fbc28003680 con 0x7fbc3c104070 2024-06-03T01:18:28.383 INFO:teuthology.orchestra.run.smithi032.stdout:e11 2024-06-03T01:18:28.383 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 10 2024-06-03T01:18:28.385 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:18:25.690015+0000 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:18:28.386 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=24277} 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:18:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 2 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{1:24277} state up:active seq 2 addr [v2:172.21.15.44:6824/372165216,v1:172.21.15.44:6825/372165216] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{-1:14498} state up:standby seq 1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:18:28.388 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.386+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 msgr2=0x7fbc2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.386+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 0x7fbc2c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc30011490 tx=0x7fbc30009260 comp rx=0 tx=0).stop 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.386+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 msgr2=0x7fbc3c1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.386+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc28000c00 tx=0x7fbc2800f690 comp rx=0 tx=0).stop 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 shutdown_connections 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbc2c06c4e0 0x7fbc2c06e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc3c101740 0x7fbc3c196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 --2- 172.21.15.32:0/1164742381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc3c104070 0x7fbc3c1969a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 >> 172.21.15.32:0/1164742381 conn(0x7fbc3c0fb150 msgr2=0x7fbc3c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 shutdown_connections 2024-06-03T01:18:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:28.387+0000 7fbc4307f700 1 -- 172.21.15.32:0/1164742381 wait complete. 2024-06-03T01:18:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2024-06-03T01:18:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] up:boot 2024-06-03T01:18:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2 up:standby 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1164742381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:28.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-06-03T01:18:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] up:boot 2024-06-03T01:18:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2 up:standby 2024-06-03T01:18:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:18:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1164742381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:18:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:29.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:30.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:active 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2 up:standby 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: Dropping low affinity active daemon mds.cephfs.smithi044.gkcyfr in favor of higher affinity standby. 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: Replacing daemon mds.cephfs.smithi044.gkcyfr as rank 1 with standby daemon mds.cephfs.smithi032.tegslf 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: osdmap e46: 6 total, 6 up, 6 in 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:standby 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:replay} 1 up:standby 2024-06-03T01:18:30.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:29 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-03T01:18:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] up:active 2024-06-03T01:18:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi044.gkcyfr=up:active} 2 up:standby 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: Dropping low affinity active daemon mds.cephfs.smithi044.gkcyfr in favor of higher affinity standby. 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: Replacing daemon mds.cephfs.smithi044.gkcyfr as rank 1 with standby daemon mds.cephfs.smithi032.tegslf 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: osdmap e46: 6 total, 6 up, 6 in 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:standby 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:replay} 1 up:standby 2024-06-03T01:18:30.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:29 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/2134621976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 msgr2=0x7f5e641010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/2134621976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e641010d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009910 tx=0x7f5e54009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/2134621976 shutdown_connections 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/2134621976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 0x7f5e64102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/2134621976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e641010d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/2134621976 >> 172.21.15.32:0/2134621976 conn(0x7f5e640fc230 msgr2=0x7f5e640fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/2134621976 shutdown_connections 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/2134621976 wait complete. 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 Processor -- start 2024-06-03T01:18:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.201+0000 7f5e6a6a3700 1 -- start start 2024-06-03T01:18:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e6a6a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e6a6a3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 0x7f5e64196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e6a6a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641970c0 con 0x7f5e64100cb0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e6a6a3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64197200 con 0x7f5e64101eb0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42626/0 (socket says 172.21.15.32:42626) 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 -- 172.21.15.32:0/1445820251 learned_addr learned my addr 172.21.15.32:0/1445820251 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e637fe700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 0x7f5e64196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 -- 172.21.15.32:0/1445820251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 msgr2=0x7f5e64196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 0x7f5e64196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 -- 172.21.15.32:0/1445820251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540095f0 con 0x7f5e64100cb0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.202+0000 7f5e63fff700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400bda0 tx=0x7f5e5400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.203+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5401d070 con 0x7f5e64100cb0 2024-06-03T01:18:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.203+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e54022470 con 0x7f5e64100cb0 2024-06-03T01:18:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.203+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6419bc50 con 0x7f5e64100cb0 2024-06-03T01:18:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.203+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5400f650 con 0x7f5e64100cb0 2024-06-03T01:18:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.203+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6419c140 con 0x7f5e64100cb0 2024-06-03T01:18:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.204+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e64109890 con 0x7f5e64100cb0 2024-06-03T01:18:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.204+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5e540225e0 con 0x7f5e64100cb0 2024-06-03T01:18:31.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.205+0000 7f5e617fa700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 0x7f5e4c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:31.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.205+0000 7f5e637fe700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 0x7f5e4c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:31.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.205+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5e5408cef0 con 0x7f5e64100cb0 2024-06-03T01:18:31.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.205+0000 7f5e637fe700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 0x7f5e4c06e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009a10 tx=0x7f5e58009190 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.209+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5e5405b1b0 con 0x7f5e64100cb0 2024-06-03T01:18:31.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:resolve 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] up:boot 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:resolve} 2 up:standby 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:31.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:31 smithi032 ceph-mon[29360]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-03T01:18:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.458+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e64109a80 con 0x7f5e64100cb0 2024-06-03T01:18:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.459+0000 7f5e617fa700 1 -- 172.21.15.32:0/1445820251 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4756 (secure 0 0 0) 0x7f5e5405ad40 con 0x7f5e64100cb0 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 msgr2=0x7f5e4c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 0x7f5e4c06e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009a10 tx=0x7f5e58009190 comp rx=0 tx=0).stop 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 msgr2=0x7f5e64196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400bda0 tx=0x7f5e5400bdd0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 shutdown_connections 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e64100cb0 0x7f5e64196560 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f5e4c06c4e0 0x7f5e4c06e9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 --2- 172.21.15.32:0/1445820251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e64101eb0 0x7f5e64196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 >> 172.21.15.32:0/1445820251 conn(0x7f5e640fc230 msgr2=0x7f5e641050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 shutdown_connections 2024-06-03T01:18:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:31.461+0000 7f5e6a6a3700 1 -- 172.21.15.32:0/1445820251 wait complete. 2024-06-03T01:18:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 14 2024-06-03T01:18:31.471 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:18:31.706 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:31.706 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:31.706 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:resolve 2024-06-03T01:18:31.706 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] up:boot 2024-06-03T01:18:31.707 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:resolve} 2 up:standby 2024-06-03T01:18:31.707 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:18:31.707 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:31 smithi044 ceph-mon[36956]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-03T01:18:32.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-06-03T01:18:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:32 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1445820251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:18:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:32 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:reconnect 2024-06-03T01:18:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:32 smithi032 ceph-mon[29360]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:reconnect} 2 up:standby 2024-06-03T01:18:32.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:32 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1445820251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:18:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:32 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:reconnect 2024-06-03T01:18:32.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:32 smithi044 ceph-mon[36956]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:reconnect} 2 up:standby 2024-06-03T01:18:32.752 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:33 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:rejoin 2024-06-03T01:18:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:33 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] up:standby 2024-06-03T01:18:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:33 smithi044 ceph-mon[36956]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:rejoin} 2 up:standby 2024-06-03T01:18:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:33 smithi044 ceph-mon[36956]: daemon mds.cephfs.smithi032.tegslf is now active in filesystem cephfs as rank 1 2024-06-03T01:18:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:33 smithi044 ceph-mon[36956]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 102 B/s wr, 3 op/s 2024-06-03T01:18:34.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:33 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:rejoin 2024-06-03T01:18:34.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:33 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] up:standby 2024-06-03T01:18:34.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:33 smithi032 ceph-mon[29360]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:rejoin} 2 up:standby 2024-06-03T01:18:34.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:33 smithi032 ceph-mon[29360]: daemon mds.cephfs.smithi032.tegslf is now active in filesystem cephfs as rank 1 2024-06-03T01:18:34.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:33 smithi032 ceph-mon[29360]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 102 B/s wr, 3 op/s 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- 172.21.15.32:0/2657794790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c80ffbc0 msgr2=0x7f43c80fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/2657794790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c80ffbc0 0x7f43c80fffe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009910 tx=0x7f43b8009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- 172.21.15.32:0/2657794790 shutdown_connections 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/2657794790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c8100dc0 0x7f43c8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/2657794790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c80ffbc0 0x7f43c80fffe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- 172.21.15.32:0/2657794790 >> 172.21.15.32:0/2657794790 conn(0x7f43c80fb160 msgr2=0x7f43c80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- 172.21.15.32:0/2657794790 shutdown_connections 2024-06-03T01:18:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- 172.21.15.32:0/2657794790 wait complete. 2024-06-03T01:18:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 Processor -- start 2024-06-03T01:18:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.533+0000 7f43cd3dd700 1 -- start start 2024-06-03T01:18:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43cd3dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c80ffbc0 0x7f43c8103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43cd3dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43cd3dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81026e0 con 0x7f43c8100dc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43cd3dd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c8102820 con 0x7f43c80ffbc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42650/0 (socket says 172.21.15.32:42650) 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 -- 172.21.15.32:0/771837214 learned_addr learned my addr 172.21.15.32:0/771837214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c6ffd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c80ffbc0 0x7f43c8103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 -- 172.21.15.32:0/771837214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c80ffbc0 msgr2=0x7f43c8103b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c80ffbc0 0x7f43c8103b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 -- 172.21.15.32:0/771837214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80095f0 con 0x7f43c8100dc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43c67fc700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8d0 tx=0x7f43bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc0098a0 con 0x7f43c8100dc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f43bc010460 con 0x7f43c8100dc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.534+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c8102b00 con 0x7f43c8100dc0 2024-06-03T01:18:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.535+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc00f5f0 con 0x7f43c8100dc0 2024-06-03T01:18:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.535+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c819e850 con 0x7f43c8100dc0 2024-06-03T01:18:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.535+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c8068af0 con 0x7f43c8100dc0 2024-06-03T01:18:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.536+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f43bc009a00 con 0x7f43c8100dc0 2024-06-03T01:18:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.536+0000 7f43affff700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 0x7f43b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.537+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f43bc08b690 con 0x7f43c8100dc0 2024-06-03T01:18:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.537+0000 7f43c6ffd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 0x7f43b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.537+0000 7f43c6ffd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 0x7f43b006e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005090 tx=0x7f43b8005a90 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.541+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f43bc059920 con 0x7f43c8100dc0 2024-06-03T01:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.832+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f43c819ed50 con 0x7f43c8100dc0 2024-06-03T01:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.832+0000 7f43affff700 1 -- 172.21.15.32:0/771837214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7f43bc0594b0 con 0x7f43c8100dc0 2024-06-03T01:18:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 msgr2=0x7f43b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 0x7f43b006e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005090 tx=0x7f43b8005a90 comp rx=0 tx=0).stop 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 msgr2=0x7f43c81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8d0 tx=0x7f43bc00dbe0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 shutdown_connections 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f43b006c4e0 0x7f43b006e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43c80ffbc0 0x7f43c8103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 --2- 172.21.15.32:0/771837214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43c8100dc0 0x7f43c81021a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.834+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 >> 172.21.15.32:0/771837214 conn(0x7f43c80fb160 msgr2=0x7f43c8068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.835+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 shutdown_connections 2024-06-03T01:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:34.835+0000 7f43cd3dd700 1 -- 172.21.15.32:0/771837214 wait complete. 2024-06-03T01:18:34.845 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:18:34.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:34 smithi044 ceph-mon[36956]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:18:34.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:34 smithi044 ceph-mon[36956]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:active 2024-06-03T01:18:34.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:34 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:18:35.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:34 smithi032 ceph-mon[29360]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:18:35.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:34 smithi032 ceph-mon[29360]: mds.? [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] up:active 2024-06-03T01:18:35.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:34 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:18:35.718 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-06-03T01:18:35.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:18:35.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:35 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/771837214' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-06-03T01:18:35.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:35 smithi032 ceph-mon[29360]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-03T01:18:35.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:35 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/771837214' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-06-03T01:18:35.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:35 smithi044 ceph-mon[36956]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-03T01:18:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:37.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-03T01:18:37.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:37.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:37.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:37.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:37.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:37.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 -- 172.21.15.32:0/309816364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 msgr2=0x7fa860105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 --2- 172.21.15.32:0/309816364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa860105730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009910 tx=0x7fa85c009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 -- 172.21.15.32:0/309816364 shutdown_connections 2024-06-03T01:18:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 --2- 172.21.15.32:0/309816364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa8600ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 --2- 172.21.15.32:0/309816364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa860105730 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 -- 172.21.15.32:0/309816364 >> 172.21.15.32:0/309816364 conn(0x7fa860074e30 msgr2=0x7fa860075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 -- 172.21.15.32:0/309816364 shutdown_connections 2024-06-03T01:18:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.078+0000 7fa8686f6700 1 -- 172.21.15.32:0/309816364 wait complete. 2024-06-03T01:18:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 Processor -- start 2024-06-03T01:18:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 -- start start 2024-06-03T01:18:38.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa86019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:38.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:38.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601955a0 con 0x7fa860105350 2024-06-03T01:18:38.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa8686f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860195710 con 0x7fa8600ff340 2024-06-03T01:18:38.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36858/0 (socket says 172.21.15.32:36858) 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 -- 172.21.15.32:0/4263743902 learned_addr learned my addr 172.21.15.32:0/4263743902 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa866492700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa86019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 -- 172.21.15.32:0/4263743902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 msgr2=0x7fa86019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa86019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 -- 172.21.15.32:0/4263743902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0095f0 con 0x7fa860105350 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa866492700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa86019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.079+0000 7fa865c91700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e920 tx=0x7fa85000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.080+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa85000c9d0 con 0x7fa860105350 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.080+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8601959f0 con 0x7fa860105350 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.080+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa850010430 con 0x7fa860105350 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.080+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa850018650 con 0x7fa860105350 2024-06-03T01:18:38.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.080+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860195f40 con 0x7fa860105350 2024-06-03T01:18:38.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.081+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa860107850 con 0x7fa860105350 2024-06-03T01:18:38.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.082+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa8500105a0 con 0x7fa860105350 2024-06-03T01:18:38.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.082+0000 7fa8577fe700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 0x7fa84c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:38.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.082+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa850014070 con 0x7fa860105350 2024-06-03T01:18:38.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.082+0000 7fa866492700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 0x7fa84c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:38.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.083+0000 7fa866492700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 0x7fa84c06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0059d0 tx=0x7fa85c005940 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:38.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.091+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa8500520e0 con 0x7fa860105350 2024-06-03T01:18:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-03T01:18:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.361+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa860196ba0 con 0x7fa860105350 2024-06-03T01:18:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.361+0000 7fa8577fe700 1 -- 172.21.15.32:0/4263743902 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4754 (secure 0 0 0) 0x7fa8500562d0 con 0x7fa860105350 2024-06-03T01:18:38.361 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:18:38.362 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:18:33.701383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6827/1879817357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1879817357},{"type":"v1","addr":"172.21.15.32:6827","nonce":1879817357}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6829/3721991653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3721991653},{"type":"v1","addr":"172.21.15.32:6829","nonce":3721991653}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-03T01:18:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 msgr2=0x7fa84c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 0x7fa84c06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0059d0 tx=0x7fa85c005940 comp rx=0 tx=0).stop 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 msgr2=0x7fa86019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e920 tx=0x7fa85000ec30 comp rx=0 tx=0).stop 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 shutdown_connections 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fa84c06c4e0 0x7fa84c06e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8600ff340 0x7fa86019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 --2- 172.21.15.32:0/4263743902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa860105350 0x7fa86019baf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 >> 172.21.15.32:0/4263743902 conn(0x7fa860074e30 msgr2=0x7fa8600fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.363+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 shutdown_connections 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:38.364+0000 7fa8686f6700 1 -- 172.21.15.32:0/4263743902 wait complete. 2024-06-03T01:18:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2024-06-03T01:18:39.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:38 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/4263743902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:18:39.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:38 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/4263743902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:18:39.469 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-06-03T01:18:39.469 INFO:teuthology.run_tasks:Running task kclient... 2024-06-03T01:18:39.498 INFO:tasks.kclient:Mounting kernel clients... 2024-06-03T01:18:39.498 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-06-03T01:18:39.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:39.499 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-06-03T01:18:39.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:39.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-06-03T01:18:39.599 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-06-03T01:18:39.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:18:39.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:39.601 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-06-03T01:18:39.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:39.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-06-03T01:18:39.690 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-06-03T01:18:39.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:18:39.693 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-06-03T01:18:39.693 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T01:18:39.693 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:39.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T01:18:40.015 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:39 smithi032 ceph-mon[29360]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-03T01:18:40.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:40.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:39 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:39 smithi044 ceph-mon[36956]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-03T01:18:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:40.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:39 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:40.516 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:41 smithi044 ceph-mon[36956]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-03T01:18:42.321 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:41 smithi032 ceph-mon[29360]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-03T01:18:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 -- 172.21.15.32:0/3245780035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 msgr2=0x7f45901070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 --2- 172.21.15.32:0/3245780035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f45901070b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4580009910 tx=0x7f4580009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 -- 172.21.15.32:0/3245780035 shutdown_connections 2024-06-03T01:18:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 --2- 172.21.15.32:0/3245780035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 0x7f45900689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 --2- 172.21.15.32:0/3245780035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f45901070b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 -- 172.21.15.32:0/3245780035 >> 172.21.15.32:0/3245780035 conn(0x7f4590075670 msgr2=0x7f4590075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 -- 172.21.15.32:0/3245780035 shutdown_connections 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.221+0000 7f4596f81700 1 -- 172.21.15.32:0/3245780035 wait complete. 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 Processor -- start 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 -- start start 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 0x7f459019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019d2b0 con 0x7f4590106cd0 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4596f81700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4590196760 con 0x7f4590068580 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36878/0 (socket says 172.21.15.32:36878) 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 -- 172.21.15.32:0/1211885160 learned_addr learned my addr 172.21.15.32:0/1211885160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:43.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f4594d1d700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 0x7f459019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 -- 172.21.15.32:0/1211885160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 msgr2=0x7f459019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 0x7f459019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.222+0000 7f458ffff700 1 -- 172.21.15.32:0/1211885160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800095f0 con 0x7f4590106cd0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f458ffff700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f458400b4b0 tx=0x7f458400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4584011800 con 0x7f4590106cd0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4584011e10 con 0x7f4590106cd0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590196a40 con 0x7f4590106cd0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458400f2e0 con 0x7f4590106cd0 2024-06-03T01:18:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.223+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590196f90 con 0x7f4590106cd0 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.224+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45901091d0 con 0x7f4590106cd0 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.225+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4584010380 con 0x7f4590106cd0 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.225+0000 7f458dffb700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 0x7f457806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.225+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f458408b430 con 0x7f4590106cd0 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.225+0000 7f4594d1d700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 0x7f457806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.226+0000 7f4594d1d700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 0x7f457806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f458001a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.230+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4584059640 con 0x7f4590106cd0 2024-06-03T01:18:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.490+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f459004f2e0 con 0x7f4590106cd0 2024-06-03T01:18:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.490+0000 7f458dffb700 1 -- 172.21.15.32:0/1211885160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f45840591d0 con 0x7f4590106cd0 2024-06-03T01:18:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-06-03T01:18:43.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 msgr2=0x7f457806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 0x7f457806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f458001a040 comp rx=0 tx=0).stop 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 msgr2=0x7f459019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f458400b4b0 tx=0x7f458400b7c0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 shutdown_connections 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f457806c4e0 0x7f457806e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4590068580 0x7f459019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 --2- 172.21.15.32:0/1211885160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4590106cd0 0x7f459019cc20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 >> 172.21.15.32:0/1211885160 conn(0x7f4590075670 msgr2=0x7f45900fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.492+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 shutdown_connections 2024-06-03T01:18:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:43.493+0000 7f4596f81700 1 -- 172.21.15.32:0/1211885160 wait complete. 2024-06-03T01:18:44.173 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:43 smithi032 ceph-mon[29360]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-03T01:18:44.173 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:43 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1211885160' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:18:44.173 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T01:18:44.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T01:18:44.195 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-03T01:18:44.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:44.195 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42275sec preferred_lft 42275sec 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-06-03T01:18:44.214 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T01:18:44.214 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-06-03T01:18:44.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:43 smithi044 ceph-mon[36956]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-03T01:18:44.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:43 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1211885160' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:18:44.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:44.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:44.487 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T01:18:44.514 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-06-03T01:18:44.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:44.514 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-06-03T01:18:44.588 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-06-03T01:18:44.588 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T01:18:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-03T01:18:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-03T01:18:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T01:18:44.589 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:44.589 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-06-03T01:18:44.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-03T01:18:44.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-03T01:18:44.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-03T01:18:44.590 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-06-03T01:18:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:44.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:44.942 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-06-03T01:18:44.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:44.960 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-06-03T01:18:45.015 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:45.015 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-06-03T01:18:45.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T01:18:45.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-03T01:18:45.015 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-06-03T01:18:45.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:45.136 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-03T01:18:45.136 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:45.136 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-06-03T01:18:45.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-03T01:18:45.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T01:18:45.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-03T01:18:45.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-03T01:18:45.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-03T01:18:45.137 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-06-03T01:18:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:45.264 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:45.264 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-06-03T01:18:45.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-06-03T01:18:45.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T01:18:45.264 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-06-03T01:18:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:45.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:45.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:18:45.348 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:45.364 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-03T01:18:45.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:18:45.364 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:45.420 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-06-03T01:18:45.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:45.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-06-03T01:18:45.529 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-06-03T01:18:45.529 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-06-03T01:18:45.529 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-06-03T01:18:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-06-03T01:18:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-06-03T01:18:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-06-03T01:18:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.44:3300" 2024-06-03T01:18:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.44:3300" 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.44:6789" 2024-06-03T01:18:45.676 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.44:6789:/ 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=12a144f0-2146-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-06-03T01:18:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-03T01:18:45.679 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-03T01:18:45.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:45.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:45.709 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:45.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:45.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:45.816 INFO:teuthology.orchestra.run.smithi032.stdout:client24309 2024-06-03T01:18:45.817 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:18:45.817 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:18:45.835 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:45.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:45.942 INFO:teuthology.orchestra.run.smithi032.stdout:client24309 2024-06-03T01:18:45.943 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:46.006 INFO:teuthology.orchestra.run.smithi032.stdout:client24309 2024-06-03T01:18:46.006 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/12a144f0-2146-11ef-bc9b-c7b262605968.client24309/status') 2024-06-03T01:18:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:46.070 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.24309 (3)172.21.15.32:0/1770476556 2024-06-03T01:18:46.070 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-06-03T01:18:46.070 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-06-03T01:18:46.070 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-06-03T01:18:46.070 INFO:teuthology.orchestra.run.smithi032.stderr:71 bytes copied, 2.9395e-05 s, 2.4 MB/s 2024-06-03T01:18:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:46.077 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-06-03T01:18:46.078 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T01:18:46.078 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:46.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:45 smithi032 ceph-mon[29360]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-03T01:18:46.158 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.158 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.158 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:46.158 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:46.158 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:18:46.159 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:45 smithi044 ceph-mon[36956]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-03T01:18:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 -- 172.21.15.32:0/2636911237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100ea0 msgr2=0x7ff17c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 --2- 172.21.15.32:0/2636911237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100ea0 0x7ff17c101280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009910 tx=0x7ff16c009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 -- 172.21.15.32:0/2636911237 shutdown_connections 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 --2- 172.21.15.32:0/2636911237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101850 0x7ff17c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 --2- 172.21.15.32:0/2636911237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100ea0 0x7ff17c101280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.801+0000 7ff182774700 1 -- 172.21.15.32:0/2636911237 >> 172.21.15.32:0/2636911237 conn(0x7ff17c0fc750 msgr2=0x7ff17c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.802+0000 7ff182774700 1 -- 172.21.15.32:0/2636911237 shutdown_connections 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.802+0000 7ff182774700 1 -- 172.21.15.32:0/2636911237 wait complete. 2024-06-03T01:18:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.802+0000 7ff182774700 1 Processor -- start 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.802+0000 7ff182774700 1 -- start start 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c100ea0 0x7ff17c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c197560 con 0x7ff17c101850 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19b370 con 0x7ff17c100ea0 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38390/0 (socket says 172.21.15.32:38390) 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 -- 172.21.15.32:0/1071613221 learned_addr learned my addr 172.21.15.32:0/1071613221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17bfff700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c100ea0 0x7ff17c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 -- 172.21.15.32:0/1071613221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c100ea0 msgr2=0x7ff17c196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c100ea0 0x7ff17c196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 -- 172.21.15.32:0/1071613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0095f0 con 0x7ff17c101850 2024-06-03T01:18:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff17b7fe700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b480 tx=0x7ff17000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff170011800 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff170011e10 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c19b650 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff170010ea0 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.803+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c19bba0 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.805+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c109240 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.805+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff170010380 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.805+0000 7ff1797fa700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 0x7ff16406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.806+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff17008b400 con 0x7ff17c101850 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.806+0000 7ff17bfff700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 0x7ff16406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.806+0000 7ff17bfff700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 0x7ff16406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006010 tx=0x7ff16c01a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:47.810+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff170059610 con 0x7ff17c101850 2024-06-03T01:18:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.055+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff17c04f2e0 con 0x7ff17c101850 2024-06-03T01:18:48.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.056+0000 7ff1797fa700 1 -- 172.21.15.32:0/1071613221 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff1700591a0 con 0x7ff17c101850 2024-06-03T01:18:48.057 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 msgr2=0x7ff16406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 0x7ff16406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006010 tx=0x7ff16c01a040 comp rx=0 tx=0).stop 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 msgr2=0x7ff17c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b480 tx=0x7ff17000b790 comp rx=0 tx=0).stop 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 shutdown_connections 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff16406c4e0 0x7ff16406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c100ea0 0x7ff17c196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 --2- 172.21.15.32:0/1071613221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c101850 0x7ff17c196ed0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 >> 172.21.15.32:0/1071613221 conn(0x7ff17c0fc750 msgr2=0x7ff17c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 shutdown_connections 2024-06-03T01:18:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:48.058+0000 7ff182774700 1 -- 172.21.15.32:0/1071613221 wait complete. 2024-06-03T01:18:48.070 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:47 smithi032 ceph-mon[29360]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-06-03T01:18:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:47 smithi044 ceph-mon[36956]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-06-03T01:18:48.622 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-03T01:18:48.622 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T01:18:48.623 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-06-03T01:18:48.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:48.623 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:48.644 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42281sec preferred_lft 42281sec 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:18:48.645 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-06-03T01:18:48.646 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-06-03T01:18:48.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T01:18:48.647 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-06-03T01:18:48.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:48.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:48.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:48.877 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T01:18:48.903 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-06-03T01:18:48.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:48.905 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-06-03T01:18:48.986 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-06-03T01:18:48.987 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T01:18:48.987 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-03T01:18:48.987 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-03T01:18:48.987 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T01:18:48.987 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:48.987 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-06-03T01:18:48.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-03T01:18:48.988 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-03T01:18:48.988 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-03T01:18:48.988 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-06-03T01:18:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:49.085 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:48 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1071613221' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:18:49.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:49.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:49.276 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-06-03T01:18:49.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:49.293 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-06-03T01:18:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:48 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1071613221' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:18:49.349 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:49.349 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-06-03T01:18:49.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-06-03T01:18:49.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-06-03T01:18:49.349 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-06-03T01:18:49.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:49.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:49.471 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-06-03T01:18:49.471 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:49.471 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-06-03T01:18:49.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-06-03T01:18:49.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T01:18:49.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-06-03T01:18:49.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-06-03T01:18:49.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-06-03T01:18:49.472 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-06-03T01:18:49.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:49.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:49.625 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:18:49.625 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-06-03T01:18:49.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-06-03T01:18:49.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T01:18:49.625 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-06-03T01:18:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:49.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:49.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:18:49.707 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:49.723 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-06-03T01:18:49.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:18:49.723 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:49.779 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-06-03T01:18:49.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:49.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-06-03T01:18:49.889 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-06-03T01:18:49.890 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-06-03T01:18:49.890 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-06-03T01:18:50.016 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-06-03T01:18:50.016 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-06-03T01:18:50.017 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-06-03T01:18:50.017 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.44:3300" 2024-06-03T01:18:50.017 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-06-03T01:18:50.025 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-06-03T01:18:50.025 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-06-03T01:18:50.025 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-06-03T01:18:50.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.44:3300" 2024-06-03T01:18:50.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-06-03T01:18:50.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.44:6789" 2024-06-03T01:18:50.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.44:6789:/ 2024-06-03T01:18:50.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=12a144f0-2146-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-03T01:18:50.027 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-06-03T01:18:50.027 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-06-03T01:18:50.027 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-03T01:18:50.027 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-06-03T01:18:50.027 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-03T01:18:50.028 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-03T01:18:50.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:18:50.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:50.054 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:50.153 INFO:teuthology.orchestra.run.smithi044.stdout:client14530 2024-06-03T01:18:50.153 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:18:50.153 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:18:50.170 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:49 smithi044 ceph-mon[36956]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 426 B/s wr, 4 op/s 2024-06-03T01:18:50.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:50.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:50.269 INFO:teuthology.orchestra.run.smithi044.stdout:client14530 2024-06-03T01:18:50.270 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:18:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:49 smithi032 ceph-mon[29360]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 426 B/s wr, 4 op/s 2024-06-03T01:18:50.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:18:50.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:50.368 INFO:teuthology.orchestra.run.smithi044.stdout:client14530 2024-06-03T01:18:50.368 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/12a144f0-2146-11ef-bc9b-c7b262605968.client14530/status') 2024-06-03T01:18:50.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:18:50.428 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.14530 (3)172.21.15.44:0/1127472913 2024-06-03T01:18:50.429 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-06-03T01:18:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-06-03T01:18:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-06-03T01:18:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:71 bytes copied, 2.4713e-05 s, 2.9 MB/s 2024-06-03T01:18:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-06-03T01:18:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:18:50.436 INFO:teuthology.run_tasks:Running task print... 2024-06-03T01:18:50.448 INFO:teuthology.task.print:**** done client 2024-06-03T01:18:50.448 INFO:teuthology.run_tasks:Running task parallel... 2024-06-03T01:18:50.467 INFO:teuthology.task.parallel:starting parallel... 2024-06-03T01:18:50.467 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-03T01:18:50.468 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-03T01:18:50.468 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:18:50.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-03T01:18:50.469 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-03T01:18:50.469 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-06-03T01:18:50.478 INFO:tasks.workunit:Pulling workunits from ref dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T01:18:50.479 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-03T01:18:50.479 INFO:tasks.workunit:timeout=3h 2024-06-03T01:18:50.479 INFO:tasks.workunit:cleanup=True 2024-06-03T01:18:50.479 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-03T01:18:50.501 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 98h/152d Inode: 1 Links: 2 2024-06-03T01:18:50.501 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:18:50.501 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-03T01:18:50.502 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-06-03T01:18:50.502 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-06-03 01:18:16.695443572 +0000 2024-06-03T01:18:50.502 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-06-03 01:18:45.703038953 +0000 2024-06-03T01:18:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-06-03 01:18:16.695443572 +0000 2024-06-03T01:18:50.502 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-03T01:18:50.502 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-03T01:18:50.574 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 80h/128d Inode: 1 Links: 3 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-03T01:18:50.593 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-06-03T01:18:50.594 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-06-03 01:18:50.567941540 +0000 2024-06-03T01:18:50.594 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-06-03 01:18:50.567941540 +0000 2024-06-03T01:18:50.594 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-06-03 01:18:16.695443572 +0000 2024-06-03T01:18:50.594 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-06-03T01:18:50.594 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-06-03T01:18:50.666 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T01:18:50.667 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T01:18:50.686 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-03T01:18:50.712 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:50.722 INFO:tasks.workunit.client.1.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-06-03T01:18:52.052 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:51 smithi032 ceph-mon[29360]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:18:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:51 smithi044 ceph-mon[36956]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:18:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 -- 172.21.15.32:0/3689498046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 msgr2=0x7f73901064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 --2- 172.21.15.32:0/3689498046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f73901064c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7380009910 tx=0x7f7380009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 -- 172.21.15.32:0/3689498046 shutdown_connections 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 --2- 172.21.15.32:0/3689498046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f73901064c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 --2- 172.21.15.32:0/3689498046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 0x7f7390103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 -- 172.21.15.32:0/3689498046 >> 172.21.15.32:0/3689498046 conn(0x7f73900fb190 msgr2=0x7f73900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 -- 172.21.15.32:0/3689498046 shutdown_connections 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 -- 172.21.15.32:0/3689498046 wait complete. 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.261+0000 7f7398423700 1 Processor -- start 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f7398423700 1 -- start start 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f7398423700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 0x7f7390196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f7398423700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f7398423700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7390196fb0 con 0x7f73901040d0 2024-06-03T01:18:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f7398423700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73901970f0 con 0x7f73901017a0 2024-06-03T01:18:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38412/0 (socket says 172.21.15.32:38412) 2024-06-03T01:18:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 -- 172.21.15.32:0/1066307811 learned_addr learned my addr 172.21.15.32:0/1066307811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73961bf700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 0x7f7390196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 -- 172.21.15.32:0/1066307811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 msgr2=0x7f7390196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 0x7f7390196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 -- 172.21.15.32:0/1066307811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800095f0 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73959be700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7380006010 tx=0x7f738000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.262+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738001d070 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.263+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f738000f460 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.263+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739019bb40 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.263+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7380021640 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.263+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739019c030 con 0x7f73901040d0 2024-06-03T01:18:52.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.264+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7390190610 con 0x7f73901040d0 2024-06-03T01:18:52.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.264+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f738000fab0 con 0x7f73901040d0 2024-06-03T01:18:52.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.265+0000 7f73877fe700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 0x7f737c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:52.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.265+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f738008cca0 con 0x7f73901040d0 2024-06-03T01:18:52.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.265+0000 7f73961bf700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 0x7f737c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:52.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.265+0000 7f73961bf700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 0x7f737c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f738c0060b0 tx=0x7f738c007b80 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:52.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.269+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7380091070 con 0x7f73901040d0 2024-06-03T01:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.470+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f739002cc70 con 0x7f73901040d0 2024-06-03T01:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.471+0000 7f73877fe700 1 -- 172.21.15.32:0/1066307811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f738005aeb0 con 0x7f73901040d0 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 msgr2=0x7f737c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 0x7f737c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f738c0060b0 tx=0x7f738c007b80 comp rx=0 tx=0).stop 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 msgr2=0x7f7390196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7380006010 tx=0x7f738000be20 comp rx=0 tx=0).stop 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 shutdown_connections 2024-06-03T01:18:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f737c06c4e0 0x7f737c06e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73901017a0 0x7f7390196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 --2- 172.21.15.32:0/1066307811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73901040d0 0x7f7390196990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 >> 172.21.15.32:0/1066307811 conn(0x7f73900fb190 msgr2=0x7f73900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 shutdown_connections 2024-06-03T01:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:52.473+0000 7f7398423700 1 -- 172.21.15.32:0/1066307811 wait complete. 2024-06-03T01:18:53.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-03T01:18:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:54.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:53 smithi044 ceph-mon[36956]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:18:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:53 smithi032 ceph-mon[29360]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:18:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- 172.21.15.32:0/804632188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b00ffc10 msgr2=0x7fe0b0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 --2- 172.21.15.32:0/804632188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b00ffc10 0x7fe0b0100030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009910 tx=0x7fe0a0009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- 172.21.15.32:0/804632188 shutdown_connections 2024-06-03T01:18:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 --2- 172.21.15.32:0/804632188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b0100e10 0x7fe0b0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 --2- 172.21.15.32:0/804632188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b00ffc10 0x7fe0b0100030 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- 172.21.15.32:0/804632188 >> 172.21.15.32:0/804632188 conn(0x7fe0b00fb190 msgr2=0x7fe0b00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- 172.21.15.32:0/804632188 shutdown_connections 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- 172.21.15.32:0/804632188 wait complete. 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 Processor -- start 2024-06-03T01:18:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.913+0000 7fe0b655a700 1 -- start start 2024-06-03T01:18:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0b655a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 0x7fe0b01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0b655a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0b655a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0199230 con 0x7fe0b0100e10 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0b655a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0199370 con 0x7fe0b00ffc10 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38428/0 (socket says 172.21.15.32:38428) 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 -- 172.21.15.32:0/3008414022 learned_addr learned my addr 172.21.15.32:0/3008414022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0affff700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 0x7fe0b01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 -- 172.21.15.32:0/3008414022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 msgr2=0x7fe0b01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 0x7fe0b01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 -- 172.21.15.32:0/3008414022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00095f0 con 0x7fe0b0100e10 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0affff700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 0x7fe0b01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0af7fe700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400d8d0 tx=0x7fe0a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.914+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a4009940 con 0x7fe0b0100e10 2024-06-03T01:18:54.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.915+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4010460 con 0x7fe0b0100e10 2024-06-03T01:18:54.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.915+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b019de20 con 0x7fe0b0100e10 2024-06-03T01:18:54.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.915+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a400f5f0 con 0x7fe0b0100e10 2024-06-03T01:18:54.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.915+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b019e340 con 0x7fe0b0100e10 2024-06-03T01:18:54.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.916+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b0108700 con 0x7fe0b0100e10 2024-06-03T01:18:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.916+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe0a40105d0 con 0x7fe0b0100e10 2024-06-03T01:18:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.917+0000 7fe0ad7fa700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 0x7fe09806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.917+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe0a408b890 con 0x7fe0b0100e10 2024-06-03T01:18:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.917+0000 7fe0affff700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 0x7fe09806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.917+0000 7fe0affff700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 0x7fe09806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000b5c0 tx=0x7fe0a0005e20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:54.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:54.921+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe0a4059aa0 con 0x7fe0b0100e10 2024-06-03T01:18:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.120+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fe0b01088f0 con 0x7fe0b0100e10 2024-06-03T01:18:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.121+0000 7fe0ad7fa700 1 -- 172.21.15.32:0/3008414022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fe0a4059630 con 0x7fe0b0100e10 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 msgr2=0x7fe09806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 0x7fe09806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000b5c0 tx=0x7fe0a0005e20 comp rx=0 tx=0).stop 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 msgr2=0x7fe0b0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400d8d0 tx=0x7fe0a400dc90 comp rx=0 tx=0).stop 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 shutdown_connections 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe09806c4e0 0x7fe09806e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0b00ffc10 0x7fe0b01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 --2- 172.21.15.32:0/3008414022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b0100e10 0x7fe0b0198c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 >> 172.21.15.32:0/3008414022 conn(0x7fe0b00fb190 msgr2=0x7fe0b0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:55.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 shutdown_connections 2024-06-03T01:18:55.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:55.123+0000 7fe0b655a700 1 -- 172.21.15.32:0/3008414022 wait complete. 2024-06-03T01:18:55.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-03T01:18:55.934 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:55.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:55 smithi044 ceph-mon[36956]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-06-03T01:18:56.057 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:55 smithi032 ceph-mon[29360]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 -- 172.21.15.32:0/3615913719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100cb0 msgr2=0x7f2a801010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 --2- 172.21.15.32:0/3615913719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100cb0 0x7f2a801010d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009910 tx=0x7f2a70009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 -- 172.21.15.32:0/3615913719 shutdown_connections 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 --2- 172.21.15.32:0/3615913719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80101eb0 0x7f2a80102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 --2- 172.21.15.32:0/3615913719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100cb0 0x7f2a801010d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 -- 172.21.15.32:0/3615913719 >> 172.21.15.32:0/3615913719 conn(0x7f2a800fc230 msgr2=0x7f2a800fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 -- 172.21.15.32:0/3615913719 shutdown_connections 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.394+0000 7f2a84c73700 1 -- 172.21.15.32:0/3615913719 wait complete. 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 Processor -- start 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 -- start start 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80100cb0 0x7f2a80074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800736c0 con 0x7f2a80101eb0 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a84c73700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80073800 con 0x7f2a80100cb0 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57384/0 (socket says 172.21.15.32:57384) 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 -- 172.21.15.32:0/2498007088 learned_addr learned my addr 172.21.15.32:0/2498007088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7e59c700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80100cb0 0x7f2a80074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 -- 172.21.15.32:0/2498007088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80100cb0 msgr2=0x7f2a80074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80100cb0 0x7f2a80074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.395+0000 7f2a7dd9b700 1 -- 172.21.15.32:0/2498007088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700095f0 con 0x7f2a80101eb0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a7dd9b700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d8d0 tx=0x7f2a7400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a740098a0 con 0x7f2a80101eb0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a80073ae0 con 0x7f2a80101eb0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2a74010460 con 0x7f2a80101eb0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7400f5f0 con 0x7f2a80101eb0 2024-06-03T01:18:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.396+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8019e520 con 0x7f2a80101eb0 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.397+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a80109810 con 0x7f2a80101eb0 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.397+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2a74009a00 con 0x7f2a80101eb0 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.398+0000 7f2a6f7fe700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 0x7f2a6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.398+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2a7408b770 con 0x7f2a80101eb0 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.398+0000 7f2a7e59c700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 0x7f2a6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.398+0000 7f2a7e59c700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 0x7f2a6806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a7000b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.402+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2a74059980 con 0x7f2a80101eb0 2024-06-03T01:18:57.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.601+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f2a801099d0 con 0x7f2a80101eb0 2024-06-03T01:18:57.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.601+0000 7f2a6f7fe700 1 -- 172.21.15.32:0/2498007088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f2a801099d0 con 0x7f2a80101eb0 2024-06-03T01:18:57.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 msgr2=0x7f2a6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:57.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 0x7f2a6806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a7000b540 comp rx=0 tx=0).stop 2024-06-03T01:18:57.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 msgr2=0x7f2a80073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:57.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d8d0 tx=0x7f2a7400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 shutdown_connections 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f2a6806c4e0 0x7f2a6806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80100cb0 0x7f2a80074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 --2- 172.21.15.32:0/2498007088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80101eb0 0x7f2a80073180 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 >> 172.21.15.32:0/2498007088 conn(0x7f2a800fc230 msgr2=0x7f2a801050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 shutdown_connections 2024-06-03T01:18:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:57.603+0000 7f2a84c73700 1 -- 172.21.15.32:0/2498007088 wait complete. 2024-06-03T01:18:58.171 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:57 smithi032 ceph-mon[29360]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-06-03T01:18:58.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-06-03T01:18:58.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:57 smithi044 ceph-mon[36956]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-06-03T01:18:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:18:59.179 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:58 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:58 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:18:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 -- 172.21.15.32:0/4157420163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4580100dc0 msgr2=0x7f4580101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 --2- 172.21.15.32:0/4157420163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4580100dc0 0x7f4580101220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4574009910 tx=0x7f4574009c20 comp rx=0 tx=0).stop 2024-06-03T01:18:59.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 -- 172.21.15.32:0/4157420163 shutdown_connections 2024-06-03T01:18:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 --2- 172.21.15.32:0/4157420163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4580100dc0 0x7f4580101220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 --2- 172.21.15.32:0/4157420163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45800ffbc0 0x7f45800fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.977+0000 7f458689d700 1 -- 172.21.15.32:0/4157420163 >> 172.21.15.32:0/4157420163 conn(0x7f45800fb160 msgr2=0x7f45800fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.978+0000 7f458689d700 1 -- 172.21.15.32:0/4157420163 shutdown_connections 2024-06-03T01:18:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.978+0000 7f458689d700 1 -- 172.21.15.32:0/4157420163 wait complete. 2024-06-03T01:18:59.979 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:18:59 smithi032 ceph-mon[29360]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-06-03T01:18:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.978+0000 7f458689d700 1 Processor -- start 2024-06-03T01:18:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.978+0000 7f458689d700 1 -- start start 2024-06-03T01:18:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f458689d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f458689d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 0x7f4580196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f458689d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4580197330 con 0x7f45800ffbc0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f458689d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4580197470 con 0x7f4580100dc0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 -- 172.21.15.32:0/1018992379 learned_addr learned my addr 172.21.15.32:0/1018992379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457f7fe700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 0x7f4580196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 -- 172.21.15.32:0/1018992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 msgr2=0x7f4580196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 0x7f4580196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 -- 172.21.15.32:0/1018992379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45740095f0 con 0x7f45800ffbc0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457f7fe700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 0x7f4580196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457ffff700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f457000d900 tx=0x7f457000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.979+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45700041f0 con 0x7f45800ffbc0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.980+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458019bf20 con 0x7f45800ffbc0 2024-06-03T01:18:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.980+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4570004350 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.980+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4570003da0 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.980+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458019c3e0 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.981+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4580068af0 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.981+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4570009730 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.982+0000 7f457d7fa700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 0x7f456806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.982+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f457008b8d0 con 0x7f45800ffbc0 2024-06-03T01:18:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.982+0000 7f457f7fe700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 0x7f456806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.982+0000 7f457f7fe700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 0x7f456806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4574006010 tx=0x7f457400b560 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:18:59.986+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4570059b60 con 0x7f45800ffbc0 2024-06-03T01:19:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:00.202+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f4580069560 con 0x7f456806c4e0 2024-06-03T01:19:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:18:59 smithi044 ceph-mon[36956]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-06-03T01:19:02.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:01 smithi044 ceph-mon[36956]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:02.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:01 smithi044 ceph-mon[36956]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-03T01:19:02.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:01 smithi032 ceph-mon[29360]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:02.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:01 smithi032 ceph-mon[29360]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-03T01:19:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:03 smithi044 ceph-mon[36956]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-03T01:19:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:03 smithi032 ceph-mon[29360]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-03T01:19:05.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:05 smithi044 ceph-mon[36956]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-06-03T01:19:06.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:05 smithi032 ceph-mon[29360]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-06-03T01:19:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:07 smithi044 ceph-mon[36956]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:07 smithi032 ceph-mon[29360]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:10.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:09 smithi044 ceph-mon[36956]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:09 smithi032 ceph-mon[29360]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:11 smithi032 ceph-mon[29360]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:11 smithi044 ceph-mon[36956]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:13 smithi044 ceph-mon[36956]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:13 smithi032 ceph-mon[29360]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:15 smithi044 ceph-mon[36956]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:16.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:15 smithi032 ceph-mon[29360]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:17 smithi044 ceph-mon[36956]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:17 smithi032 ceph-mon[29360]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:20.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:19 smithi044 ceph-mon[36956]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:20.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:19 smithi032 ceph-mon[29360]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:21 smithi044 ceph-mon[36956]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:21 smithi032 ceph-mon[29360]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:23 smithi044 ceph-mon[36956]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:23 smithi032 ceph-mon[29360]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:25.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:25 smithi044 ceph-mon[36956]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:25 smithi032 ceph-mon[29360]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:27 smithi044 ceph-mon[36956]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:28.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:27 smithi032 ceph-mon[29360]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:29 smithi032 ceph-mon[29360]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:19:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:29 smithi044 ceph-mon[36956]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:19:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:31 smithi032 ceph-mon[29360]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:19:32.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:31 smithi044 ceph-mon[36956]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:19:34.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:33 smithi032 ceph-mon[29360]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:19:34.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:33 smithi044 ceph-mon[36956]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:19:35.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:35 smithi044 ceph-mon[36956]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:19:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:35 smithi032 ceph-mon[29360]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:19:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:37 smithi044 ceph-mon[36956]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-03T01:19:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:37 smithi032 ceph-mon[29360]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-03T01:19:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:39 smithi044 ceph-mon[36956]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-03T01:19:40.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:39 smithi032 ceph-mon[29360]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-03T01:19:42.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:41 smithi032 ceph-mon[29360]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:41 smithi044 ceph-mon[36956]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:44.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:43 smithi032 ceph-mon[29360]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:44.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:43 smithi044 ceph-mon[36956]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:19:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:19:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:45 smithi044 ceph-mon[36956]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:19:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:45 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:45 smithi032 ceph-mon[29360]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:19:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:45 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:47 smithi044 ceph-mon[36956]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:19:48.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:47 smithi032 ceph-mon[29360]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:19:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:48 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:49.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:48 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:49.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:48 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:49.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:48 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:50.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:49 smithi032 ceph-mon[29360]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:50.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:49 smithi044 ceph-mon[36956]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:51 smithi044 ceph-mon[36956]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:51 smithi032 ceph-mon[29360]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:52.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.708+0000 7f457d7fa700 1 -- 172.21.15.32:0/1018992379 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f4580069560 con 0x7f456806c4e0 2024-06-03T01:19:52.708 INFO:teuthology.orchestra.run.smithi032.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:19:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.710+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 msgr2=0x7f456806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.710+0000 7f458689d700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 0x7f456806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4574006010 tx=0x7f457400b560 comp rx=0 tx=0).stop 2024-06-03T01:19:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.710+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 msgr2=0x7f45801967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.710+0000 7f458689d700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f457000d900 tx=0x7f457000dcc0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 shutdown_connections 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45800ffbc0 0x7f45801967d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f456806c4e0 0x7f456806e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 --2- 172.21.15.32:0/1018992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4580100dc0 0x7f4580196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 >> 172.21.15.32:0/1018992379 conn(0x7f45800fb160 msgr2=0x7f4580068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 shutdown_connections 2024-06-03T01:19:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:52.711+0000 7f458689d700 1 -- 172.21.15.32:0/1018992379 wait complete. 2024-06-03T01:19:53.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-06-03T01:19:53.938 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:19:53.938 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:53.938 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:53.938 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:53.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:53.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:53.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:19:53.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:53 smithi032 ceph-mon[29360]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:53.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:19:53.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:19:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:53 smithi044 ceph-mon[36956]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:54.159 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: Upgrade: Target is version 19.0.0-2946-g3d89accf (unknown) 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:55.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: Upgrade: Need to upgrade myself (mgr.smithi032.kfcfaa) 2024-06-03T01:19:55.960 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:55 smithi032 ceph-mon[29360]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:55.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: Upgrade: Target is version 19.0.0-2946-g3d89accf (unknown) 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: Upgrade: Need to upgrade myself (mgr.smithi032.kfcfaa) 2024-06-03T01:19:55.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:55 smithi044 ceph-mon[36956]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 -- 172.21.15.32:0/3185204968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40ffc10 msgr2=0x7f4de4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 --2- 172.21.15.32:0/3185204968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40ffc10 0x7f4de4100030 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009910 tx=0x7f4dd4009c20 comp rx=0 tx=0).stop 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 -- 172.21.15.32:0/3185204968 shutdown_connections 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 --2- 172.21.15.32:0/3185204968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 0x7f4de4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 --2- 172.21.15.32:0/3185204968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40ffc10 0x7f4de4100030 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.268+0000 7f4dea290700 1 -- 172.21.15.32:0/3185204968 >> 172.21.15.32:0/3185204968 conn(0x7f4de40fb190 msgr2=0x7f4de40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.269+0000 7f4dea290700 1 -- 172.21.15.32:0/3185204968 shutdown_connections 2024-06-03T01:19:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.269+0000 7f4dea290700 1 -- 172.21.15.32:0/3185204968 wait complete. 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.270+0000 7f4dea290700 1 Processor -- start 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.270+0000 7f4dea290700 1 -- start start 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4dea290700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 0x7f4de41966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4dea290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4dea290700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de4197140 con 0x7f4de4196c30 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4dea290700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de41972b0 con 0x7f4de4100e10 2024-06-03T01:19:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4de3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 0x7f4de41966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4ddbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4ddbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51600/0 (socket says 172.21.15.32:51600) 2024-06-03T01:19:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.271+0000 7f4ddbfff700 1 -- 172.21.15.32:0/1650943921 learned_addr learned my addr 172.21.15.32:0/1650943921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.272+0000 7f4ddbfff700 1 -- 172.21.15.32:0/1650943921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 msgr2=0x7f4de41966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.272+0000 7f4ddbfff700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 0x7f4de41966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.272+0000 7f4ddbfff700 1 -- 172.21.15.32:0/1650943921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40095f0 con 0x7f4de4196c30 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.272+0000 7f4ddbfff700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00ece0 tx=0x7f4dcc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.272+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dcc00cab0 con 0x7f4de4196c30 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.273+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4dcc00cc10 con 0x7f4de4196c30 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.273+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dcc018650 con 0x7f4de4196c30 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.273+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de419c240 con 0x7f4de4196c30 2024-06-03T01:19:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.273+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de419c6d0 con 0x7f4de4196c30 2024-06-03T01:19:56.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.274+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de4108700 con 0x7f4de4196c30 2024-06-03T01:19:56.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.275+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4dcc010a70 con 0x7f4de4196c30 2024-06-03T01:19:56.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.275+0000 7f4de1ffb700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 0x7f4dd006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.275+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4dcc014070 con 0x7f4de4196c30 2024-06-03T01:19:56.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.276+0000 7f4de3fff700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 0x7f4dd006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.276+0000 7f4de3fff700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 0x7f4dd006e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4006010 tx=0x7f4dd4009f90 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.281+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4dcc0520e0 con 0x7f4de4196c30 2024-06-03T01:19:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.530+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4de40fc5f0 con 0x7f4dd006c4e0 2024-06-03T01:19:56.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.532+0000 7f4de1ffb700 1 -- 172.21.15.32:0/1650943921 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f4de40fc5f0 con 0x7f4dd006c4e0 2024-06-03T01:19:56.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 msgr2=0x7f4dd006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 0x7f4dd006e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4006010 tx=0x7f4dd4009f90 comp rx=0 tx=0).stop 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 msgr2=0x7f4de419bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00ece0 tx=0x7f4dcc009ea0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 shutdown_connections 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f4dd006c4e0 0x7f4dd006e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4de4100e10 0x7f4de41966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 --2- 172.21.15.32:0/1650943921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de4196c30 0x7f4de419bca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.534+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 >> 172.21.15.32:0/1650943921 conn(0x7f4de40fb190 msgr2=0x7f4de40fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.535+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 shutdown_connections 2024-06-03T01:19:56.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.535+0000 7f4dea290700 1 -- 172.21.15.32:0/1650943921 wait complete. 2024-06-03T01:19:56.548 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:19:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 -- 172.21.15.32:0/508042942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c100e10 msgr2=0x7fb89c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 --2- 172.21.15.32:0/508042942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c100e10 0x7fb89c101270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb890009910 tx=0x7fb890009c20 comp rx=0 tx=0).stop 2024-06-03T01:19:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 -- 172.21.15.32:0/508042942 shutdown_connections 2024-06-03T01:19:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 --2- 172.21.15.32:0/508042942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c100e10 0x7fb89c101270 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 --2- 172.21.15.32:0/508042942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c0ffc10 0x7fb89c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.679+0000 7fb8a1678700 1 -- 172.21.15.32:0/508042942 >> 172.21.15.32:0/508042942 conn(0x7fb89c0fb190 msgr2=0x7fb89c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.680+0000 7fb8a1678700 1 -- 172.21.15.32:0/508042942 shutdown_connections 2024-06-03T01:19:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.680+0000 7fb8a1678700 1 -- 172.21.15.32:0/508042942 wait complete. 2024-06-03T01:19:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 Processor -- start 2024-06-03T01:19:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 -- start start 2024-06-03T01:19:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 0x7fb89c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c195f50 con 0x7fb89c0ffc10 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb8a1678700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c196090 con 0x7fb89c100e10 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb89affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb89a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 0x7fb89c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb89affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51610/0 (socket says 172.21.15.32:51610) 2024-06-03T01:19:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.681+0000 7fb89affd700 1 -- 172.21.15.32:0/644992575 learned_addr learned my addr 172.21.15.32:0/644992575 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb89affd700 1 -- 172.21.15.32:0/644992575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 msgr2=0x7fb89c195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb89affd700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 0x7fb89c195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb89affd700 1 -- 172.21.15.32:0/644992575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900095f0 con 0x7fb89c0ffc10 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb89a7fc700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 0x7fb89c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb89affd700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00ea60 tx=0x7fb88c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c013070 con 0x7fb89c0ffc10 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb88c00b940 con 0x7fb89c0ffc10 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c0685e0 con 0x7fb89c0ffc10 2024-06-03T01:19:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c003d60 con 0x7fb89c0ffc10 2024-06-03T01:19:56.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.682+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c068b30 con 0x7fb89c0ffc10 2024-06-03T01:19:56.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.684+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c108720 con 0x7fb89c0ffc10 2024-06-03T01:19:56.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.685+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb88c004200 con 0x7fb89c0ffc10 2024-06-03T01:19:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.685+0000 7fb883fff700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 0x7fb88406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.686+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb88c08b0f0 con 0x7fb89c0ffc10 2024-06-03T01:19:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.686+0000 7fb89a7fc700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 0x7fb88406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.686+0000 7fb89a7fc700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 0x7fb88406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb890009fd0 tx=0x7fb890005940 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:56.694+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb88c059380 con 0x7fb89c0ffc10 2024-06-03T01:19:56.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:56 smithi032 ceph-mon[29360]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 on smithi044 2024-06-03T01:19:56.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:56 smithi044 ceph-mon[36956]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 on smithi044 2024-06-03T01:19:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.065+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb89c105760 con 0x7fb88406c4e0 2024-06-03T01:19:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.066+0000 7fb883fff700 1 -- 172.21.15.32:0/644992575 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb89c105760 con 0x7fb88406c4e0 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 msgr2=0x7fb88406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 0x7fb88406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb890009fd0 tx=0x7fb890005940 comp rx=0 tx=0).stop 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 msgr2=0x7fb89c1953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00ea60 tx=0x7fb88c00ed70 comp rx=0 tx=0).stop 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 shutdown_connections 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb89c0ffc10 0x7fb89c1953f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fb88406c4e0 0x7fb88406e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 --2- 172.21.15.32:0/644992575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb89c100e10 0x7fb89c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.068+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 >> 172.21.15.32:0/644992575 conn(0x7fb89c0fb190 msgr2=0x7fb89c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.069+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 shutdown_connections 2024-06-03T01:19:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.069+0000 7fb8a1678700 1 -- 172.21.15.32:0/644992575 wait complete. 2024-06-03T01:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/2132629117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 msgr2=0x7f7bd4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/2132629117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4101270 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009910 tx=0x7f7bc8009c20 comp rx=0 tx=0).stop 2024-06-03T01:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/2132629117 shutdown_connections 2024-06-03T01:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/2132629117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4101270 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/2132629117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/2132629117 >> 172.21.15.32:0/2132629117 conn(0x7f7bd40fb190 msgr2=0x7f7bd40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/2132629117 shutdown_connections 2024-06-03T01:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.230+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/2132629117 wait complete. 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 Processor -- start 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 -- start start 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd41991e0 con 0x7f7bd4100e10 2024-06-03T01:19:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.231+0000 7f7bd8b9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd4199320 con 0x7f7bd40ffc10 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:48860/0 (socket says 172.21.15.32:48860) 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd259c700 1 -- 172.21.15.32:0/1207065275 learned_addr learned my addr 172.21.15.32:0/1207065275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd1d9b700 1 -- 172.21.15.32:0/1207065275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 msgr2=0x7f7bd4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd1d9b700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd1d9b700 1 -- 172.21.15.32:0/1207065275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80095f0 con 0x7f7bd4100e10 2024-06-03T01:19:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.232+0000 7f7bd1d9b700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4198bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8000c00 tx=0x7f7bc800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.233+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc801d070 con 0x7f7bd4100e10 2024-06-03T01:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.233+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7bc8005dc0 con 0x7f7bd4100e10 2024-06-03T01:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.233+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd419dd70 con 0x7f7bd4100e10 2024-06-03T01:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.233+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc8017400 con 0x7f7bd4100e10 2024-06-03T01:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.233+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd419e260 con 0x7f7bd4100e10 2024-06-03T01:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.234+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd4108700 con 0x7f7bd4100e10 2024-06-03T01:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.235+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7bc8003810 con 0x7f7bd4100e10 2024-06-03T01:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.236+0000 7f7bc37fe700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 0x7f7bbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.236+0000 7f7bd259c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 0x7f7bbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.237+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7bc808cb30 con 0x7f7bd4100e10 2024-06-03T01:19:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.237+0000 7f7bd259c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 0x7f7bbc06e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4005950 tx=0x7f7bc4009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.241+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7bc805ade0 con 0x7f7bd4100e10 2024-06-03T01:19:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.625+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd4068c90 con 0x7f7bbc06c530 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.637+0000 7f7bc37fe700 1 -- 172.21.15.32:0/1207065275 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f7bd4068c90 con 0x7f7bbc06c530 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 72s ago 7m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (7m) 72s ago 7m 7490k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (6m) 77s ago 6m 7360k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (7m) 72s ago 7m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:19:57.637 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (6m) 77s ago 6m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 72s ago 6m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (101s) 72s ago 101s 16.2M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (92s) 72s ago 92s 14.8M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (98s) 77s ago 98s 11.5M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (90s) 77s ago 89s 14.8M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:9283,8765,8443 running (8m) 72s ago 8m 489M - 18.2.1 5be31c24972a 5db22291b59c 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (6m) 77s ago 6m 443M - 18.2.1 5be31c24972a 5fbe526898cf 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (8m) 72s ago 8m 44.6M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (6m) 77s ago 5m 35.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 72s ago 7m 14.2M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 77s ago 6m 16.3M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:19:57.638 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 72s ago 5m 46.8M 3948M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (4m) 72s ago 4m 46.7M 3948M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 72s ago 4m 49.5M 3948M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (3m) 77s ago 3m 47.0M 4972M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (3m) 77s ago 3m 46.2M 4972M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (3m) 77s ago 3m 46.1M 4972M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 72s ago 6m 57.9M - 2.43.0 a07b618ecd1d 55ba7950dc8e 2024-06-03T01:19:57.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 msgr2=0x7f7bbc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 0x7f7bbc06e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4005950 tx=0x7f7bc4009260 comp rx=0 tx=0).stop 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 msgr2=0x7f7bd4198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4198bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8000c00 tx=0x7f7bc800f690 comp rx=0 tx=0).stop 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 shutdown_connections 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f7bbc06c530 0x7f7bbc06e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd40ffc10 0x7f7bd4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 --2- 172.21.15.32:0/1207065275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd4100e10 0x7f7bd4198bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 >> 172.21.15.32:0/1207065275 conn(0x7f7bd40fb190 msgr2=0x7f7bd4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:57.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 shutdown_connections 2024-06-03T01:19:57.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.640+0000 7f7bd8b9c700 1 -- 172.21.15.32:0/1207065275 wait complete. 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.783+0000 7ff64cd66700 1 -- 172.21.15.32:0/1971589728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff648100e10 msgr2=0x7ff648101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.783+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1971589728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff648100e10 0x7ff648101270 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009910 tx=0x7ff63c009c20 comp rx=0 tx=0).stop 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 -- 172.21.15.32:0/1971589728 shutdown_connections 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1971589728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff648100e10 0x7ff648101270 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1971589728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6480ffc10 0x7ff648100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 -- 172.21.15.32:0/1971589728 >> 172.21.15.32:0/1971589728 conn(0x7ff6480fb190 msgr2=0x7ff6480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 -- 172.21.15.32:0/1971589728 shutdown_connections 2024-06-03T01:19:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.784+0000 7ff64cd66700 1 -- 172.21.15.32:0/1971589728 wait complete. 2024-06-03T01:19:57.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.785+0000 7ff64cd66700 1 Processor -- start 2024-06-03T01:19:57.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.785+0000 7ff64cd66700 1 -- start start 2024-06-03T01:19:57.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.785+0000 7ff64cd66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36902/0 (socket says 172.21.15.32:36902) 2024-06-03T01:19:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 -- 172.21.15.32:0/1625367628 learned_addr learned my addr 172.21.15.32:0/1625367628 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:19:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 0x7ff648194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648194da0 con 0x7ff6480ffc10 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648194ee0 con 0x7ff648100e10 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff645d9b700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 0x7ff648194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 -- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 msgr2=0x7ff648194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 0x7ff648194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0095f0 con 0x7ff6480ffc10 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff645d9b700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 0x7ff648194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.786+0000 7ff64659c700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b3c0 tx=0x7ff6380094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.787+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff638009c20 con 0x7ff6480ffc10 2024-06-03T01:19:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.787+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff648199990 con 0x7ff6480ffc10 2024-06-03T01:19:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.787+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff6380044d0 con 0x7ff6480ffc10 2024-06-03T01:19:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.787+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648199ee0 con 0x7ff6480ffc10 2024-06-03T01:19:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.787+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff638023440 con 0x7ff6480ffc10 2024-06-03T01:19:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.788+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6481075f0 con 0x7ff6480ffc10 2024-06-03T01:19:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.790+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff638004020 con 0x7ff6480ffc10 2024-06-03T01:19:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.790+0000 7ff6377fe700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 0x7ff63006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.791+0000 7ff645d9b700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 0x7ff63006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.792+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff63808f1e0 con 0x7ff6480ffc10 2024-06-03T01:19:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.792+0000 7ff645d9b700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 0x7ff63006e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c009f70 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:57.799+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff63805d3f0 con 0x7ff6480ffc10 2024-06-03T01:19:57.917 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:57 smithi032 ceph-mon[29360]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:57.917 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:57 smithi032 ceph-mon[29360]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:57 smithi044 ceph-mon[36956]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:57 smithi044 ceph-mon[36956]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:19:58.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.133+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff64819a630 con 0x7ff6480ffc10 2024-06-03T01:19:58.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.134+0000 7ff6377fe700 1 -- 172.21.15.32:0/1625367628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7ff63805cf80 con 0x7ff6480ffc10 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:19:58.135 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:19:58.136 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 msgr2=0x7ff63006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 0x7ff63006e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c009f70 comp rx=0 tx=0).stop 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 msgr2=0x7ff648194240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b3c0 tx=0x7ff6380094c0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 shutdown_connections 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6480ffc10 0x7ff648194240 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b3c0 tx=0x7ff6380094c0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7ff63006c530 0x7ff63006e9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 --2- 172.21.15.32:0/1625367628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff648100e10 0x7ff648194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.136+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 >> 172.21.15.32:0/1625367628 conn(0x7ff6480fb190 msgr2=0x7ff648104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.137+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 shutdown_connections 2024-06-03T01:19:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.137+0000 7ff64cd66700 1 -- 172.21.15.32:0/1625367628 wait complete. 2024-06-03T01:19:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 -- 172.21.15.32:0/345054767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80ffbb0 msgr2=0x7fc5a80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 --2- 172.21.15.32:0/345054767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a80fffd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009910 tx=0x7fc59c009c20 comp rx=0 tx=0).stop 2024-06-03T01:19:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 -- 172.21.15.32:0/345054767 shutdown_connections 2024-06-03T01:19:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 --2- 172.21.15.32:0/345054767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8100db0 0x7fc5a8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 --2- 172.21.15.32:0/345054767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a80fffd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 -- 172.21.15.32:0/345054767 >> 172.21.15.32:0/345054767 conn(0x7fc5a80fb150 msgr2=0x7fc5a80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 -- 172.21.15.32:0/345054767 shutdown_connections 2024-06-03T01:19:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.255+0000 7fc5aecda700 1 -- 172.21.15.32:0/345054767 wait complete. 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 Processor -- start 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 -- start start 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a8100db0 0x7fc5a8196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8197060 con 0x7fc5a8100db0 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.256+0000 7fc5aecda700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81971a0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:48888/0 (socket says 172.21.15.32:48888) 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 -- 172.21.15.32:0/1322722174 learned_addr learned my addr 172.21.15.32:0/1322722174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a8100db0 0x7fc5a8196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 -- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a8100db0 msgr2=0x7fc5a8196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a8100db0 0x7fc5a8196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.257+0000 7fc5aca76700 1 -- 172.21.15.32:0/1322722174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0095f0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5aca76700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc59c01d070 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc59c005dc0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a8101fd0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc59c0226a0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.258+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a81024c0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.260+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc59c017400 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.260+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8190550 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.261+0000 7fc5a5ffb700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 0x7fc59806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.261+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 0x7fc59806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.261+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc59c08df70 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.262+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 0x7fc59806e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc594009bf0 tx=0x7fc594009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:58.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.270+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc59c05c0e0 con 0x7fc5a80ffbb0 2024-06-03T01:19:58.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.495+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5a80611d0 con 0x7fc59806c4e0 2024-06-03T01:19:58.496 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "0/2 daemons upgraded", 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 image on host smithi044", 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:19:58.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.496+0000 7fc5a5ffb700 1 -- 172.21.15.32:0/1322722174 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc5a80611d0 con 0x7fc59806c4e0 2024-06-03T01:19:58.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 msgr2=0x7fc59806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 0x7fc59806e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc594009bf0 tx=0x7fc594009260 comp rx=0 tx=0).stop 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 msgr2=0x7fc5a8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c00f740 comp rx=0 tx=0).stop 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 shutdown_connections 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fc59806c4e0 0x7fc59806e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a80ffbb0 0x7fc5a8196500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 --2- 172.21.15.32:0/1322722174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a8100db0 0x7fc5a8196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:58.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 >> 172.21.15.32:0/1322722174 conn(0x7fc5a80fb150 msgr2=0x7fc5a80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:58.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 shutdown_connections 2024-06-03T01:19:58.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:19:58.499+0000 7fc5aecda700 1 -- 172.21.15.32:0/1322722174 wait complete. 2024-06-03T01:19:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:58 smithi044 ceph-mon[36956]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:58 smithi044 ceph-mon[36956]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:58 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1625367628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:19:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:58 smithi032 ceph-mon[29360]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:58 smithi032 ceph-mon[29360]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:19:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:58 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1625367628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:20:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:59 smithi032 ceph-mon[29360]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:59 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:19:59 smithi032 ceph-mon[29360]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:59 smithi044 ceph-mon[36956]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:59 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:19:59 smithi044 ceph-mon[36956]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:00 smithi032 ceph-mon[29360]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:20:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:00 smithi032 ceph-mon[29360]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:20:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:00 smithi032 ceph-mon[29360]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:20:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:00 smithi044 ceph-mon[36956]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:20:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:00 smithi044 ceph-mon[36956]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:20:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:00 smithi044 ceph-mon[36956]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:20:02.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:01 smithi032 ceph-mon[29360]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:02.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:01 smithi044 ceph-mon[36956]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:03 smithi032 ceph-mon[29360]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:03 smithi044 ceph-mon[36956]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:05.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:05 smithi044 ceph-mon[36956]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:06.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:05 smithi032 ceph-mon[29360]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:07 smithi032 ceph-mon[29360]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:08.421 INFO:tasks.workunit.client.1.smithi044.stderr:Updating files: 73% (9236/12565) Updating files: 74% (9299/12565) Updating files: 75% (9424/12565) Updating files: 76% (9550/12565) Updating files: 77% (9676/12565) Updating files: 78% (9801/12565) Updating files: 79% (9927/12565) Updating files: 80% (10052/12565) Updating files: 81% (10178/12565) Updating files: 82% (10304/12565) Updating files: 83% (10429/12565) Updating files: 84% (10555/12565) Updating files: 85% (10681/12565) Updating files: 86% (10806/12565) Updating files: 87% (10932/12565) Updating files: 88% (11058/12565) Updating files: 89% (11183/12565) Updating files: 90% (11309/12565) Updating files: 91% (11435/12565) Updating files: 92% (11560/12565) Updating files: 93% (11686/12565) Updating files: 94% (11812/12565) Updating files: 95% (11937/12565) Updating files: 96% (12063/12565) Updating files: 97% (12189/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-03T01:20:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:07 smithi044 ceph-mon[36956]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:Note: switching to 'dfcc4b3c2c55532c4c04fa47551c9df1dffdf746'. 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-03T01:20:09.151 INFO:tasks.workunit.client.1.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: git switch -c 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr:Or undo this operation with: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: git switch - 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-06-03T01:20:09.152 INFO:tasks.workunit.client.1.smithi044.stderr:HEAD is now at dfcc4b3c2c5 Merge pull request #57814 from zdover23/wip-doc-2024-06-01-backport-57804-to-squid 2024-06-03T01:20:09.156 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-06-03T01:20:09.222 INFO:tasks.workunit.client.1.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-03T01:20:09.224 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-06-03T01:20:09.224 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-03T01:20:10.093 INFO:tasks.workunit.client.0.smithi032.stderr:Updating files: 96% (12166/12565) Updating files: 97% (12189/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-03T01:20:10.244 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-03T01:20:10.317 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-03T01:20:10.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:09 smithi032 ceph-mon[29360]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:10.360 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-06-03T01:20:10.362 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-06-03T01:20:10.362 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to 'dfcc4b3c2c55532c4c04fa47551c9df1dffdf746'. 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-03T01:20:10.388 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-06-03T01:20:10.389 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at dfcc4b3c2c5 Merge pull request #57814 from zdover23/wip-doc-2024-06-01-backport-57804-to-squid 2024-06-03T01:20:10.393 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-03T01:20:10.406 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-06-03T01:20:10.409 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:20:10.409 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-06-03T01:20:10.429 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-03T01:20:10.431 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T01:20:10.431 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-03T01:20:10.467 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-06-03T01:20:10.468 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-06-03T01:20:10.468 DEBUG:teuthology.orchestra.run.smithi044:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-06-03T01:20:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:09 smithi044 ceph-mon[36956]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:10.494 INFO:tasks.workunit.client.1.smithi044.stderr:+ mkdir -p fsstress 2024-06-03T01:20:10.496 INFO:tasks.workunit.client.1.smithi044.stderr:+ pushd fsstress 2024-06-03T01:20:10.496 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-03T01:20:10.496 INFO:tasks.workunit.client.1.smithi044.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-06-03T01:20:11.118 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-03T01:20:11.174 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-03T01:20:11.215 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T01:20:11.217 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T01:20:11.217 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-03T01:20:11.257 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T01:20:11.258 INFO:tasks.workunit.client.1.smithi044.stderr:+ tar xzf ltp-full.tgz 2024-06-03T01:20:11.260 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:20:11.260 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-03T01:20:11.317 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-06-03T01:20:11.318 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-06-03T01:20:11.319 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-06-03T01:20:11.383 INFO:tasks.workunit.client.0.smithi032.stderr:+ mkdir -p fsstress 2024-06-03T01:20:11.385 INFO:tasks.workunit.client.0.smithi032.stderr:+ pushd fsstress 2024-06-03T01:20:11.385 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-03T01:20:11.385 INFO:tasks.workunit.client.0.smithi032.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-06-03T01:20:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:11 smithi032 ceph-mon[29360]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:11 smithi044 ceph-mon[36956]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:12.592 INFO:tasks.workunit.client.0.smithi032.stderr:+ tar xzf ltp-full.tgz 2024-06-03T01:20:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:13 smithi032 ceph-mon[29360]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:13 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:13 smithi044 ceph-mon[36956]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:20:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:13 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:15.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:15 smithi032 ceph-mon[29360]: pgmap v237: 65 pgs: 65 active+clean; 4.6 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-06-03T01:20:15.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:15 smithi044 ceph-mon[36956]: pgmap v237: 65 pgs: 65 active+clean; 4.6 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-06-03T01:20:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:17 smithi032 ceph-mon[29360]: pgmap v238: 65 pgs: 65 active+clean; 8.6 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s wr, 0 op/s 2024-06-03T01:20:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:17 smithi044 ceph-mon[36956]: pgmap v238: 65 pgs: 65 active+clean; 8.6 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s wr, 0 op/s 2024-06-03T01:20:20.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:19 smithi032 ceph-mon[29360]: pgmap v239: 65 pgs: 65 active+clean; 30 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 184 op/s 2024-06-03T01:20:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:19 smithi044 ceph-mon[36956]: pgmap v239: 65 pgs: 65 active+clean; 30 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 184 op/s 2024-06-03T01:20:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:21 smithi032 ceph-mon[29360]: pgmap v240: 65 pgs: 65 active+clean; 41 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 241 op/s 2024-06-03T01:20:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:21 smithi044 ceph-mon[36956]: pgmap v240: 65 pgs: 65 active+clean; 41 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 241 op/s 2024-06-03T01:20:23.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:22 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:20:23.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:22 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:20:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:23 smithi032 ceph-mon[29360]: pgmap v241: 65 pgs: 65 active+clean; 48 MiB data, 348 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 288 op/s 2024-06-03T01:20:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:23 smithi044 ceph-mon[36956]: pgmap v241: 65 pgs: 65 active+clean; 48 MiB data, 348 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 288 op/s 2024-06-03T01:20:25.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:25 smithi044 ceph-mon[36956]: pgmap v242: 65 pgs: 65 active+clean; 64 MiB data, 383 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s wr, 294 op/s 2024-06-03T01:20:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:25 smithi032 ceph-mon[29360]: pgmap v242: 65 pgs: 65 active+clean; 64 MiB data, 383 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s wr, 294 op/s 2024-06-03T01:20:28.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:27 smithi044 ceph-mon[36956]: pgmap v243: 65 pgs: 65 active+clean; 67 MiB data, 390 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 295 op/s 2024-06-03T01:20:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:20:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:20:28.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:27 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:27 smithi032 ceph-mon[29360]: pgmap v243: 65 pgs: 65 active+clean; 67 MiB data, 390 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 295 op/s 2024-06-03T01:20:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:20:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:20:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:27 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 -- 172.21.15.32:0/1411613931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 msgr2=0x7feb4c106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1411613931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c106470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009910 tx=0x7feb3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 -- 172.21.15.32:0/1411613931 shutdown_connections 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1411613931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c106470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1411613931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 -- 172.21.15.32:0/1411613931 >> 172.21.15.32:0/1411613931 conn(0x7feb4c0fb160 msgr2=0x7feb4c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 -- 172.21.15.32:0/1411613931 shutdown_connections 2024-06-03T01:20:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.656+0000 7feb53a9c700 1 -- 172.21.15.32:0/1411613931 wait complete. 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.657+0000 7feb53a9c700 1 Processor -- start 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.657+0000 7feb53a9c700 1 -- start start 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb53a9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb53a9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb53a9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c197270 con 0x7feb4c104080 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb53a9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1973b0 con 0x7feb4c101750 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb51838700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb51037700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb51838700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c1967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:35196/0 (socket says 172.21.15.32:35196) 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.658+0000 7feb51838700 1 -- 172.21.15.32:0/1368036772 learned_addr learned my addr 172.21.15.32:0/1368036772 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.659+0000 7feb51037700 1 -- 172.21.15.32:0/1368036772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 msgr2=0x7feb4c1967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.659+0000 7feb51037700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c1967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.659+0000 7feb51037700 1 -- 172.21.15.32:0/1368036772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0095f0 con 0x7feb4c104080 2024-06-03T01:20:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.659+0000 7feb51037700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c196ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb3c000c00 tx=0x7feb3c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.659+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3c01d070 con 0x7feb4c104080 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.660+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb3c00f460 con 0x7feb4c104080 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.660+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3c021640 con 0x7feb4c104080 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.660+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c19be10 con 0x7feb4c104080 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.660+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c19c280 con 0x7feb4c104080 2024-06-03T01:20:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.661+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c190630 con 0x7feb4c104080 2024-06-03T01:20:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.662+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7feb3c00fa20 con 0x7feb4c104080 2024-06-03T01:20:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.663+0000 7feb42ffd700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 0x7feb3806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.663+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7feb3c08ca50 con 0x7feb4c104080 2024-06-03T01:20:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.663+0000 7feb51838700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 0x7feb3806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.664+0000 7feb51838700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 0x7feb3806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ad20 tx=0x7feb48005f90 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:28.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:28.668+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb3c05ace0 con 0x7feb4c104080 2024-06-03T01:20:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.026+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb4c02d090 con 0x7feb3806c4e0 2024-06-03T01:20:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.028+0000 7feb42ffd700 1 -- 172.21.15.32:0/1368036772 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7feb4c02d090 con 0x7feb3806c4e0 2024-06-03T01:20:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 msgr2=0x7feb3806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 0x7feb3806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ad20 tx=0x7feb48005f90 comp rx=0 tx=0).stop 2024-06-03T01:20:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 msgr2=0x7feb4c196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c196ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb3c000c00 tx=0x7feb3c00bd70 comp rx=0 tx=0).stop 2024-06-03T01:20:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 shutdown_connections 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7feb3806c4e0 0x7feb3806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb4c101750 0x7feb4c1967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 --2- 172.21.15.32:0/1368036772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb4c104080 0x7feb4c196ce0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 >> 172.21.15.32:0/1368036772 conn(0x7feb4c0fb160 msgr2=0x7feb4c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.031+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 shutdown_connections 2024-06-03T01:20:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.032+0000 7feb53a9c700 1 -- 172.21.15.32:0/1368036772 wait complete. 2024-06-03T01:20:29.047 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:20:29.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3557394007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc100ca0 msgr2=0x7fe2dc1010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3557394007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc1010c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009910 tx=0x7fe2cc009c20 comp rx=0 tx=0).stop 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3557394007 shutdown_connections 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3557394007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc101ea0 0x7fe2dc102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3557394007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc1010c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3557394007 >> 172.21.15.32:0/3557394007 conn(0x7fe2dc0fc260 msgr2=0x7fe2dc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3557394007 shutdown_connections 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.198+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3557394007 wait complete. 2024-06-03T01:20:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 Processor -- start 2024-06-03T01:20:29.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 -- start start 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc101ea0 0x7fe2dc072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc072970 con 0x7fe2dc101ea0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2e15b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc1a4240 con 0x7fe2dc100ca0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.199+0000 7fe2da7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc101ea0 0x7fe2dc072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:35214/0 (socket says 172.21.15.32:35214) 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 -- 172.21.15.32:0/3269021465 learned_addr learned my addr 172.21.15.32:0/3269021465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 -- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc101ea0 msgr2=0x7fe2dc072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc101ea0 0x7fe2dc072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 -- 172.21.15.32:0/3269021465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc0095f0 con 0x7fe2dc100ca0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2daffd700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc000c00 tx=0x7fe2cc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2cc01d070 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe2cc005dc0 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.200+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2cc022470 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.201+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc072fc0 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.201+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc073460 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.202+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc109800 con 0x7fe2dc100ca0 2024-06-03T01:20:29.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.203+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe2cc003810 con 0x7fe2dc100ca0 2024-06-03T01:20:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.203+0000 7fe2c3fff700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 0x7fe2c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.204+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe2cc08cd60 con 0x7fe2dc100ca0 2024-06-03T01:20:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.204+0000 7fe2da7fc700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 0x7fe2c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.204+0000 7fe2da7fc700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 0x7fe2c406e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009c60 tx=0x7fe2d0009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.208+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe2cc05af50 con 0x7fe2dc100ca0 2024-06-03T01:20:29.288 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:28 smithi044 ceph-mon[36956]: Upgrade: Updating mgr.smithi044.wszjlw 2024-06-03T01:20:29.288 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:28 smithi044 ceph-mon[36956]: Deploying daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:20:29.288 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:28 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:28 smithi032 ceph-mon[29360]: Upgrade: Updating mgr.smithi044.wszjlw 2024-06-03T01:20:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:28 smithi032 ceph-mon[29360]: Deploying daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:20:29.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:28 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:29.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.554+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2dc1067f0 con 0x7fe2c406c530 2024-06-03T01:20:29.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.556+0000 7fe2c3fff700 1 -- 172.21.15.32:0/3269021465 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe2dc1067f0 con 0x7fe2c406c530 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 msgr2=0x7fe2c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 0x7fe2c406e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009c60 tx=0x7fe2d0009260 comp rx=0 tx=0).stop 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 msgr2=0x7fe2dc071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc000c00 tx=0x7fe2cc00f690 comp rx=0 tx=0).stop 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 shutdown_connections 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fe2c406c530 0x7fe2c406e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc100ca0 0x7fe2dc071e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 --2- 172.21.15.32:0/3269021465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2dc101ea0 0x7fe2dc072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.560+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 >> 172.21.15.32:0/3269021465 conn(0x7fe2dc0fc260 msgr2=0x7fe2dc1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.561+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 shutdown_connections 2024-06-03T01:20:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.561+0000 7fe2e15b5700 1 -- 172.21.15.32:0/3269021465 wait complete. 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 -- 172.21.15.32:0/453396900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 msgr2=0x7fbef4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 --2- 172.21.15.32:0/453396900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4103b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009910 tx=0x7fbef0009c20 comp rx=0 tx=0).stop 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 -- 172.21.15.32:0/453396900 shutdown_connections 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 --2- 172.21.15.32:0/453396900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 --2- 172.21.15.32:0/453396900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4103b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 -- 172.21.15.32:0/453396900 >> 172.21.15.32:0/453396900 conn(0x7fbef40fb190 msgr2=0x7fbef40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 -- 172.21.15.32:0/453396900 shutdown_connections 2024-06-03T01:20:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.712+0000 7fbefba96700 1 -- 172.21.15.32:0/453396900 wait complete. 2024-06-03T01:20:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.713+0000 7fbefba96700 1 Processor -- start 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbefba96700 1 -- start start 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbefba96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbefba96700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbefba96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4196fc0 con 0x7fbef41017a0 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbefba96700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4197100 con 0x7fbef41040d0 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45758/0 (socket says 172.21.15.32:45758) 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9832700 1 -- 172.21.15.32:0/371723695 learned_addr learned my addr 172.21.15.32:0/371723695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9031700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9031700 1 -- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 msgr2=0x7fbef4196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9031700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.714+0000 7fbef9031700 1 -- 172.21.15.32:0/371723695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef00095f0 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.715+0000 7fbef9031700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41969a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbee400be10 tx=0x7fbee400dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.715+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee4012c60 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.715+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbee4016070 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.715+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee401b700 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.715+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40ff5f0 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.716+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef40ffb40 con 0x7fbef41040d0 2024-06-03T01:20:29.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.717+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4190610 con 0x7fbef41040d0 2024-06-03T01:20:29.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.718+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbee401c400 con 0x7fbef41040d0 2024-06-03T01:20:29.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.718+0000 7fbeeaffd700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 0x7fbee006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:29.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.718+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbee408f3d0 con 0x7fbef41040d0 2024-06-03T01:20:29.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.718+0000 7fbef9832700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 0x7fbee006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:29.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.719+0000 7fbef9832700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 0x7fbee006e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b5c0 tx=0x7fbef0005940 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:29.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:29.723+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbee40560e0 con 0x7fbef41040d0 2024-06-03T01:20:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.036+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef4100fa0 con 0x7fbee006c4e0 2024-06-03T01:20:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.049+0000 7fbeeaffd700 1 -- 172.21.15.32:0/371723695 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7fbef4100fa0 con 0x7fbee006c4e0 2024-06-03T01:20:30.050 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 105s ago 7m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (7m) 105s ago 7m 7490k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (6m) 110s ago 6m 7360k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (7m) 105s ago 7m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (6m) 110s ago 6m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 105s ago 7m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (2m) 105s ago 2m 16.2M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (2m) 105s ago 2m 14.8M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (2m) 110s ago 2m 11.5M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (2m) 110s ago 2m 14.8M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:9283,8765,8443 running (9m) 105s ago 9m 489M - 18.2.1 5be31c24972a 5db22291b59c 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (6m) 110s ago 6m 443M - 18.2.1 5be31c24972a 5fbe526898cf 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (9m) 105s ago 9m 44.6M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:20:30.051 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (6m) 110s ago 6m 35.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 105s ago 7m 14.2M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 110s ago 6m 16.3M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 105s ago 5m 46.8M 3948M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 105s ago 5m 46.7M 3948M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 105s ago 4m 49.5M 3948M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (4m) 110s ago 4m 47.0M 4972M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (4m) 110s ago 4m 46.2M 4972M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (3m) 110s ago 3m 46.1M 4972M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:20:30.052 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 105s ago 7m 57.9M - 2.43.0 a07b618ecd1d 55ba7950dc8e 2024-06-03T01:20:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 msgr2=0x7fbee006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 0x7fbee006e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b5c0 tx=0x7fbef0005940 comp rx=0 tx=0).stop 2024-06-03T01:20:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 msgr2=0x7fbef41969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41969a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbee400be10 tx=0x7fbee400dbb0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 shutdown_connections 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbef41017a0 0x7fbef4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7fbee006c4e0 0x7fbee006e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 --2- 172.21.15.32:0/371723695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbef41040d0 0x7fbef41969a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 >> 172.21.15.32:0/371723695 conn(0x7fbef40fb190 msgr2=0x7fbef40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 shutdown_connections 2024-06-03T01:20:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.052+0000 7fbefba96700 1 -- 172.21.15.32:0/371723695 wait complete. 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.199+0000 7f8c48319700 1 -- 172.21.15.32:0/925234046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c40100520 msgr2=0x7f8c40100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.199+0000 7f8c48319700 1 --2- 172.21.15.32:0/925234046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c40100520 0x7f8c40100980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009870 tx=0x7f8c30009b80 comp rx=0 tx=0).stop 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 -- 172.21.15.32:0/925234046 shutdown_connections 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 --2- 172.21.15.32:0/925234046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c40100520 0x7f8c40100980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 --2- 172.21.15.32:0/925234046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c400ff320 0x7f8c400ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 -- 172.21.15.32:0/925234046 >> 172.21.15.32:0/925234046 conn(0x7f8c400fa8c0 msgr2=0x7f8c400fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 -- 172.21.15.32:0/925234046 shutdown_connections 2024-06-03T01:20:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.200+0000 7f8c48319700 1 -- 172.21.15.32:0/925234046 wait complete. 2024-06-03T01:20:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 Processor -- start 2024-06-03T01:20:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 -- start start 2024-06-03T01:20:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c400ff320 0x7f8c401986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40199230 con 0x7f8c40100520 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c48319700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40199370 con 0x7f8c400ff320 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.201+0000 7f8c458b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45774/0 (socket says 172.21.15.32:45774) 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 -- 172.21.15.32:0/1884612275 learned_addr learned my addr 172.21.15.32:0/1884612275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:20:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c460b5700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c400ff320 0x7f8c401986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 -- 172.21.15.32:0/1884612275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c400ff320 msgr2=0x7f8c401986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c400ff320 0x7f8c401986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 -- 172.21.15.32:0/1884612275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009520 con 0x7f8c40100520 2024-06-03T01:20:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.202+0000 7f8c458b4700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000f690 tx=0x7f8c3000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.203+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3001c070 con 0x7f8c40100520 2024-06-03T01:20:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.203+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c3000beb0 con 0x7f8c40100520 2024-06-03T01:20:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.203+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4019ddc0 con 0x7f8c40100520 2024-06-03T01:20:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.203+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4019e2b0 con 0x7f8c40100520 2024-06-03T01:20:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.204+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c300173f0 con 0x7f8c40100520 2024-06-03T01:20:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.205+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c24005320 con 0x7f8c40100520 2024-06-03T01:20:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.206+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8c30020070 con 0x7f8c40100520 2024-06-03T01:20:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.207+0000 7f8c377fe700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 0x7f8c2c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.207+0000 7f8c460b5700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 0x7f8c2c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.207+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8c3008ca90 con 0x7f8c40100520 2024-06-03T01:20:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.208+0000 7f8c460b5700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 0x7f8c2c06eb10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c40100380 tx=0x7f8c3c00a420 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.215+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c3005ad00 con 0x7f8c40100520 2024-06-03T01:20:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:29 smithi044 ceph-mon[36956]: pgmap v244: 65 pgs: 65 active+clean; 80 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s wr, 417 op/s 2024-06-03T01:20:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:29 smithi032 ceph-mon[29360]: pgmap v244: 65 pgs: 65 active+clean; 80 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s wr, 417 op/s 2024-06-03T01:20:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.697+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c24006200 con 0x7f8c40100520 2024-06-03T01:20:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.697+0000 7f8c377fe700 1 -- 172.21.15.32:0/1884612275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f8c3005a890 con 0x7f8c40100520 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:20:30.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:20:30.700 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:20:30.700 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-06-03T01:20:30.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:20:30.700 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.700+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 msgr2=0x7f8c2c06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.700+0000 7f8c48319700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 0x7f8c2c06eb10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c40100380 tx=0x7f8c3c00a420 comp rx=0 tx=0).stop 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 msgr2=0x7f8c40198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000f690 tx=0x7f8c3000f770 comp rx=0 tx=0).stop 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 shutdown_connections 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f8c2c06c650 0x7f8c2c06eb10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c400ff320 0x7f8c401986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 --2- 172.21.15.32:0/1884612275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c40100520 0x7f8c40198c10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 >> 172.21.15.32:0/1884612275 conn(0x7f8c400fa8c0 msgr2=0x7f8c40103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 shutdown_connections 2024-06-03T01:20:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.701+0000 7f8c48319700 1 -- 172.21.15.32:0/1884612275 wait complete. 2024-06-03T01:20:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 -- 172.21.15.32:0/3805791352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841017a0 msgr2=0x7f3684103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 --2- 172.21.15.32:0/3805791352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841017a0 0x7f3684103b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3680009910 tx=0x7f3680009c20 comp rx=0 tx=0).stop 2024-06-03T01:20:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 -- 172.21.15.32:0/3805791352 shutdown_connections 2024-06-03T01:20:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 --2- 172.21.15.32:0/3805791352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841040d0 0x7f36841064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 --2- 172.21.15.32:0/3805791352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841017a0 0x7f3684103b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 -- 172.21.15.32:0/3805791352 >> 172.21.15.32:0/3805791352 conn(0x7f36840fb190 msgr2=0x7f36840fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 -- 172.21.15.32:0/3805791352 shutdown_connections 2024-06-03T01:20:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.871+0000 7f368c4be700 1 -- 172.21.15.32:0/3805791352 wait complete. 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 Processor -- start 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 -- start start 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 0x7f3684196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 0x7f36841969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f3689a59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 0x7f36841969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f3689a59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 0x7f36841969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45788/0 (socket says 172.21.15.32:45788) 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f3689a59700 1 -- 172.21.15.32:0/432501456 learned_addr learned my addr 172.21.15.32:0/432501456 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368a25a700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 0x7f3684196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3684196fc0 con 0x7f36841040d0 2024-06-03T01:20:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.872+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3684197100 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f368a25a700 1 -- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 msgr2=0x7f36841969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f368a25a700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 0x7f36841969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f368a25a700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800095f0 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f368a25a700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 0x7f3684196460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3680009910 tx=0x7f368000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368001d070 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f368000f460 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3680021620 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.873+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368419bb50 con 0x7f36841017a0 2024-06-03T01:20:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.874+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368419c040 con 0x7f36841017a0 2024-06-03T01:20:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.875+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3684190610 con 0x7f36841017a0 2024-06-03T01:20:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.876+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f368000fa40 con 0x7f36841017a0 2024-06-03T01:20:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.876+0000 7f367b7fe700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 0x7f367006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.877+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f368008ca00 con 0x7f36841017a0 2024-06-03T01:20:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.877+0000 7f3689a59700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 0x7f367006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.877+0000 7f3689a59700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 0x7f367006e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3674005950 tx=0x7f367400b500 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:30.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:30.881+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f368005abf0 con 0x7f36841017a0 2024-06-03T01:20:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.221+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 --> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36840611d0 con 0x7f367006c4e0 2024-06-03T01:20:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.222+0000 7f367b7fe700 1 -- 172.21.15.32:0/432501456 <== mgr.14219 v2:172.21.15.32:6800/2678186916 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f36840611d0 con 0x7f367006c4e0 2024-06-03T01:20:31.223 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:20:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "0/2 daemons upgraded", 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mgr daemons", 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:20:31.224 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:20:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 msgr2=0x7f367006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 0x7f367006e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3674005950 tx=0x7f367400b500 comp rx=0 tx=0).stop 2024-06-03T01:20:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 msgr2=0x7f3684196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 0x7f3684196460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3680009910 tx=0x7f368000bd70 comp rx=0 tx=0).stop 2024-06-03T01:20:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 shutdown_connections 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:6800/2678186916,v1:172.21.15.32:6801/2678186916] conn(0x7f367006c4e0 0x7f367006e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36841017a0 0x7f3684196460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 --2- 172.21.15.32:0/432501456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36841040d0 0x7f36841969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 >> 172.21.15.32:0/432501456 conn(0x7f36840fb190 msgr2=0x7f36840fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 shutdown_connections 2024-06-03T01:20:31.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:20:31.225+0000 7f368c4be700 1 -- 172.21.15.32:0/432501456 wait complete. 2024-06-03T01:20:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:30 smithi032 ceph-mon[29360]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:30 smithi032 ceph-mon[29360]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:30 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1884612275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:20:31.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:30 smithi044 ceph-mon[36956]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:31.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:30 smithi044 ceph-mon[36956]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:31.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:30 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1884612275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:20:32.293 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:31 smithi044 ceph-mon[36956]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:32.294 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:31 smithi044 ceph-mon[36956]: pgmap v245: 65 pgs: 65 active+clean; 87 MiB data, 476 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s wr, 282 op/s 2024-06-03T01:20:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:31 smithi032 ceph-mon[29360]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:31 smithi032 ceph-mon[29360]: pgmap v245: 65 pgs: 65 active+clean; 87 MiB data, 476 MiB used, 536 GiB / 536 GiB avail; 5.0 MiB/s wr, 282 op/s 2024-06-03T01:20:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:32 smithi032 ceph-mon[29360]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:33.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:32 smithi044 ceph-mon[36956]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:20:34.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:33 smithi032 ceph-mon[29360]: pgmap v246: 65 pgs: 65 active+clean; 110 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s wr, 445 op/s 2024-06-03T01:20:34.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:33 smithi044 ceph-mon[36956]: pgmap v246: 65 pgs: 65 active+clean; 110 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s wr, 445 op/s 2024-06-03T01:20:35.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:35 smithi044 ceph-mon[36956]: pgmap v247: 65 pgs: 65 active+clean; 142 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s wr, 649 op/s 2024-06-03T01:20:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:35 smithi032 ceph-mon[29360]: pgmap v247: 65 pgs: 65 active+clean; 142 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s wr, 649 op/s 2024-06-03T01:20:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:37 smithi044 ceph-mon[36956]: pgmap v248: 65 pgs: 65 active+clean; 146 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s wr, 720 op/s 2024-06-03T01:20:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:37 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:37 smithi032 ceph-mon[29360]: pgmap v248: 65 pgs: 65 active+clean; 146 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s wr, 720 op/s 2024-06-03T01:20:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:37 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:40.328 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:39 smithi044 ceph-mon[36956]: pgmap v249: 65 pgs: 65 active+clean; 167 MiB data, 777 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s wr, 764 op/s 2024-06-03T01:20:40.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:39 smithi032 ceph-mon[29360]: pgmap v249: 65 pgs: 65 active+clean; 167 MiB data, 777 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s wr, 764 op/s 2024-06-03T01:20:40.775 INFO:tasks.workunit.client.1.smithi044.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-06-03T01:20:40.784 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-03T01:20:40.784 INFO:tasks.workunit.client.1.smithi044.stderr:+ make 2024-06-03T01:20:40.927 INFO:tasks.workunit.client.1.smithi044.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-06-03T01:20:41.101 INFO:tasks.workunit.client.0.smithi032.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-06-03T01:20:41.109 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-03T01:20:41.109 INFO:tasks.workunit.client.0.smithi032.stderr:+ make 2024-06-03T01:20:41.258 INFO:tasks.workunit.client.1.smithi044.stderr:++ readlink -f fsstress 2024-06-03T01:20:41.258 INFO:tasks.workunit.client.1.smithi044.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-06-03T01:20:41.258 INFO:tasks.workunit.client.1.smithi044.stderr:+ popd 2024-06-03T01:20:41.259 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-03T01:20:41.259 INFO:tasks.workunit.client.1.smithi044.stderr:+ popd 2024-06-03T01:20:41.259 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp 2024-06-03T01:20:41.259 INFO:tasks.workunit.client.1.smithi044.stderr:++ mktemp -d -p . 2024-06-03T01:20:41.260 INFO:tasks.workunit.client.1.smithi044.stderr:+ T=./tmp.57AOhJ9MAj 2024-06-03T01:20:41.260 INFO:tasks.workunit.client.1.smithi044.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.57AOhJ9MAj -l 1 -n 1000 -p 10 -v 2024-06-03T01:20:41.261 INFO:tasks.workunit.client.1.smithi044.stdout:seed = 1717115054 2024-06-03T01:20:41.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/0: stat - no entries 2024-06-03T01:20:41.264 INFO:tasks.workunit.client.1.smithi044.stdout:0/1: link - no file 2024-06-03T01:20:41.264 INFO:tasks.workunit.client.1.smithi044.stdout:0/2: creat f0 x:0 0 0 2024-06-03T01:20:41.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/0: fdatasync - no filename 2024-06-03T01:20:41.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/1: dwrite - no filename 2024-06-03T01:20:41.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/2: rmdir - no directory 2024-06-03T01:20:41.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/3: dread - no filename 2024-06-03T01:20:41.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/4: fsync - no filename 2024-06-03T01:20:41.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/5: link - no file 2024-06-03T01:20:41.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/6: readlink - no filename 2024-06-03T01:20:41.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/7: chown . 228 1 2024-06-03T01:20:41.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/8: dwrite - no filename 2024-06-03T01:20:41.268 INFO:tasks.workunit.client.1.smithi044.stdout:1/0: mkdir d0 0 2024-06-03T01:20:41.268 INFO:tasks.workunit.client.1.smithi044.stdout:1/1: dread - no filename 2024-06-03T01:20:41.269 INFO:tasks.workunit.client.1.smithi044.stdout:3/0: link - no file 2024-06-03T01:20:41.269 INFO:tasks.workunit.client.1.smithi044.stdout:3/1: chown . 3144484 1 2024-06-03T01:20:41.269 INFO:tasks.workunit.client.1.smithi044.stdout:4/0: chown . 120051616 1 2024-06-03T01:20:41.269 INFO:tasks.workunit.client.1.smithi044.stdout:4/1: fsync - no filename 2024-06-03T01:20:41.270 INFO:tasks.workunit.client.1.smithi044.stdout:4/2: write - no filename 2024-06-03T01:20:41.270 INFO:tasks.workunit.client.1.smithi044.stdout:1/2: mknod d0/c1 0 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:4/3: mknod c0 0 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:4/4: chown c0 8042 1 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/3: rmdir d0 39 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/4: chown d0 960 1 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/5: write - no filename 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/6: fdatasync - no filename 2024-06-03T01:20:41.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/7: write - no filename 2024-06-03T01:20:41.273 INFO:tasks.workunit.client.1.smithi044.stdout:6/0: dwrite - no filename 2024-06-03T01:20:41.273 INFO:tasks.workunit.client.1.smithi044.stdout:5/0: symlink l0 0 2024-06-03T01:20:41.273 INFO:tasks.workunit.client.1.smithi044.stdout:5/1: read - no filename 2024-06-03T01:20:41.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/5: creat f1 x:0 0 0 2024-06-03T01:20:41.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/6: dread - f1 zero size 2024-06-03T01:20:41.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/7: dread - f1 zero size 2024-06-03T01:20:41.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/8: dread - f1 zero size 2024-06-03T01:20:41.275 INFO:tasks.workunit.client.1.smithi044.stdout:4/9: write f1 [268240,62071] 0 2024-06-03T01:20:41.275 INFO:tasks.workunit.client.1.smithi044.stdout:1/8: link d0/c1 d0/c2 0 2024-06-03T01:20:41.275 INFO:tasks.workunit.client.1.smithi044.stdout:6/1: symlink l0 0 2024-06-03T01:20:41.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/2: creat f1 x:0 0 0 2024-06-03T01:20:41.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/3: chown l0 6472955 1 2024-06-03T01:20:41.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/4: write f1 [635324,46791] 0 2024-06-03T01:20:41.277 INFO:tasks.workunit.client.1.smithi044.stdout:7/0: mkdir d0 0 2024-06-03T01:20:41.277 INFO:tasks.workunit.client.1.smithi044.stdout:7/1: write - no filename 2024-06-03T01:20:41.277 INFO:tasks.workunit.client.1.smithi044.stdout:7/2: write - no filename 2024-06-03T01:20:41.278 INFO:tasks.workunit.client.1.smithi044.stdout:1/9: mkdir d0/d3 0 2024-06-03T01:20:41.279 INFO:tasks.workunit.client.1.smithi044.stdout:6/2: symlink l1 0 2024-06-03T01:20:41.280 INFO:tasks.workunit.client.1.smithi044.stdout:5/5: mkdir d2 0 2024-06-03T01:20:41.280 INFO:tasks.workunit.client.1.smithi044.stdout:5/6: chown l0 318 1 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/0: dread - no filename 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/1: chown . 3094 1 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/2: dwrite - no filename 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/3: chown . 14 1 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/4: dread - no filename 2024-06-03T01:20:41.282 INFO:tasks.workunit.client.1.smithi044.stdout:8/5: dwrite - no filename 2024-06-03T01:20:41.283 INFO:tasks.workunit.client.1.smithi044.stdout:7/3: creat d0/f1 x:0 0 0 2024-06-03T01:20:41.283 INFO:tasks.workunit.client.1.smithi044.stdout:1/10: mkdir d0/d3/d4 0 2024-06-03T01:20:41.284 INFO:tasks.workunit.client.1.smithi044.stdout:6/3: mkdir d2 0 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:5/7: creat d2/f3 x:0 0 0 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/6: creat f0 x:0 0 0 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/7: fdatasync f0 0 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/8: stat f0 0 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/9: dread - f0 zero size 2024-06-03T01:20:41.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/10: chown f0 448031 1 2024-06-03T01:20:41.289 INFO:tasks.workunit.client.1.smithi044.stdout:1/11: creat d0/f5 x:0 0 0 2024-06-03T01:20:41.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/4: mknod d2/c3 0 2024-06-03T01:20:41.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/11: symlink l1 0 2024-06-03T01:20:41.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/12: write f0 [352234,48047] 0 2024-06-03T01:20:41.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/13: creat f2 x:0 0 0 2024-06-03T01:20:41.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/14: creat f3 x:0 0 0 2024-06-03T01:20:41.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/15: write f2 [850194,22794] 0 2024-06-03T01:20:41.291 INFO:tasks.workunit.client.1.smithi044.stdout:8/16: truncate f3 919621 0 2024-06-03T01:20:41.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/12: mknod d0/c6 0 2024-06-03T01:20:41.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/13: chown d0/c2 12 1 2024-06-03T01:20:41.292 INFO:tasks.workunit.client.1.smithi044.stdout:6/5: mkdir d2/d4 0 2024-06-03T01:20:41.296 INFO:tasks.workunit.client.1.smithi044.stdout:6/6: mknod d2/c5 0 2024-06-03T01:20:41.297 INFO:tasks.workunit.client.1.smithi044.stdout:6/7: chown d2/c5 362 1 2024-06-03T01:20:41.297 INFO:tasks.workunit.client.1.smithi044.stdout:6/8: dread - no filename 2024-06-03T01:20:41.297 INFO:tasks.workunit.client.1.smithi044.stdout:8/17: mknod c4 0 2024-06-03T01:20:41.306 INFO:tasks.workunit.client.1.smithi044.stdout:6/9: mkdir d2/d6 0 2024-06-03T01:20:41.306 INFO:tasks.workunit.client.1.smithi044.stdout:0/3: dwrite f0 [0,4194304] 0 2024-06-03T01:20:41.308 INFO:tasks.workunit.client.1.smithi044.stdout:6/10: creat d2/d4/f7 x:0 0 0 2024-06-03T01:20:41.312 INFO:tasks.workunit.client.1.smithi044.stdout:6/11: rename l1 to d2/d4/l8 0 2024-06-03T01:20:41.312 INFO:tasks.workunit.client.1.smithi044.stdout:6/12: truncate d2/d4/f7 727575 0 2024-06-03T01:20:41.625 INFO:tasks.workunit.client.0.smithi032.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-06-03T01:20:41.949 INFO:tasks.workunit.client.0.smithi032.stderr:++ readlink -f fsstress 2024-06-03T01:20:41.950 INFO:tasks.workunit.client.0.smithi032.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-06-03T01:20:41.950 INFO:tasks.workunit.client.0.smithi032.stderr:+ popd 2024-06-03T01:20:41.955 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-03T01:20:41.955 INFO:tasks.workunit.client.0.smithi032.stderr:+ popd 2024-06-03T01:20:41.955 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp 2024-06-03T01:20:41.955 INFO:tasks.workunit.client.0.smithi032.stderr:++ mktemp -d -p . 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.0.smithi032.stderr:+ T=./tmp.TgdXWrVXgM 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.0.smithi032.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.TgdXWrVXgM -l 1 -n 1000 -p 10 -v 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.1.smithi044.stdout:0/4: dread f0 [0,4194304] 0 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.1.smithi044.stdout:0/5: creat f1 x:0 0 0 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.1.smithi044.stdout:0/6: chown f0 1038306 1 2024-06-03T01:20:41.957 INFO:tasks.workunit.client.1.smithi044.stdout:0/7: write f1 [368553,24813] 0 2024-06-03T01:20:41.958 INFO:tasks.workunit.client.0.smithi032.stdout:seed = 1716763723 2024-06-03T01:20:41.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/0: mkdir d0 0 2024-06-03T01:20:41.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/1: dwrite - no filename 2024-06-03T01:20:41.964 INFO:tasks.workunit.client.0.smithi032.stdout:1/0: fdatasync - no filename 2024-06-03T01:20:41.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/1: mknod c0 0 2024-06-03T01:20:41.966 INFO:tasks.workunit.client.0.smithi032.stdout:2/0: write - no filename 2024-06-03T01:20:41.967 INFO:tasks.workunit.client.0.smithi032.stdout:2/1: mkdir d0 0 2024-06-03T01:20:41.968 INFO:tasks.workunit.client.0.smithi032.stdout:3/0: chown . 58 1 2024-06-03T01:20:41.969 INFO:tasks.workunit.client.0.smithi032.stdout:2/2: rmdir d0 0 2024-06-03T01:20:41.969 INFO:tasks.workunit.client.0.smithi032.stdout:2/3: fsync - no filename 2024-06-03T01:20:41.970 INFO:tasks.workunit.client.0.smithi032.stdout:3/1: mknod c0 0 2024-06-03T01:20:41.971 INFO:tasks.workunit.client.0.smithi032.stdout:4/0: dread - no filename 2024-06-03T01:20:41.974 INFO:tasks.workunit.client.0.smithi032.stdout:3/2: creat f1 x:0 0 0 2024-06-03T01:20:41.975 INFO:tasks.workunit.client.0.smithi032.stdout:3/3: read - f1 zero size 2024-06-03T01:20:41.975 INFO:tasks.workunit.client.0.smithi032.stdout:2/4: creat f1 x:0 0 0 2024-06-03T01:20:41.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/1: creat f0 x:0 0 0 2024-06-03T01:20:41.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/2: dread - f0 zero size 2024-06-03T01:20:41.976 INFO:tasks.workunit.client.1.smithi044.stdout:6/13: dwrite d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:41.979 INFO:tasks.workunit.client.0.smithi032.stdout:6/0: dwrite - no filename 2024-06-03T01:20:41.979 INFO:tasks.workunit.client.0.smithi032.stdout:3/4: rename c0 to c2 0 2024-06-03T01:20:41.979 INFO:tasks.workunit.client.0.smithi032.stdout:5/0: creat f0 x:0 0 0 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/1: mkdir d0 0 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/2: fsync - no filename 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/3: link - no file 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/4: truncate - no filename 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/5: link - no file 2024-06-03T01:20:41.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/6: link - no file 2024-06-03T01:20:41.982 INFO:tasks.workunit.client.0.smithi032.stdout:3/5: mknod c3 0 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:7/0: link - no file 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/7: rmdir d0 0 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/8: stat - no entries 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/9: stat - no entries 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/10: dread - no filename 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/11: write - no filename 2024-06-03T01:20:41.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/12: dwrite - no filename 2024-06-03T01:20:41.990 INFO:tasks.workunit.client.0.smithi032.stdout:7/1: symlink l0 0 2024-06-03T01:20:41.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/13: mknod c1 0 2024-06-03T01:20:41.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/14: write - no filename 2024-06-03T01:20:41.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/15: chown c1 2256 1 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:8/0: readlink - no filename 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:7/2: symlink l1 0 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:7/3: stat l0 0 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:7/4: chown l0 923522164 1 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:6/16: mknod c2 0 2024-06-03T01:20:42.000 INFO:tasks.workunit.client.0.smithi032.stdout:6/17: truncate - no filename 2024-06-03T01:20:42.003 INFO:tasks.workunit.client.0.smithi032.stdout:8/1: symlink l0 0 2024-06-03T01:20:42.004 INFO:tasks.workunit.client.0.smithi032.stdout:8/2: dwrite - no filename 2024-06-03T01:20:42.005 INFO:tasks.workunit.client.1.smithi044.stdout:7/4: rmdir d0 39 2024-06-03T01:20:42.005 INFO:tasks.workunit.client.1.smithi044.stdout:7/5: readlink - no filename 2024-06-03T01:20:42.006 INFO:tasks.workunit.client.0.smithi032.stdout:7/5: rename l1 to l2 0 2024-06-03T01:20:42.006 INFO:tasks.workunit.client.0.smithi032.stdout:7/6: dread - no filename 2024-06-03T01:20:42.006 INFO:tasks.workunit.client.0.smithi032.stdout:7/7: fdatasync - no filename 2024-06-03T01:20:42.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/3: creat f1 x:0 0 0 2024-06-03T01:20:42.011 INFO:tasks.workunit.client.0.smithi032.stdout:6/18: creat f3 x:0 0 0 2024-06-03T01:20:42.011 INFO:tasks.workunit.client.0.smithi032.stdout:6/19: creat f4 x:0 0 0 2024-06-03T01:20:42.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/0: mkdir d0 0 2024-06-03T01:20:42.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/1: dwrite - no filename 2024-06-03T01:20:42.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/2: dread - no filename 2024-06-03T01:20:42.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/3: truncate - no filename 2024-06-03T01:20:42.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/4: readlink - no filename 2024-06-03T01:20:42.016 INFO:tasks.workunit.client.0.smithi032.stdout:7/8: mknod c3 0 2024-06-03T01:20:42.016 INFO:tasks.workunit.client.0.smithi032.stdout:7/9: fsync - no filename 2024-06-03T01:20:42.016 INFO:tasks.workunit.client.0.smithi032.stdout:7/10: dread - no filename 2024-06-03T01:20:42.016 INFO:tasks.workunit.client.0.smithi032.stdout:7/11: dread - no filename 2024-06-03T01:20:42.018 INFO:tasks.workunit.client.0.smithi032.stdout:6/20: mknod c5 0 2024-06-03T01:20:42.018 INFO:tasks.workunit.client.0.smithi032.stdout:9/5: creat d0/f1 x:0 0 0 2024-06-03T01:20:42.018 INFO:tasks.workunit.client.0.smithi032.stdout:9/6: dread - d0/f1 zero size 2024-06-03T01:20:42.018 INFO:tasks.workunit.client.0.smithi032.stdout:9/7: chown d0 1252 1 2024-06-03T01:20:42.018 INFO:tasks.workunit.client.0.smithi032.stdout:9/8: creat d0/f2 x:0 0 0 2024-06-03T01:20:42.021 INFO:tasks.workunit.client.0.smithi032.stdout:7/12: link c3 c4 0 2024-06-03T01:20:42.023 INFO:tasks.workunit.client.0.smithi032.stdout:7/13: symlink l5 0 2024-06-03T01:20:42.023 INFO:tasks.workunit.client.0.smithi032.stdout:7/14: truncate - no filename 2024-06-03T01:20:42.023 INFO:tasks.workunit.client.0.smithi032.stdout:7/15: truncate - no filename 2024-06-03T01:20:42.024 INFO:tasks.workunit.client.0.smithi032.stdout:7/16: dread - no filename 2024-06-03T01:20:42.024 INFO:tasks.workunit.client.0.smithi032.stdout:7/17: dwrite - no filename 2024-06-03T01:20:42.024 INFO:tasks.workunit.client.0.smithi032.stdout:7/18: creat f6 x:0 0 0 2024-06-03T01:20:42.101 INFO:tasks.workunit.client.1.smithi044.stdout:1/14: dwrite d0/f5 [0,4194304] 0 2024-06-03T01:20:42.125 INFO:tasks.workunit.client.1.smithi044.stdout:0/8: dwrite f1 [0,4194304] 0 2024-06-03T01:20:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:41 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:41 smithi044 ceph-mon[36956]: pgmap v250: 65 pgs: 65 active+clean; 167 MiB data, 789 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s wr, 642 op/s 2024-06-03T01:20:42.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:42.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:41 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:42.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:41 smithi032 ceph-mon[29360]: pgmap v250: 65 pgs: 65 active+clean; 167 MiB data, 789 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s wr, 642 op/s 2024-06-03T01:20:42.372 INFO:tasks.workunit.client.1.smithi044.stdout:8/18: dread f2 [0,4194304] 0 2024-06-03T01:20:42.372 INFO:tasks.workunit.client.1.smithi044.stdout:8/19: stat f3 0 2024-06-03T01:20:42.373 INFO:tasks.workunit.client.1.smithi044.stdout:8/20: truncate f3 1813077 0 2024-06-03T01:20:42.397 INFO:tasks.workunit.client.1.smithi044.stdout:4/10: dwrite f1 [0,4194304] 0 2024-06-03T01:20:42.433 INFO:tasks.workunit.client.1.smithi044.stdout:4/11: read f1 [3127414,34703] 0 2024-06-03T01:20:42.462 INFO:tasks.workunit.client.1.smithi044.stdout:6/14: link d2/c5 d2/c9 0 2024-06-03T01:20:42.462 INFO:tasks.workunit.client.1.smithi044.stdout:6/15: chown d2/d4 15 1 2024-06-03T01:20:42.474 INFO:tasks.workunit.client.1.smithi044.stdout:7/6: getdents d0 0 2024-06-03T01:20:42.494 INFO:tasks.workunit.client.1.smithi044.stdout:1/15: getdents d0/d3 0 2024-06-03T01:20:42.507 INFO:tasks.workunit.client.1.smithi044.stdout:0/9: mknod c2 0 2024-06-03T01:20:42.507 INFO:tasks.workunit.client.1.smithi044.stdout:0/10: chown c2 4 1 2024-06-03T01:20:42.518 INFO:tasks.workunit.client.1.smithi044.stdout:4/12: dread f1 [0,4194304] 0 2024-06-03T01:20:42.561 INFO:tasks.workunit.client.1.smithi044.stdout:8/21: mkdir d5 0 2024-06-03T01:20:42.576 INFO:tasks.workunit.client.1.smithi044.stdout:0/11: write f1 [287557,101744] 0 2024-06-03T01:20:42.579 INFO:tasks.workunit.client.1.smithi044.stdout:6/16: rename l0 to d2/d4/la 0 2024-06-03T01:20:42.584 INFO:tasks.workunit.client.1.smithi044.stdout:7/7: unlink d0/f1 0 2024-06-03T01:20:42.584 INFO:tasks.workunit.client.1.smithi044.stdout:7/8: read - no filename 2024-06-03T01:20:42.584 INFO:tasks.workunit.client.1.smithi044.stdout:7/9: unlink - no file 2024-06-03T01:20:42.593 INFO:tasks.workunit.client.1.smithi044.stdout:1/16: mkdir d0/d3/d4/d7 0 2024-06-03T01:20:42.604 INFO:tasks.workunit.client.1.smithi044.stdout:8/22: creat d5/f6 x:0 0 0 2024-06-03T01:20:42.607 INFO:tasks.workunit.client.1.smithi044.stdout:0/12: mkdir d3 0 2024-06-03T01:20:42.616 INFO:tasks.workunit.client.1.smithi044.stdout:1/17: creat d0/d3/d4/f8 x:0 0 0 2024-06-03T01:20:42.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/13: rename f1 to d3/f4 0 2024-06-03T01:20:42.625 INFO:tasks.workunit.client.1.smithi044.stdout:1/18: creat d0/d3/f9 x:0 0 0 2024-06-03T01:20:42.788 INFO:tasks.workunit.client.1.smithi044.stdout:4/13: write f1 [4335081,92524] 0 2024-06-03T01:20:42.789 INFO:tasks.workunit.client.1.smithi044.stdout:4/14: creat f2 x:0 0 0 2024-06-03T01:20:42.790 INFO:tasks.workunit.client.0.smithi032.stdout:2/5: dwrite f1 [0,4194304] 0 2024-06-03T01:20:42.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/15: unlink f1 0 2024-06-03T01:20:42.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/16: dread - f2 zero size 2024-06-03T01:20:42.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/17: dread - f2 zero size 2024-06-03T01:20:42.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/18: fdatasync f2 0 2024-06-03T01:20:42.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/19: truncate f2 677576 0 2024-06-03T01:20:42.798 INFO:tasks.workunit.client.1.smithi044.stdout:7/10: getdents d0 0 2024-06-03T01:20:42.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/11: dwrite - no filename 2024-06-03T01:20:42.799 INFO:tasks.workunit.client.0.smithi032.stdout:8/4: dwrite f1 [0,4194304] 0 2024-06-03T01:20:42.802 INFO:tasks.workunit.client.1.smithi044.stdout:7/12: creat d0/f2 x:0 0 0 2024-06-03T01:20:42.806 INFO:tasks.workunit.client.1.smithi044.stdout:7/13: mkdir d0/d3 0 2024-06-03T01:20:42.806 INFO:tasks.workunit.client.1.smithi044.stdout:7/14: dread - d0/f2 zero size 2024-06-03T01:20:42.806 INFO:tasks.workunit.client.1.smithi044.stdout:7/15: readlink - no filename 2024-06-03T01:20:42.806 INFO:tasks.workunit.client.1.smithi044.stdout:7/16: chown d0/f2 47004 1 2024-06-03T01:20:42.814 INFO:tasks.workunit.client.1.smithi044.stdout:1/19: rename d0/d3/d4 to d0/da 0 2024-06-03T01:20:42.814 INFO:tasks.workunit.client.1.smithi044.stdout:1/20: creat d0/fb x:0 0 0 2024-06-03T01:20:42.842 INFO:tasks.workunit.client.1.smithi044.stdout:4/20: write f2 [126799,103878] 0 2024-06-03T01:20:42.843 INFO:tasks.workunit.client.1.smithi044.stdout:4/21: readlink - no filename 2024-06-03T01:20:42.843 INFO:tasks.workunit.client.1.smithi044.stdout:4/22: chown c0 1830925351 1 2024-06-03T01:20:42.843 INFO:tasks.workunit.client.1.smithi044.stdout:4/23: creat f3 x:0 0 0 2024-06-03T01:20:42.852 INFO:tasks.workunit.client.1.smithi044.stdout:1/21: write d0/f5 [1548772,83109] 0 2024-06-03T01:20:42.852 INFO:tasks.workunit.client.1.smithi044.stdout:1/22: creat d0/fc x:0 0 0 2024-06-03T01:20:42.852 INFO:tasks.workunit.client.1.smithi044.stdout:1/23: write d0/f5 [3469397,96091] 0 2024-06-03T01:20:42.852 INFO:tasks.workunit.client.1.smithi044.stdout:1/24: creat d0/fd x:0 0 0 2024-06-03T01:20:42.853 INFO:tasks.workunit.client.1.smithi044.stdout:1/25: rename d0/d3 to d0/d3/de 22 2024-06-03T01:20:42.856 INFO:tasks.workunit.client.1.smithi044.stdout:1/26: symlink d0/da/d7/lf 0 2024-06-03T01:20:42.856 INFO:tasks.workunit.client.1.smithi044.stdout:1/27: write d0/fb [701539,60220] 0 2024-06-03T01:20:43.016 INFO:tasks.workunit.client.1.smithi044.stdout:4/24: dread f2 [0,4194304] 0 2024-06-03T01:20:43.021 INFO:tasks.workunit.client.1.smithi044.stdout:6/17: dread d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:43.033 INFO:tasks.workunit.client.1.smithi044.stdout:0/14: dwrite d3/f4 [0,4194304] 0 2024-06-03T01:20:43.035 INFO:tasks.workunit.client.1.smithi044.stdout:0/15: rename f0 to d3/f5 0 2024-06-03T01:20:43.257 INFO:tasks.workunit.client.0.smithi032.stdout:9/9: dwrite d0/f2 [0,4194304] 0 2024-06-03T01:20:43.257 INFO:tasks.workunit.client.0.smithi032.stdout:9/10: creat d0/f3 x:0 0 0 2024-06-03T01:20:43.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/2: getdents . 0 2024-06-03T01:20:43.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/3: getdents . 0 2024-06-03T01:20:43.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/4: write - no filename 2024-06-03T01:20:43.289 INFO:tasks.workunit.client.0.smithi032.stdout:8/5: fsync f1 0 2024-06-03T01:20:43.289 INFO:tasks.workunit.client.0.smithi032.stdout:8/6: readlink l0 0 2024-06-03T01:20:43.289 INFO:tasks.workunit.client.0.smithi032.stdout:8/7: creat f2 x:0 0 0 2024-06-03T01:20:43.289 INFO:tasks.workunit.client.0.smithi032.stdout:4/3: getdents . 0 2024-06-03T01:20:43.292 INFO:tasks.workunit.client.1.smithi044.stdout:1/28: truncate d0/fb 558118 0 2024-06-03T01:20:43.292 INFO:tasks.workunit.client.1.smithi044.stdout:1/29: write d0/fc [639081,108700] 0 2024-06-03T01:20:43.293 INFO:tasks.workunit.client.1.smithi044.stdout:0/16: fsync d3/f5 0 2024-06-03T01:20:43.296 INFO:tasks.workunit.client.1.smithi044.stdout:1/30: getdents d0/d3 0 2024-06-03T01:20:43.296 INFO:tasks.workunit.client.1.smithi044.stdout:1/31: write d0/f5 [3187669,113931] 0 2024-06-03T01:20:43.296 INFO:tasks.workunit.client.1.smithi044.stdout:1/32: dread - d0/fd zero size 2024-06-03T01:20:43.297 INFO:tasks.workunit.client.1.smithi044.stdout:1/33: symlink d0/l10 0 2024-06-03T01:20:43.300 INFO:tasks.workunit.client.1.smithi044.stdout:1/34: link d0/da/d7/lf d0/d3/l11 0 2024-06-03T01:20:43.307 INFO:tasks.workunit.client.0.smithi032.stdout:3/6: unlink c2 0 2024-06-03T01:20:43.307 INFO:tasks.workunit.client.0.smithi032.stdout:3/7: fdatasync f1 0 2024-06-03T01:20:43.307 INFO:tasks.workunit.client.0.smithi032.stdout:5/1: getdents . 0 2024-06-03T01:20:43.307 INFO:tasks.workunit.client.0.smithi032.stdout:5/2: chown f0 58512980 1 2024-06-03T01:20:43.307 INFO:tasks.workunit.client.0.smithi032.stdout:5/3: rmdir - no directory 2024-06-03T01:20:43.308 INFO:tasks.workunit.client.1.smithi044.stdout:7/17: dwrite d0/f2 [0,4194304] 0 2024-06-03T01:20:43.308 INFO:tasks.workunit.client.1.smithi044.stdout:7/18: chown d0/d3 6115980 1 2024-06-03T01:20:43.372 INFO:tasks.workunit.client.1.smithi044.stdout:7/19: read d0/f2 [464225,66412] 0 2024-06-03T01:20:43.372 INFO:tasks.workunit.client.1.smithi044.stdout:7/20: creat d0/f4 x:0 0 0 2024-06-03T01:20:43.372 INFO:tasks.workunit.client.1.smithi044.stdout:7/21: rename d0/d3 to d0/d3/d5 22 2024-06-03T01:20:43.374 INFO:tasks.workunit.client.1.smithi044.stdout:1/35: dwrite d0/fd [0,4194304] 0 2024-06-03T01:20:43.380 INFO:tasks.workunit.client.1.smithi044.stdout:1/36: unlink d0/c6 0 2024-06-03T01:20:43.382 INFO:tasks.workunit.client.1.smithi044.stdout:1/37: mknod d0/c12 0 2024-06-03T01:20:43.383 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:43.384 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:43 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:43.384 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:43 smithi044 ceph-mon[36956]: pgmap v251: 65 pgs: 65 active+clean; 204 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 10 MiB/s wr, 731 op/s 2024-06-03T01:20:43.416 INFO:tasks.workunit.client.1.smithi044.stdout:0/17: dread d3/f4 [0,4194304] 0 2024-06-03T01:20:43.419 INFO:tasks.workunit.client.1.smithi044.stdout:0/18: rename d3/f5 to d3/f6 0 2024-06-03T01:20:43.419 INFO:tasks.workunit.client.1.smithi044.stdout:0/19: chown d3/f6 115 1 2024-06-03T01:20:43.423 INFO:tasks.workunit.client.1.smithi044.stdout:1/38: dwrite d0/fd [0,4194304] 0 2024-06-03T01:20:43.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/39: mkdir d0/da/d7/d13 0 2024-06-03T01:20:43.426 INFO:tasks.workunit.client.1.smithi044.stdout:1/40: fdatasync d0/d3/f9 0 2024-06-03T01:20:43.426 INFO:tasks.workunit.client.1.smithi044.stdout:1/41: mkdir d0/d3/d14 0 2024-06-03T01:20:43.426 INFO:tasks.workunit.client.1.smithi044.stdout:1/42: chown d0/da/d7 6939915 1 2024-06-03T01:20:43.430 INFO:tasks.workunit.client.1.smithi044.stdout:1/43: dread d0/fb [0,4194304] 0 2024-06-03T01:20:43.433 INFO:tasks.workunit.client.1.smithi044.stdout:1/44: rename d0/fc to d0/f15 0 2024-06-03T01:20:43.434 INFO:tasks.workunit.client.1.smithi044.stdout:1/45: rmdir d0/da/d7 39 2024-06-03T01:20:43.434 INFO:tasks.workunit.client.1.smithi044.stdout:1/46: readlink d0/d3/l11 0 2024-06-03T01:20:43.435 INFO:tasks.workunit.client.1.smithi044.stdout:7/22: truncate d0/f2 2110847 0 2024-06-03T01:20:43.437 INFO:tasks.workunit.client.1.smithi044.stdout:1/47: mkdir d0/d16 0 2024-06-03T01:20:43.437 INFO:tasks.workunit.client.1.smithi044.stdout:1/48: readlink d0/da/d7/lf 0 2024-06-03T01:20:43.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/21: dwrite f4 [0,4194304] 0 2024-06-03T01:20:43.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/22: creat f6 x:0 0 0 2024-06-03T01:20:43.484 INFO:tasks.workunit.client.1.smithi044.stdout:1/49: dread d0/f5 [0,4194304] 0 2024-06-03T01:20:43.484 INFO:tasks.workunit.client.1.smithi044.stdout:0/20: dread d3/f4 [0,4194304] 0 2024-06-03T01:20:43.498 INFO:tasks.workunit.client.1.smithi044.stdout:7/23: dwrite d0/f2 [0,4194304] 0 2024-06-03T01:20:43.498 INFO:tasks.workunit.client.1.smithi044.stdout:7/24: chown d0 44 1 2024-06-03T01:20:43.500 INFO:tasks.workunit.client.1.smithi044.stdout:7/25: mknod d0/c6 0 2024-06-03T01:20:43.502 INFO:tasks.workunit.client.1.smithi044.stdout:7/26: creat d0/d3/f7 x:0 0 0 2024-06-03T01:20:43.502 INFO:tasks.workunit.client.1.smithi044.stdout:7/27: stat d0/d3 0 2024-06-03T01:20:43.502 INFO:tasks.workunit.client.1.smithi044.stdout:7/28: write d0/d3/f7 [1019710,5244] 0 2024-06-03T01:20:43.504 INFO:tasks.workunit.client.1.smithi044.stdout:7/29: mkdir d0/d3/d8 0 2024-06-03T01:20:43.504 INFO:tasks.workunit.client.1.smithi044.stdout:7/30: chown d0/d3 4597586 1 2024-06-03T01:20:43.508 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:43.508 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:43 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:43.508 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:43 smithi032 ceph-mon[29360]: pgmap v251: 65 pgs: 65 active+clean; 204 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 10 MiB/s wr, 731 op/s 2024-06-03T01:20:43.514 INFO:tasks.workunit.client.1.smithi044.stdout:7/31: read d0/f2 [1969371,78479] 0 2024-06-03T01:20:43.516 INFO:tasks.workunit.client.1.smithi044.stdout:7/32: mknod d0/c9 0 2024-06-03T01:20:43.516 INFO:tasks.workunit.client.1.smithi044.stdout:7/33: creat d0/d3/fa x:0 0 0 2024-06-03T01:20:43.516 INFO:tasks.workunit.client.1.smithi044.stdout:0/21: dwrite d3/f4 [0,4194304] 0 2024-06-03T01:20:43.516 INFO:tasks.workunit.client.1.smithi044.stdout:0/22: readlink - no filename 2024-06-03T01:20:43.518 INFO:tasks.workunit.client.1.smithi044.stdout:7/34: rename d0/c6 to d0/d3/cb 0 2024-06-03T01:20:43.518 INFO:tasks.workunit.client.1.smithi044.stdout:7/35: write d0/f4 [725696,107511] 0 2024-06-03T01:20:43.519 INFO:tasks.workunit.client.1.smithi044.stdout:7/36: symlink d0/d3/d8/lc 0 2024-06-03T01:20:43.527 INFO:tasks.workunit.client.1.smithi044.stdout:1/50: dwrite d0/da/f8 [0,4194304] 0 2024-06-03T01:20:43.527 INFO:tasks.workunit.client.1.smithi044.stdout:1/51: truncate d0/d3/f9 922648 0 2024-06-03T01:20:43.530 INFO:tasks.workunit.client.1.smithi044.stdout:1/52: creat d0/da/d7/f17 x:0 0 0 2024-06-03T01:20:43.530 INFO:tasks.workunit.client.1.smithi044.stdout:1/53: readlink d0/d3/l11 0 2024-06-03T01:20:43.530 INFO:tasks.workunit.client.1.smithi044.stdout:1/54: write d0/f5 [2925560,114482] 0 2024-06-03T01:20:43.546 INFO:tasks.workunit.client.1.smithi044.stdout:7/37: rmdir d0/d3 39 2024-06-03T01:20:43.546 INFO:tasks.workunit.client.1.smithi044.stdout:1/55: dread d0/fb [0,4194304] 0 2024-06-03T01:20:43.546 INFO:tasks.workunit.client.1.smithi044.stdout:7/38: write d0/f2 [1203627,13642] 0 2024-06-03T01:20:43.547 INFO:tasks.workunit.client.1.smithi044.stdout:1/56: write d0/d3/f9 [424192,121776] 0 2024-06-03T01:20:43.548 INFO:tasks.workunit.client.1.smithi044.stdout:1/57: mknod d0/d16/c18 0 2024-06-03T01:20:43.550 INFO:tasks.workunit.client.1.smithi044.stdout:1/58: creat d0/d3/f19 x:0 0 0 2024-06-03T01:20:43.550 INFO:tasks.workunit.client.1.smithi044.stdout:1/59: rename d0 to d0/d16/d1a 22 2024-06-03T01:20:43.554 INFO:tasks.workunit.client.1.smithi044.stdout:1/60: rename d0/fb to d0/da/d7/d13/f1b 0 2024-06-03T01:20:43.555 INFO:tasks.workunit.client.1.smithi044.stdout:6/18: dwrite d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:43.558 INFO:tasks.workunit.client.1.smithi044.stdout:6/19: mkdir d2/d6/db 0 2024-06-03T01:20:43.558 INFO:tasks.workunit.client.1.smithi044.stdout:6/20: truncate d2/d4/f7 4833540 0 2024-06-03T01:20:43.561 INFO:tasks.workunit.client.1.smithi044.stdout:6/21: link d2/c5 d2/d4/cc 0 2024-06-03T01:20:43.582 INFO:tasks.workunit.client.1.smithi044.stdout:1/61: dwrite d0/d3/f9 [0,4194304] 0 2024-06-03T01:20:43.582 INFO:tasks.workunit.client.1.smithi044.stdout:1/62: readlink d0/d3/l11 0 2024-06-03T01:20:43.582 INFO:tasks.workunit.client.1.smithi044.stdout:1/63: readlink d0/da/d7/lf 0 2024-06-03T01:20:43.584 INFO:tasks.workunit.client.1.smithi044.stdout:1/64: mknod d0/c1c 0 2024-06-03T01:20:43.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/65: creat d0/da/f1d x:0 0 0 2024-06-03T01:20:43.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/66: write d0/fd [4740754,2050] 0 2024-06-03T01:20:43.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/67: readlink d0/l10 0 2024-06-03T01:20:43.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/68: creat d0/da/f1e x:0 0 0 2024-06-03T01:20:43.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/25: dwrite f3 [0,4194304] 0 2024-06-03T01:20:43.738 INFO:tasks.workunit.client.1.smithi044.stdout:0/23: write d3/f6 [3142317,83311] 0 2024-06-03T01:20:43.738 INFO:tasks.workunit.client.1.smithi044.stdout:0/24: fsync d3/f4 0 2024-06-03T01:20:43.748 INFO:tasks.workunit.client.1.smithi044.stdout:6/22: dwrite d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:43.748 INFO:tasks.workunit.client.1.smithi044.stdout:6/23: fsync d2/d4/f7 0 2024-06-03T01:20:43.748 INFO:tasks.workunit.client.1.smithi044.stdout:1/69: dwrite d0/da/f1d [0,4194304] 0 2024-06-03T01:20:43.748 INFO:tasks.workunit.client.1.smithi044.stdout:1/70: creat d0/da/f1f x:0 0 0 2024-06-03T01:20:43.748 INFO:tasks.workunit.client.1.smithi044.stdout:1/71: stat d0/da/d7/d13/f1b 0 2024-06-03T01:20:43.750 INFO:tasks.workunit.client.1.smithi044.stdout:6/24: mknod d2/d6/cd 0 2024-06-03T01:20:43.757 INFO:tasks.workunit.client.1.smithi044.stdout:6/25: creat d2/fe x:0 0 0 2024-06-03T01:20:43.770 INFO:tasks.workunit.client.1.smithi044.stdout:0/25: dwrite d3/f6 [4194304,4194304] 0 2024-06-03T01:20:43.773 INFO:tasks.workunit.client.1.smithi044.stdout:6/26: dwrite d2/fe [0,4194304] 0 2024-06-03T01:20:43.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/11: dwrite d0/f2 [0,4194304] 0 2024-06-03T01:20:43.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/12: dread - d0/f1 zero size 2024-06-03T01:20:43.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/13: fdatasync d0/f3 0 2024-06-03T01:20:43.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/14: truncate d0/f3 607205 0 2024-06-03T01:20:43.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/15: stat d0/f1 0 2024-06-03T01:20:43.786 INFO:tasks.workunit.client.1.smithi044.stdout:0/26: dwrite d3/f6 [0,4194304] 0 2024-06-03T01:20:43.793 INFO:tasks.workunit.client.1.smithi044.stdout:7/39: dwrite d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:43.793 INFO:tasks.workunit.client.1.smithi044.stdout:7/40: chown d0/d3/fa 32564052 1 2024-06-03T01:20:43.793 INFO:tasks.workunit.client.0.smithi032.stdout:6/23: dwrite f6 [0,4194304] 0 2024-06-03T01:20:43.793 INFO:tasks.workunit.client.0.smithi032.stdout:6/24: stat f4 0 2024-06-03T01:20:43.794 INFO:tasks.workunit.client.1.smithi044.stdout:1/72: rename d0/da to d0/d3/d20 0 2024-06-03T01:20:43.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/27: dread d3/f4 [0,4194304] 0 2024-06-03T01:20:43.800 INFO:tasks.workunit.client.1.smithi044.stdout:1/73: dread d0/d3/d20/d7/d13/f1b [0,4194304] 0 2024-06-03T01:20:43.801 INFO:tasks.workunit.client.1.smithi044.stdout:1/74: write d0/d3/d20/f1f [599249,107064] 0 2024-06-03T01:20:43.801 INFO:tasks.workunit.client.1.smithi044.stdout:0/28: mknod d3/c7 0 2024-06-03T01:20:43.802 INFO:tasks.workunit.client.1.smithi044.stdout:1/75: getdents d0/d16 0 2024-06-03T01:20:43.803 INFO:tasks.workunit.client.1.smithi044.stdout:0/29: rmdir d3 39 2024-06-03T01:20:43.805 INFO:tasks.workunit.client.1.smithi044.stdout:1/76: rename d0/d3/d20/d7/d13/f1b to d0/d3/d20/f21 0 2024-06-03T01:20:43.806 INFO:tasks.workunit.client.1.smithi044.stdout:1/77: dread - d0/d3/f19 zero size 2024-06-03T01:20:43.806 INFO:tasks.workunit.client.1.smithi044.stdout:1/78: stat d0/c1 0 2024-06-03T01:20:43.809 INFO:tasks.workunit.client.1.smithi044.stdout:1/79: read d0/fd [4306877,29143] 0 2024-06-03T01:20:43.809 INFO:tasks.workunit.client.1.smithi044.stdout:0/30: dread d3/f4 [0,4194304] 0 2024-06-03T01:20:43.811 INFO:tasks.workunit.client.1.smithi044.stdout:1/80: mknod d0/d3/d20/d7/c22 0 2024-06-03T01:20:43.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/25: dread f4 [0,4194304] 0 2024-06-03T01:20:43.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/26: readlink - no filename 2024-06-03T01:20:43.813 INFO:tasks.workunit.client.1.smithi044.stdout:0/31: write d3/f4 [1000375,128467] 0 2024-06-03T01:20:43.814 INFO:tasks.workunit.client.1.smithi044.stdout:1/81: rename d0/d3/d20/f1f to d0/d3/f23 0 2024-06-03T01:20:43.816 INFO:tasks.workunit.client.1.smithi044.stdout:0/32: unlink c2 0 2024-06-03T01:20:43.818 INFO:tasks.workunit.client.1.smithi044.stdout:7/41: dwrite d0/d3/fa [0,4194304] 0 2024-06-03T01:20:43.818 INFO:tasks.workunit.client.1.smithi044.stdout:6/27: dwrite d2/fe [4194304,4194304] 0 2024-06-03T01:20:43.818 INFO:tasks.workunit.client.1.smithi044.stdout:6/28: chown d2/d6/cd 424538 1 2024-06-03T01:20:43.818 INFO:tasks.workunit.client.1.smithi044.stdout:7/42: creat d0/fd x:0 0 0 2024-06-03T01:20:43.819 INFO:tasks.workunit.client.1.smithi044.stdout:6/29: fsync d2/fe 0 2024-06-03T01:20:43.827 INFO:tasks.workunit.client.1.smithi044.stdout:7/43: creat d0/d3/d8/fe x:0 0 0 2024-06-03T01:20:43.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/30: mkdir d2/d6/db/df 0 2024-06-03T01:20:43.835 INFO:tasks.workunit.client.1.smithi044.stdout:7/44: read d0/d3/fa [32578,46324] 0 2024-06-03T01:20:43.837 INFO:tasks.workunit.client.1.smithi044.stdout:0/33: dwrite d3/f4 [0,4194304] 0 2024-06-03T01:20:43.838 INFO:tasks.workunit.client.1.smithi044.stdout:7/45: link d0/d3/cb d0/d3/cf 0 2024-06-03T01:20:43.838 INFO:tasks.workunit.client.1.smithi044.stdout:7/46: creat d0/d3/d8/f10 x:0 0 0 2024-06-03T01:20:43.840 INFO:tasks.workunit.client.1.smithi044.stdout:0/34: link d3/f6 d3/f8 0 2024-06-03T01:20:43.844 INFO:tasks.workunit.client.1.smithi044.stdout:7/47: mknod d0/d3/c11 0 2024-06-03T01:20:43.845 INFO:tasks.workunit.client.1.smithi044.stdout:6/31: dwrite d2/fe [4194304,4194304] 0 2024-06-03T01:20:43.845 INFO:tasks.workunit.client.1.smithi044.stdout:0/35: mkdir d3/d9 0 2024-06-03T01:20:43.846 INFO:tasks.workunit.client.1.smithi044.stdout:0/36: readlink - no filename 2024-06-03T01:20:43.848 INFO:tasks.workunit.client.1.smithi044.stdout:7/48: creat d0/d3/f12 x:0 0 0 2024-06-03T01:20:43.850 INFO:tasks.workunit.client.1.smithi044.stdout:7/49: link d0/c9 d0/c13 0 2024-06-03T01:20:43.850 INFO:tasks.workunit.client.1.smithi044.stdout:7/50: dread - d0/d3/d8/fe zero size 2024-06-03T01:20:43.850 INFO:tasks.workunit.client.1.smithi044.stdout:7/51: read - d0/d3/d8/fe zero size 2024-06-03T01:20:43.857 INFO:tasks.workunit.client.1.smithi044.stdout:6/32: dread d2/fe [4194304,4194304] 0 2024-06-03T01:20:43.857 INFO:tasks.workunit.client.1.smithi044.stdout:0/37: write d3/f8 [7891682,55107] 0 2024-06-03T01:20:43.859 INFO:tasks.workunit.client.1.smithi044.stdout:6/33: mknod d2/d4/c10 0 2024-06-03T01:20:43.860 INFO:tasks.workunit.client.1.smithi044.stdout:0/38: symlink d3/la 0 2024-06-03T01:20:43.861 INFO:tasks.workunit.client.1.smithi044.stdout:6/34: link d2/c5 d2/c11 0 2024-06-03T01:20:43.863 INFO:tasks.workunit.client.1.smithi044.stdout:0/39: creat d3/d9/fb x:0 0 0 2024-06-03T01:20:43.865 INFO:tasks.workunit.client.1.smithi044.stdout:0/40: creat d3/fc x:0 0 0 2024-06-03T01:20:43.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/35: dread d2/fe [4194304,4194304] 0 2024-06-03T01:20:43.866 INFO:tasks.workunit.client.1.smithi044.stdout:0/41: mknod d3/cd 0 2024-06-03T01:20:43.867 INFO:tasks.workunit.client.1.smithi044.stdout:0/42: chown d3/la 11 1 2024-06-03T01:20:43.869 INFO:tasks.workunit.client.1.smithi044.stdout:6/36: creat d2/d6/db/f12 x:0 0 0 2024-06-03T01:20:43.869 INFO:tasks.workunit.client.1.smithi044.stdout:6/37: chown d2/c11 6125 1 2024-06-03T01:20:43.871 INFO:tasks.workunit.client.1.smithi044.stdout:6/38: link d2/d4/f7 d2/d6/f13 0 2024-06-03T01:20:43.873 INFO:tasks.workunit.client.1.smithi044.stdout:7/52: dwrite d0/f4 [0,4194304] 0 2024-06-03T01:20:43.878 INFO:tasks.workunit.client.1.smithi044.stdout:0/43: dread d3/f4 [0,4194304] 0 2024-06-03T01:20:43.878 INFO:tasks.workunit.client.1.smithi044.stdout:0/44: truncate d3/f6 9368680 0 2024-06-03T01:20:43.882 INFO:tasks.workunit.client.1.smithi044.stdout:7/53: rename d0/d3/f12 to d0/d3/d8/f14 0 2024-06-03T01:20:43.884 INFO:tasks.workunit.client.1.smithi044.stdout:7/54: symlink d0/l15 0 2024-06-03T01:20:43.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/39: rmdir d2 39 2024-06-03T01:20:43.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/40: write d2/d6/db/f12 [113382,54989] 0 2024-06-03T01:20:43.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/41: chown d2/c3 11649471 1 2024-06-03T01:20:43.891 INFO:tasks.workunit.client.1.smithi044.stdout:7/55: mkdir d0/d16 0 2024-06-03T01:20:43.891 INFO:tasks.workunit.client.1.smithi044.stdout:7/56: chown d0/d16 4 1 2024-06-03T01:20:43.891 INFO:tasks.workunit.client.1.smithi044.stdout:7/57: creat d0/d3/d8/f17 x:0 0 0 2024-06-03T01:20:43.894 INFO:tasks.workunit.client.1.smithi044.stdout:0/45: dwrite d3/d9/fb [0,4194304] 0 2024-06-03T01:20:43.894 INFO:tasks.workunit.client.1.smithi044.stdout:0/46: creat d3/fe x:0 0 0 2024-06-03T01:20:43.901 INFO:tasks.workunit.client.1.smithi044.stdout:6/42: write d2/d4/f7 [593296,29489] 0 2024-06-03T01:20:43.904 INFO:tasks.workunit.client.1.smithi044.stdout:7/58: getdents d0/d3/d8 0 2024-06-03T01:20:43.917 INFO:tasks.workunit.client.1.smithi044.stdout:0/47: truncate d3/f6 608959 0 2024-06-03T01:20:43.917 INFO:tasks.workunit.client.1.smithi044.stdout:0/48: write d3/d9/fb [4369966,51669] 0 2024-06-03T01:20:43.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/43: dwrite d2/fe [0,4194304] 0 2024-06-03T01:20:43.927 INFO:tasks.workunit.client.1.smithi044.stdout:6/44: write d2/d6/db/f12 [161698,47692] 0 2024-06-03T01:20:43.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/45: dread d2/d6/db/f12 [0,4194304] 0 2024-06-03T01:20:43.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/46: fsync d2/fe 0 2024-06-03T01:20:43.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/47: readlink d2/d4/la 0 2024-06-03T01:20:43.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/48: creat d2/f14 x:0 0 0 2024-06-03T01:20:43.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/49: fsync d2/fe 0 2024-06-03T01:20:43.937 INFO:tasks.workunit.client.1.smithi044.stdout:6/50: symlink d2/d6/l15 0 2024-06-03T01:20:43.938 INFO:tasks.workunit.client.1.smithi044.stdout:7/59: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-06-03T01:20:43.938 INFO:tasks.workunit.client.1.smithi044.stdout:7/60: creat d0/d3/f18 x:0 0 0 2024-06-03T01:20:43.940 INFO:tasks.workunit.client.1.smithi044.stdout:7/61: mknod d0/d3/d8/c19 0 2024-06-03T01:20:43.941 INFO:tasks.workunit.client.1.smithi044.stdout:7/62: mknod d0/d16/c1a 0 2024-06-03T01:20:43.943 INFO:tasks.workunit.client.1.smithi044.stdout:7/63: creat d0/d3/d8/f1b x:0 0 0 2024-06-03T01:20:43.943 INFO:tasks.workunit.client.1.smithi044.stdout:7/64: dread - d0/d3/d8/fe zero size 2024-06-03T01:20:43.944 INFO:tasks.workunit.client.1.smithi044.stdout:6/51: getdents d2 0 2024-06-03T01:20:43.945 INFO:tasks.workunit.client.1.smithi044.stdout:7/65: getdents d0 0 2024-06-03T01:20:43.945 INFO:tasks.workunit.client.1.smithi044.stdout:7/66: readlink d0/l15 0 2024-06-03T01:20:43.946 INFO:tasks.workunit.client.1.smithi044.stdout:7/67: mknod d0/d3/c1c 0 2024-06-03T01:20:43.946 INFO:tasks.workunit.client.1.smithi044.stdout:7/68: rmdir d0/d16 39 2024-06-03T01:20:43.968 INFO:tasks.workunit.client.1.smithi044.stdout:6/52: dwrite d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:43.968 INFO:tasks.workunit.client.1.smithi044.stdout:7/69: dwrite d0/d3/fa [4194304,4194304] 0 2024-06-03T01:20:43.970 INFO:tasks.workunit.client.1.smithi044.stdout:6/53: rename d2/d6/l15 to d2/d6/db/l16 0 2024-06-03T01:20:43.973 INFO:tasks.workunit.client.1.smithi044.stdout:7/70: rename d0/d3/d8/f17 to d0/f1d 0 2024-06-03T01:20:43.973 INFO:tasks.workunit.client.1.smithi044.stdout:7/71: chown d0/d3/c11 337 1 2024-06-03T01:20:43.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/72: mknod d0/d3/c1e 0 2024-06-03T01:20:43.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/73: write d0/f1d [174792,107027] 0 2024-06-03T01:20:43.984 INFO:tasks.workunit.client.1.smithi044.stdout:7/74: dread d0/d3/d8/f10 [0,4194304] 0 2024-06-03T01:20:43.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/75: stat d0/c13 0 2024-06-03T01:20:43.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/76: fdatasync d0/fd 0 2024-06-03T01:20:43.985 INFO:tasks.workunit.client.1.smithi044.stdout:6/54: dwrite d2/fe [4194304,4194304] 0 2024-06-03T01:20:43.985 INFO:tasks.workunit.client.1.smithi044.stdout:6/55: mkdir d2/d6/db/df/d17 0 2024-06-03T01:20:43.986 INFO:tasks.workunit.client.1.smithi044.stdout:6/56: creat d2/d6/db/df/d17/f18 x:0 0 0 2024-06-03T01:20:43.987 INFO:tasks.workunit.client.1.smithi044.stdout:6/57: dread d2/d6/db/f12 [0,4194304] 0 2024-06-03T01:20:43.988 INFO:tasks.workunit.client.1.smithi044.stdout:6/58: creat d2/d6/db/df/f19 x:0 0 0 2024-06-03T01:20:43.992 INFO:tasks.workunit.client.1.smithi044.stdout:7/77: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:43.992 INFO:tasks.workunit.client.1.smithi044.stdout:7/78: creat d0/d3/d8/f1f x:0 0 0 2024-06-03T01:20:43.994 INFO:tasks.workunit.client.1.smithi044.stdout:7/79: symlink d0/d3/l20 0 2024-06-03T01:20:43.994 INFO:tasks.workunit.client.1.smithi044.stdout:7/80: truncate d0/f1d 556099 0 2024-06-03T01:20:43.995 INFO:tasks.workunit.client.1.smithi044.stdout:7/81: truncate d0/f2 697338 0 2024-06-03T01:20:43.997 INFO:tasks.workunit.client.1.smithi044.stdout:6/59: write d2/d4/f7 [2335583,68431] 0 2024-06-03T01:20:43.998 INFO:tasks.workunit.client.1.smithi044.stdout:6/60: rmdir d2/d4 39 2024-06-03T01:20:44.011 INFO:tasks.workunit.client.1.smithi044.stdout:7/82: dwrite d0/d3/fa [0,4194304] 0 2024-06-03T01:20:44.013 INFO:tasks.workunit.client.1.smithi044.stdout:7/83: mkdir d0/d21 0 2024-06-03T01:20:44.013 INFO:tasks.workunit.client.1.smithi044.stdout:7/84: chown d0/d16 416 1 2024-06-03T01:20:44.013 INFO:tasks.workunit.client.1.smithi044.stdout:7/85: truncate d0/f1d 1231275 0 2024-06-03T01:20:44.014 INFO:tasks.workunit.client.1.smithi044.stdout:7/86: creat d0/d16/f22 x:0 0 0 2024-06-03T01:20:44.014 INFO:tasks.workunit.client.1.smithi044.stdout:7/87: chown d0/d3/d8/c19 5716 1 2024-06-03T01:20:44.015 INFO:tasks.workunit.client.1.smithi044.stdout:7/88: mkdir d0/d16/d23 0 2024-06-03T01:20:44.030 INFO:tasks.workunit.client.1.smithi044.stdout:7/89: dwrite d0/fd [0,4194304] 0 2024-06-03T01:20:44.031 INFO:tasks.workunit.client.1.smithi044.stdout:7/90: creat d0/d16/d23/f24 x:0 0 0 2024-06-03T01:20:44.032 INFO:tasks.workunit.client.1.smithi044.stdout:7/91: mknod d0/d21/c25 0 2024-06-03T01:20:44.033 INFO:tasks.workunit.client.1.smithi044.stdout:7/92: symlink d0/d16/d23/l26 0 2024-06-03T01:20:44.034 INFO:tasks.workunit.client.1.smithi044.stdout:7/93: mkdir d0/d21/d27 0 2024-06-03T01:20:44.034 INFO:tasks.workunit.client.1.smithi044.stdout:7/94: fdatasync d0/d3/d8/f10 0 2024-06-03T01:20:44.034 INFO:tasks.workunit.client.1.smithi044.stdout:7/95: chown d0/d3/f18 0 1 2024-06-03T01:20:44.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/96: dread d0/d3/d8/f10 [0,4194304] 0 2024-06-03T01:20:44.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/97: chown d0/c9 40 1 2024-06-03T01:20:44.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/98: dread d0/fd [0,4194304] 0 2024-06-03T01:20:44.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/99: write d0/f2 [380438,102287] 0 2024-06-03T01:20:44.048 INFO:tasks.workunit.client.1.smithi044.stdout:7/100: creat d0/d21/f28 x:0 0 0 2024-06-03T01:20:44.048 INFO:tasks.workunit.client.1.smithi044.stdout:7/101: fdatasync d0/d3/d8/f1f 0 2024-06-03T01:20:44.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/102: dread d0/f2 [0,4194304] 0 2024-06-03T01:20:44.052 INFO:tasks.workunit.client.1.smithi044.stdout:7/103: link d0/d3/d8/c19 d0/d21/d27/c29 0 2024-06-03T01:20:44.053 INFO:tasks.workunit.client.1.smithi044.stdout:7/104: fsync d0/fd 0 2024-06-03T01:20:44.070 INFO:tasks.workunit.client.1.smithi044.stdout:7/105: dwrite d0/d3/d8/fe [0,4194304] 0 2024-06-03T01:20:44.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/106: symlink d0/d21/d27/l2a 0 2024-06-03T01:20:44.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/107: fdatasync d0/d16/d23/f24 0 2024-06-03T01:20:44.077 INFO:tasks.workunit.client.1.smithi044.stdout:7/108: read d0/d3/d8/fe [1205238,62976] 0 2024-06-03T01:20:44.079 INFO:tasks.workunit.client.1.smithi044.stdout:7/109: rename d0/d3/l20 to d0/d3/d8/l2b 0 2024-06-03T01:20:44.103 INFO:tasks.workunit.client.1.smithi044.stdout:7/110: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-06-03T01:20:44.104 INFO:tasks.workunit.client.1.smithi044.stdout:7/111: write d0/f2 [1522127,126303] 0 2024-06-03T01:20:44.104 INFO:tasks.workunit.client.1.smithi044.stdout:7/112: chown d0/l15 8544 1 2024-06-03T01:20:44.104 INFO:tasks.workunit.client.1.smithi044.stdout:7/113: rename d0/d3/d8/fe to d0/d3/f2c 0 2024-06-03T01:20:44.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/114: mknod d0/d3/c2d 0 2024-06-03T01:20:44.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/115: mknod d0/c2e 0 2024-06-03T01:20:44.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/116: stat d0/d3/d8/f1f 0 2024-06-03T01:20:44.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/117: chown d0/d3/c2d 37 1 2024-06-03T01:20:44.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/118: fdatasync d0/f4 0 2024-06-03T01:20:44.107 INFO:tasks.workunit.client.1.smithi044.stdout:7/119: unlink d0/d3/d8/f1f 0 2024-06-03T01:20:44.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/120: dread d0/f4 [0,4194304] 0 2024-06-03T01:20:44.128 INFO:tasks.workunit.client.1.smithi044.stdout:7/121: dwrite d0/d3/fa [8388608,4194304] 0 2024-06-03T01:20:44.128 INFO:tasks.workunit.client.1.smithi044.stdout:7/122: fdatasync d0/d3/fa 0 2024-06-03T01:20:44.134 INFO:tasks.workunit.client.1.smithi044.stdout:7/123: rmdir d0/d3/d8 39 2024-06-03T01:20:44.152 INFO:tasks.workunit.client.1.smithi044.stdout:7/124: dwrite d0/d16/d23/f24 [0,4194304] 0 2024-06-03T01:20:44.160 INFO:tasks.workunit.client.1.smithi044.stdout:7/125: write d0/d3/d8/f10 [1513240,77406] 0 2024-06-03T01:20:44.162 INFO:tasks.workunit.client.1.smithi044.stdout:7/126: creat d0/d3/d8/f2f x:0 0 0 2024-06-03T01:20:44.163 INFO:tasks.workunit.client.1.smithi044.stdout:7/127: mkdir d0/d3/d30 0 2024-06-03T01:20:44.164 INFO:tasks.workunit.client.1.smithi044.stdout:7/128: mknod d0/d21/d27/c31 0 2024-06-03T01:20:44.178 INFO:tasks.workunit.client.1.smithi044.stdout:7/129: dwrite d0/f4 [0,4194304] 0 2024-06-03T01:20:44.180 INFO:tasks.workunit.client.1.smithi044.stdout:7/130: mkdir d0/d3/d8/d32 0 2024-06-03T01:20:44.180 INFO:tasks.workunit.client.1.smithi044.stdout:7/131: dread - d0/d3/d8/f14 zero size 2024-06-03T01:20:44.181 INFO:tasks.workunit.client.1.smithi044.stdout:7/132: mkdir d0/d3/d8/d32/d33 0 2024-06-03T01:20:44.182 INFO:tasks.workunit.client.1.smithi044.stdout:7/133: mknod d0/d21/d27/c34 0 2024-06-03T01:20:44.184 INFO:tasks.workunit.client.1.smithi044.stdout:7/134: creat d0/d21/d27/f35 x:0 0 0 2024-06-03T01:20:44.185 INFO:tasks.workunit.client.1.smithi044.stdout:7/135: dread d0/f1d [0,4194304] 0 2024-06-03T01:20:44.187 INFO:tasks.workunit.client.1.smithi044.stdout:7/136: symlink d0/d3/d8/d32/d33/l36 0 2024-06-03T01:20:44.187 INFO:tasks.workunit.client.1.smithi044.stdout:7/137: creat d0/d3/d8/f37 x:0 0 0 2024-06-03T01:20:44.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/138: write d0/fd [3773238,26984] 0 2024-06-03T01:20:44.192 INFO:tasks.workunit.client.1.smithi044.stdout:7/139: creat d0/d3/d8/d32/f38 x:0 0 0 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/140: dwrite d0/d3/f18 [0,4194304] 0 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/141: creat d0/d3/d8/f39 x:0 0 0 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/142: chown d0/d16 270611759 1 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/143: dread - d0/d3/d8/f14 zero size 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/144: chown d0/d3/cb 434018 1 2024-06-03T01:20:44.209 INFO:tasks.workunit.client.1.smithi044.stdout:7/145: write d0/d3/d8/f1b [100728,61326] 0 2024-06-03T01:20:44.211 INFO:tasks.workunit.client.1.smithi044.stdout:7/146: rename d0/d3/c11 to d0/d16/d23/c3a 0 2024-06-03T01:20:44.215 INFO:tasks.workunit.client.1.smithi044.stdout:7/147: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:44.215 INFO:tasks.workunit.client.1.smithi044.stdout:7/148: creat d0/d21/f3b x:0 0 0 2024-06-03T01:20:44.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/149: write d0/f4 [4089159,112765] 0 2024-06-03T01:20:44.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/150: creat d0/d16/f3c x:0 0 0 2024-06-03T01:20:44.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/151: creat d0/d16/d23/f3d x:0 0 0 2024-06-03T01:20:44.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/152: dread - d0/d21/f3b zero size 2024-06-03T01:20:44.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/153: creat d0/f3e x:0 0 0 2024-06-03T01:20:44.229 INFO:tasks.workunit.client.1.smithi044.stdout:7/154: getdents d0/d3 0 2024-06-03T01:20:44.231 INFO:tasks.workunit.client.1.smithi044.stdout:7/155: creat d0/d3/f3f x:0 0 0 2024-06-03T01:20:44.321 INFO:tasks.workunit.client.0.smithi032.stdout:2/6: mknod c2 0 2024-06-03T01:20:44.321 INFO:tasks.workunit.client.0.smithi032.stdout:2/7: fdatasync f1 0 2024-06-03T01:20:44.330 INFO:tasks.workunit.client.0.smithi032.stdout:2/8: dread f1 [0,4194304] 0 2024-06-03T01:20:44.502 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:44 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:44 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:45.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/5: mkdir d1 0 2024-06-03T01:20:45.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/6: write - no filename 2024-06-03T01:20:45.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/7: dread - no filename 2024-06-03T01:20:45.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/8: dwrite - no filename 2024-06-03T01:20:45.435 INFO:tasks.workunit.client.0.smithi032.stdout:8/8: rename l0 to l3 0 2024-06-03T01:20:45.435 INFO:tasks.workunit.client.0.smithi032.stdout:8/9: fdatasync f1 0 2024-06-03T01:20:45.435 INFO:tasks.workunit.client.0.smithi032.stdout:4/4: symlink l1 0 2024-06-03T01:20:45.435 INFO:tasks.workunit.client.0.smithi032.stdout:4/5: truncate f0 189404 0 2024-06-03T01:20:45.436 INFO:tasks.workunit.client.0.smithi032.stdout:4/6: truncate f0 362186 0 2024-06-03T01:20:45.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/8: rename f1 to f4 0 2024-06-03T01:20:45.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/9: write f4 [357820,7563] 0 2024-06-03T01:20:45.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/10: chown f4 81 1 2024-06-03T01:20:45.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/11: chown f4 802 1 2024-06-03T01:20:45.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/12: rmdir - no directory 2024-06-03T01:20:45.453 INFO:tasks.workunit.client.0.smithi032.stdout:4/7: dwrite f0 [0,4194304] 0 2024-06-03T01:20:45.454 INFO:tasks.workunit.client.0.smithi032.stdout:5/4: mknod c1 0 2024-06-03T01:20:45.454 INFO:tasks.workunit.client.0.smithi032.stdout:5/5: truncate f0 359265 0 2024-06-03T01:20:45.454 INFO:tasks.workunit.client.0.smithi032.stdout:5/6: write f0 [467739,15749] 0 2024-06-03T01:20:45.479 INFO:tasks.workunit.client.0.smithi032.stdout:3/13: dwrite f4 [0,4194304] 0 2024-06-03T01:20:45.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/7: dwrite f0 [0,4194304] 0 2024-06-03T01:20:45.491 INFO:tasks.workunit.client.0.smithi032.stdout:5/8: write f0 [981030,96865] 0 2024-06-03T01:20:45.585 INFO:tasks.workunit.client.0.smithi032.stdout:8/10: write f1 [4316608,46749] 0 2024-06-03T01:20:45.876 INFO:tasks.workunit.client.0.smithi032.stdout:9/16: mknod d0/c4 0 2024-06-03T01:20:45.906 INFO:tasks.workunit.client.0.smithi032.stdout:6/27: symlink l7 0 2024-06-03T01:20:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:45 smithi044 ceph-mon[36956]: pgmap v252: 65 pgs: 65 active+clean; 244 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 12 MiB/s wr, 823 op/s 2024-06-03T01:20:46.022 INFO:tasks.workunit.client.0.smithi032.stdout:1/9: rmdir d1 0 2024-06-03T01:20:46.026 INFO:tasks.workunit.client.0.smithi032.stdout:4/8: rename l1 to l2 0 2024-06-03T01:20:46.031 INFO:tasks.workunit.client.0.smithi032.stdout:5/9: mkdir d2 0 2024-06-03T01:20:46.032 INFO:tasks.workunit.client.0.smithi032.stdout:8/11: symlink l4 0 2024-06-03T01:20:46.036 INFO:tasks.workunit.client.0.smithi032.stdout:6/28: mkdir d8 0 2024-06-03T01:20:46.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/9: truncate f1 885516 0 2024-06-03T01:20:46.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/10: rmdir - no directory 2024-06-03T01:20:46.040 INFO:tasks.workunit.client.0.smithi032.stdout:4/9: mknod c3 0 2024-06-03T01:20:46.041 INFO:tasks.workunit.client.0.smithi032.stdout:4/10: rmdir - no directory 2024-06-03T01:20:46.043 INFO:tasks.workunit.client.0.smithi032.stdout:8/12: mknod c5 0 2024-06-03T01:20:46.045 INFO:tasks.workunit.client.0.smithi032.stdout:4/11: mkdir d4 0 2024-06-03T01:20:46.048 INFO:tasks.workunit.client.0.smithi032.stdout:8/13: mknod c6 0 2024-06-03T01:20:46.049 INFO:tasks.workunit.client.0.smithi032.stdout:4/12: symlink d4/l5 0 2024-06-03T01:20:46.058 INFO:tasks.workunit.client.0.smithi032.stdout:5/10: dwrite f0 [0,4194304] 0 2024-06-03T01:20:46.059 INFO:tasks.workunit.client.0.smithi032.stdout:6/29: dwrite f4 [0,4194304] 0 2024-06-03T01:20:46.059 INFO:tasks.workunit.client.0.smithi032.stdout:6/30: dread - f3 zero size 2024-06-03T01:20:46.071 INFO:tasks.workunit.client.0.smithi032.stdout:6/31: dread f4 [0,4194304] 0 2024-06-03T01:20:46.084 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:45 smithi032 ceph-mon[29360]: pgmap v252: 65 pgs: 65 active+clean; 244 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 12 MiB/s wr, 823 op/s 2024-06-03T01:20:46.088 INFO:tasks.workunit.client.0.smithi032.stdout:6/32: dwrite f4 [0,4194304] 0 2024-06-03T01:20:46.099 INFO:tasks.workunit.client.0.smithi032.stdout:6/33: write f4 [152208,38500] 0 2024-06-03T01:20:46.099 INFO:tasks.workunit.client.0.smithi032.stdout:6/34: readlink l7 0 2024-06-03T01:20:46.099 INFO:tasks.workunit.client.0.smithi032.stdout:6/35: fsync f3 0 2024-06-03T01:20:46.100 INFO:tasks.workunit.client.0.smithi032.stdout:6/36: creat d8/f9 x:0 0 0 2024-06-03T01:20:46.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/37: symlink d8/la 0 2024-06-03T01:20:46.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/38: creat d8/fb x:0 0 0 2024-06-03T01:20:46.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/39: stat d8/fb 0 2024-06-03T01:20:46.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/40: dread - f3 zero size 2024-06-03T01:20:46.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/41: write f4 [2175872,3893] 0 2024-06-03T01:20:46.102 INFO:tasks.workunit.client.0.smithi032.stdout:6/42: symlink d8/lc 0 2024-06-03T01:20:46.123 INFO:tasks.workunit.client.0.smithi032.stdout:6/43: dread f6 [0,4194304] 0 2024-06-03T01:20:46.123 INFO:tasks.workunit.client.0.smithi032.stdout:6/44: dread - d8/fb zero size 2024-06-03T01:20:46.124 INFO:tasks.workunit.client.0.smithi032.stdout:6/45: rename f6 to d8/fd 0 2024-06-03T01:20:46.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/14: rename f4 to f5 0 2024-06-03T01:20:46.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/15: readlink - no filename 2024-06-03T01:20:46.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/16: stat f5 0 2024-06-03T01:20:46.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/17: truncate f5 4876158 0 2024-06-03T01:20:46.331 INFO:tasks.workunit.client.0.smithi032.stdout:3/18: symlink l6 0 2024-06-03T01:20:46.331 INFO:tasks.workunit.client.0.smithi032.stdout:3/19: chown l6 376264453 1 2024-06-03T01:20:46.331 INFO:tasks.workunit.client.0.smithi032.stdout:3/20: symlink l7 0 2024-06-03T01:20:46.332 INFO:tasks.workunit.client.0.smithi032.stdout:3/21: unlink l7 0 2024-06-03T01:20:46.354 INFO:tasks.workunit.client.0.smithi032.stdout:3/22: dwrite f5 [0,4194304] 0 2024-06-03T01:20:46.355 INFO:tasks.workunit.client.0.smithi032.stdout:3/23: dread f5 [4194304,4194304] 0 2024-06-03T01:20:46.366 INFO:tasks.workunit.client.0.smithi032.stdout:3/24: write f5 [3302519,88107] 0 2024-06-03T01:20:46.485 INFO:tasks.workunit.client.0.smithi032.stdout:4/13: truncate f0 3446693 0 2024-06-03T01:20:46.487 INFO:tasks.workunit.client.0.smithi032.stdout:4/14: rmdir d4 39 2024-06-03T01:20:46.491 INFO:tasks.workunit.client.0.smithi032.stdout:4/15: creat d4/f6 x:0 0 0 2024-06-03T01:20:46.496 INFO:tasks.workunit.client.0.smithi032.stdout:8/14: truncate f1 4179765 0 2024-06-03T01:20:46.496 INFO:tasks.workunit.client.0.smithi032.stdout:8/15: read - f2 zero size 2024-06-03T01:20:46.496 INFO:tasks.workunit.client.0.smithi032.stdout:8/16: dread - f2 zero size 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/17: truncate d0/f2 2469373 0 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/18: chown d0/f3 13963 1 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/19: creat d0/f5 x:0 0 0 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/20: write d0/f5 [707140,86185] 0 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/21: creat d0/f6 x:0 0 0 2024-06-03T01:20:46.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/22: dread - d0/f1 zero size 2024-06-03T01:20:46.505 INFO:tasks.workunit.client.0.smithi032.stdout:9/23: symlink d0/l7 0 2024-06-03T01:20:46.505 INFO:tasks.workunit.client.0.smithi032.stdout:9/24: dread - d0/f1 zero size 2024-06-03T01:20:46.506 INFO:tasks.workunit.client.0.smithi032.stdout:9/25: dread d0/f3 [0,4194304] 0 2024-06-03T01:20:46.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/25: dwrite f5 [4194304,4194304] 0 2024-06-03T01:20:46.514 INFO:tasks.workunit.client.0.smithi032.stdout:5/11: dwrite f0 [4194304,4194304] 0 2024-06-03T01:20:46.514 INFO:tasks.workunit.client.0.smithi032.stdout:5/12: getdents d2 0 2024-06-03T01:20:46.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/11: dwrite f1 [0,4194304] 0 2024-06-03T01:20:46.535 INFO:tasks.workunit.client.0.smithi032.stdout:5/13: dwrite f0 [4194304,4194304] 0 2024-06-03T01:20:46.536 INFO:tasks.workunit.client.0.smithi032.stdout:2/12: dwrite f1 [0,4194304] 0 2024-06-03T01:20:46.536 INFO:tasks.workunit.client.0.smithi032.stdout:2/13: write f1 [4598364,108471] 0 2024-06-03T01:20:46.547 INFO:tasks.workunit.client.0.smithi032.stdout:5/14: rmdir d2 0 2024-06-03T01:20:46.556 INFO:tasks.workunit.client.0.smithi032.stdout:2/14: dwrite f1 [0,4194304] 0 2024-06-03T01:20:46.593 INFO:tasks.workunit.client.0.smithi032.stdout:3/26: truncate f5 5591122 0 2024-06-03T01:20:46.596 INFO:tasks.workunit.client.0.smithi032.stdout:3/27: getdents . 0 2024-06-03T01:20:46.596 INFO:tasks.workunit.client.0.smithi032.stdout:3/28: chown f5 38 1 2024-06-03T01:20:46.597 INFO:tasks.workunit.client.0.smithi032.stdout:8/17: dread f1 [0,4194304] 0 2024-06-03T01:20:46.598 INFO:tasks.workunit.client.0.smithi032.stdout:8/18: mknod c7 0 2024-06-03T01:20:46.599 INFO:tasks.workunit.client.0.smithi032.stdout:4/16: dread f0 [0,4194304] 0 2024-06-03T01:20:46.599 INFO:tasks.workunit.client.0.smithi032.stdout:4/17: creat d4/f7 x:0 0 0 2024-06-03T01:20:46.603 INFO:tasks.workunit.client.0.smithi032.stdout:2/15: truncate f1 758623 0 2024-06-03T01:20:46.610 INFO:tasks.workunit.client.0.smithi032.stdout:2/16: rename f1 to f3 0 2024-06-03T01:20:46.610 INFO:tasks.workunit.client.0.smithi032.stdout:2/17: write f3 [1154096,102925] 0 2024-06-03T01:20:46.612 INFO:tasks.workunit.client.0.smithi032.stdout:9/26: truncate d0/f2 2128979 0 2024-06-03T01:20:46.612 INFO:tasks.workunit.client.0.smithi032.stdout:9/27: chown d0/f3 7743436 1 2024-06-03T01:20:46.613 INFO:tasks.workunit.client.0.smithi032.stdout:9/28: mkdir d0/d8 0 2024-06-03T01:20:46.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/29: rmdir d0/d8 0 2024-06-03T01:20:46.616 INFO:tasks.workunit.client.0.smithi032.stdout:3/29: read f5 [4328839,27304] 0 2024-06-03T01:20:46.616 INFO:tasks.workunit.client.0.smithi032.stdout:2/18: fdatasync f3 0 2024-06-03T01:20:46.616 INFO:tasks.workunit.client.0.smithi032.stdout:9/30: mknod d0/c9 0 2024-06-03T01:20:46.617 INFO:tasks.workunit.client.0.smithi032.stdout:3/30: creat f8 x:0 0 0 2024-06-03T01:20:46.619 INFO:tasks.workunit.client.0.smithi032.stdout:2/19: rename c2 to c4 0 2024-06-03T01:20:46.620 INFO:tasks.workunit.client.0.smithi032.stdout:9/31: link d0/f6 d0/fa 0 2024-06-03T01:20:46.621 INFO:tasks.workunit.client.0.smithi032.stdout:2/20: mkdir d5 0 2024-06-03T01:20:46.623 INFO:tasks.workunit.client.0.smithi032.stdout:9/32: mknod d0/cb 0 2024-06-03T01:20:46.623 INFO:tasks.workunit.client.0.smithi032.stdout:9/33: stat d0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:9/34: creat d0/fc x:0 0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/31: dread f5 [0,4194304] 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/32: creat f9 x:0 0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/33: creat fa x:0 0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/34: creat fb x:0 0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/35: rmdir - no directory 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/36: truncate f8 418500 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/37: creat fc x:0 0 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/38: fdatasync f8 0 2024-06-03T01:20:46.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/39: creat fd x:0 0 0 2024-06-03T01:20:46.625 INFO:tasks.workunit.client.0.smithi032.stdout:9/35: dread d0/f3 [0,4194304] 0 2024-06-03T01:20:46.625 INFO:tasks.workunit.client.0.smithi032.stdout:2/21: symlink d5/l6 0 2024-06-03T01:20:46.630 INFO:tasks.workunit.client.0.smithi032.stdout:3/40: mkdir de 0 2024-06-03T01:20:46.630 INFO:tasks.workunit.client.0.smithi032.stdout:3/41: dread - fc zero size 2024-06-03T01:20:46.631 INFO:tasks.workunit.client.0.smithi032.stdout:2/22: symlink d5/l7 0 2024-06-03T01:20:46.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/36: rename d0/fc to d0/fd 0 2024-06-03T01:20:46.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/37: dread - d0/f1 zero size 2024-06-03T01:20:46.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/38: chown d0/l7 1419875024 1 2024-06-03T01:20:46.632 INFO:tasks.workunit.client.0.smithi032.stdout:2/23: symlink d5/l8 0 2024-06-03T01:20:46.633 INFO:tasks.workunit.client.0.smithi032.stdout:2/24: rename d5/l8 to d5/l9 0 2024-06-03T01:20:46.634 INFO:tasks.workunit.client.0.smithi032.stdout:2/25: mkdir d5/da 0 2024-06-03T01:20:46.634 INFO:tasks.workunit.client.0.smithi032.stdout:2/26: creat d5/da/fb x:0 0 0 2024-06-03T01:20:46.634 INFO:tasks.workunit.client.0.smithi032.stdout:2/27: creat d5/da/fc x:0 0 0 2024-06-03T01:20:46.641 INFO:tasks.workunit.client.0.smithi032.stdout:9/39: write d0/f2 [86495,52433] 0 2024-06-03T01:20:46.642 INFO:tasks.workunit.client.0.smithi032.stdout:9/40: getdents d0 0 2024-06-03T01:20:46.644 INFO:tasks.workunit.client.0.smithi032.stdout:9/41: unlink d0/f2 0 2024-06-03T01:20:46.645 INFO:tasks.workunit.client.0.smithi032.stdout:2/28: unlink d5/l9 0 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/42: dwrite fd [0,4194304] 0 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/43: fsync f8 0 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/44: write f9 [224005,83836] 0 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/45: dread - fc zero size 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/46: chown fa 4449371 1 2024-06-03T01:20:46.647 INFO:tasks.workunit.client.0.smithi032.stdout:2/29: fsync d5/da/fb 0 2024-06-03T01:20:46.651 INFO:tasks.workunit.client.0.smithi032.stdout:9/42: rmdir d0 39 2024-06-03T01:20:46.652 INFO:tasks.workunit.client.0.smithi032.stdout:3/47: dread f5 [0,4194304] 0 2024-06-03T01:20:46.652 INFO:tasks.workunit.client.0.smithi032.stdout:9/43: dread - d0/f1 zero size 2024-06-03T01:20:46.653 INFO:tasks.workunit.client.0.smithi032.stdout:2/30: dread f3 [0,4194304] 0 2024-06-03T01:20:46.654 INFO:tasks.workunit.client.0.smithi032.stdout:9/44: creat d0/fe x:0 0 0 2024-06-03T01:20:46.654 INFO:tasks.workunit.client.0.smithi032.stdout:9/45: dread - d0/f1 zero size 2024-06-03T01:20:46.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/31: rmdir d5 39 2024-06-03T01:20:46.658 INFO:tasks.workunit.client.0.smithi032.stdout:3/48: dread f9 [0,4194304] 0 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/32: creat d5/fd x:0 0 0 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/33: dread - d5/da/fc zero size 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/34: dread - d5/da/fb zero size 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/35: truncate f3 1384486 0 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/36: truncate d5/fd 1016135 0 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/37: creat d5/fe x:0 0 0 2024-06-03T01:20:46.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/38: chown d5 438313 1 2024-06-03T01:20:46.661 INFO:tasks.workunit.client.0.smithi032.stdout:2/39: dread d5/fd [0,4194304] 0 2024-06-03T01:20:46.797 INFO:tasks.workunit.client.0.smithi032.stdout:9/46: dwrite d0/f6 [0,4194304] 0 2024-06-03T01:20:46.797 INFO:tasks.workunit.client.0.smithi032.stdout:9/47: truncate d0/fe 660367 0 2024-06-03T01:20:46.798 INFO:tasks.workunit.client.0.smithi032.stdout:9/48: creat d0/ff x:0 0 0 2024-06-03T01:20:46.799 INFO:tasks.workunit.client.0.smithi032.stdout:9/49: truncate d0/fa 5200788 0 2024-06-03T01:20:46.799 INFO:tasks.workunit.client.0.smithi032.stdout:9/50: chown d0/fa 587511 1 2024-06-03T01:20:46.799 INFO:tasks.workunit.client.0.smithi032.stdout:9/51: write d0/f3 [1568547,59506] 0 2024-06-03T01:20:46.804 INFO:tasks.workunit.client.0.smithi032.stdout:2/40: dwrite f3 [0,4194304] 0 2024-06-03T01:20:46.805 INFO:tasks.workunit.client.0.smithi032.stdout:2/41: creat d5/ff x:0 0 0 2024-06-03T01:20:46.805 INFO:tasks.workunit.client.0.smithi032.stdout:2/42: fdatasync d5/da/fb 0 2024-06-03T01:20:46.807 INFO:tasks.workunit.client.0.smithi032.stdout:9/52: rmdir d0 39 2024-06-03T01:20:46.809 INFO:tasks.workunit.client.0.smithi032.stdout:9/53: fdatasync d0/f3 0 2024-06-03T01:20:46.810 INFO:tasks.workunit.client.0.smithi032.stdout:9/54: readlink d0/l7 0 2024-06-03T01:20:46.810 INFO:tasks.workunit.client.0.smithi032.stdout:9/55: rmdir d0 39 2024-06-03T01:20:46.811 INFO:tasks.workunit.client.0.smithi032.stdout:9/56: dread - d0/f1 zero size 2024-06-03T01:20:46.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/57: link d0/c9 d0/c10 0 2024-06-03T01:20:46.815 INFO:tasks.workunit.client.0.smithi032.stdout:9/58: mkdir d0/d11 0 2024-06-03T01:20:46.822 INFO:tasks.workunit.client.0.smithi032.stdout:2/43: dwrite d5/da/fc [0,4194304] 0 2024-06-03T01:20:46.825 INFO:tasks.workunit.client.0.smithi032.stdout:2/44: link d5/fd d5/da/f10 0 2024-06-03T01:20:46.833 INFO:tasks.workunit.client.0.smithi032.stdout:9/59: dwrite d0/f6 [4194304,4194304] 0 2024-06-03T01:20:46.840 INFO:tasks.workunit.client.0.smithi032.stdout:2/45: dwrite d5/fd [0,4194304] 0 2024-06-03T01:20:46.841 INFO:tasks.workunit.client.0.smithi032.stdout:2/46: read - d5/da/fb zero size 2024-06-03T01:20:46.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/47: mkdir d5/d11 0 2024-06-03T01:20:46.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/48: chown d5/fe 3204 1 2024-06-03T01:20:46.844 INFO:tasks.workunit.client.0.smithi032.stdout:2/49: symlink d5/l12 0 2024-06-03T01:20:46.844 INFO:tasks.workunit.client.0.smithi032.stdout:2/50: dread - d5/fe zero size 2024-06-03T01:20:46.844 INFO:tasks.workunit.client.0.smithi032.stdout:2/51: chown d5/fe 3725477 1 2024-06-03T01:20:46.845 INFO:tasks.workunit.client.0.smithi032.stdout:2/52: rename d5/da/fc to d5/f13 0 2024-06-03T01:20:46.846 INFO:tasks.workunit.client.0.smithi032.stdout:2/53: rmdir d5/d11 0 2024-06-03T01:20:46.847 INFO:tasks.workunit.client.0.smithi032.stdout:9/60: dwrite d0/ff [0,4194304] 0 2024-06-03T01:20:46.848 INFO:tasks.workunit.client.0.smithi032.stdout:9/61: write d0/f1 [108812,34156] 0 2024-06-03T01:20:46.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/62: rename d0/c9 to d0/c12 0 2024-06-03T01:20:46.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/63: chown d0/f6 1 1 2024-06-03T01:20:46.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/64: write d0/f1 [1136998,95075] 0 2024-06-03T01:20:46.853 INFO:tasks.workunit.client.0.smithi032.stdout:9/65: symlink d0/d11/l13 0 2024-06-03T01:20:46.855 INFO:tasks.workunit.client.0.smithi032.stdout:2/54: dread d5/f13 [0,4194304] 0 2024-06-03T01:20:46.862 INFO:tasks.workunit.client.0.smithi032.stdout:9/66: read d0/ff [2062771,41988] 0 2024-06-03T01:20:46.864 INFO:tasks.workunit.client.0.smithi032.stdout:9/67: creat d0/d11/f14 x:0 0 0 2024-06-03T01:20:46.872 INFO:tasks.workunit.client.0.smithi032.stdout:2/55: dwrite d5/fd [0,4194304] 0 2024-06-03T01:20:46.873 INFO:tasks.workunit.client.0.smithi032.stdout:2/56: symlink d5/l14 0 2024-06-03T01:20:46.875 INFO:tasks.workunit.client.0.smithi032.stdout:2/57: symlink d5/l15 0 2024-06-03T01:20:46.875 INFO:tasks.workunit.client.0.smithi032.stdout:2/58: fdatasync d5/ff 0 2024-06-03T01:20:46.876 INFO:tasks.workunit.client.0.smithi032.stdout:2/59: mknod d5/da/c16 0 2024-06-03T01:20:46.877 INFO:tasks.workunit.client.0.smithi032.stdout:2/60: rmdir d5/da 39 2024-06-03T01:20:46.879 INFO:tasks.workunit.client.0.smithi032.stdout:2/61: rename d5/da/fb to d5/f17 0 2024-06-03T01:20:46.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/68: dwrite d0/f5 [0,4194304] 0 2024-06-03T01:20:46.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/69: stat d0/f3 0 2024-06-03T01:20:46.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/70: write d0/d11/f14 [481915,124225] 0 2024-06-03T01:20:46.882 INFO:tasks.workunit.client.0.smithi032.stdout:9/71: read d0/f1 [893772,31340] 0 2024-06-03T01:20:46.884 INFO:tasks.workunit.client.0.smithi032.stdout:9/72: getdents d0 0 2024-06-03T01:20:46.886 INFO:tasks.workunit.client.0.smithi032.stdout:9/73: creat d0/f15 x:0 0 0 2024-06-03T01:20:46.895 INFO:tasks.workunit.client.0.smithi032.stdout:9/74: write d0/f5 [3258963,114706] 0 2024-06-03T01:20:46.903 INFO:tasks.workunit.client.0.smithi032.stdout:9/75: dread d0/d11/f14 [0,4194304] 0 2024-06-03T01:20:46.907 INFO:tasks.workunit.client.0.smithi032.stdout:9/76: dread d0/f3 [0,4194304] 0 2024-06-03T01:20:46.907 INFO:tasks.workunit.client.0.smithi032.stdout:9/77: chown d0/f15 4071 1 2024-06-03T01:20:46.908 INFO:tasks.workunit.client.0.smithi032.stdout:9/78: symlink d0/l16 0 2024-06-03T01:20:46.925 INFO:tasks.workunit.client.0.smithi032.stdout:9/79: dwrite d0/fe [0,4194304] 0 2024-06-03T01:20:46.927 INFO:tasks.workunit.client.0.smithi032.stdout:9/80: link d0/l16 d0/d11/l17 0 2024-06-03T01:20:46.931 INFO:tasks.workunit.client.0.smithi032.stdout:9/81: rmdir d0 39 2024-06-03T01:20:46.948 INFO:tasks.workunit.client.0.smithi032.stdout:9/82: dwrite d0/ff [0,4194304] 0 2024-06-03T01:20:46.955 INFO:tasks.workunit.client.0.smithi032.stdout:9/83: write d0/f6 [107929,80883] 0 2024-06-03T01:20:46.957 INFO:tasks.workunit.client.0.smithi032.stdout:9/84: rmdir d0/d11 39 2024-06-03T01:20:46.957 INFO:tasks.workunit.client.0.smithi032.stdout:9/85: read - d0/f15 zero size 2024-06-03T01:20:46.958 INFO:tasks.workunit.client.0.smithi032.stdout:9/86: mknod d0/d11/c18 0 2024-06-03T01:20:46.979 INFO:tasks.workunit.client.0.smithi032.stdout:9/87: dwrite d0/f15 [0,4194304] 0 2024-06-03T01:20:46.998 INFO:tasks.workunit.client.0.smithi032.stdout:9/88: dwrite d0/fa [4194304,4194304] 0 2024-06-03T01:20:47.000 INFO:tasks.workunit.client.0.smithi032.stdout:9/89: creat d0/f19 x:0 0 0 2024-06-03T01:20:47.002 INFO:tasks.workunit.client.0.smithi032.stdout:9/90: mknod d0/c1a 0 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:20:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:46 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:20:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:20:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:20:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:20:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:20:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:20:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:46 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:20:47.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:47 smithi044 ceph-mon[36956]: mgrmap e20: smithi032.kfcfaa(active, since 7m), standbys: smithi044.wszjlw 2024-06-03T01:20:47.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:47 smithi044 ceph-mon[36956]: pgmap v253: 65 pgs: 65 active+clean; 269 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-06-03T01:20:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:47 smithi032 ceph-mon[29360]: mgrmap e20: smithi032.kfcfaa(active, since 7m), standbys: smithi044.wszjlw 2024-06-03T01:20:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:47 smithi032 ceph-mon[29360]: pgmap v253: 65 pgs: 65 active+clean; 269 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:6/61: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:7/156: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:0/49: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:4/26: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:8/23: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:1/82: sync 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:1/83: readlink d0/d3/d20/d7/lf 0 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:0/50: write d3/f8 [1182854,7224] 0 2024-06-03T01:20:48.585 INFO:tasks.workunit.client.1.smithi044.stdout:5/8: sync 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/2: sync 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/3: unlink - no file 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/4: dread - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/5: unlink - no file 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/9: sync 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/6: fdatasync - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/7: dwrite - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/8: dwrite - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/10: dwrite - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/9: readlink - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/0: sync 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/10: dwrite - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/1: rename - no filename 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/2: chown . 3 1 2024-06-03T01:20:48.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/3: dread - no filename 2024-06-03T01:20:48.587 INFO:tasks.workunit.client.1.smithi044.stdout:9/4: read - no filename 2024-06-03T01:20:48.587 INFO:tasks.workunit.client.1.smithi044.stdout:9/5: write - no filename 2024-06-03T01:20:48.587 INFO:tasks.workunit.client.1.smithi044.stdout:9/6: dwrite - no filename 2024-06-03T01:20:48.588 INFO:tasks.workunit.client.1.smithi044.stdout:7/157: getdents d0/d3/d30 0 2024-06-03T01:20:48.589 INFO:tasks.workunit.client.1.smithi044.stdout:1/84: rename d0/d3/f23 to d0/d3/d20/d7/d13/f24 0 2024-06-03T01:20:48.589 INFO:tasks.workunit.client.1.smithi044.stdout:0/51: write d3/d9/fb [4320579,85996] 0 2024-06-03T01:20:48.591 INFO:tasks.workunit.client.1.smithi044.stdout:3/11: mknod c0 0 2024-06-03T01:20:48.591 INFO:tasks.workunit.client.1.smithi044.stdout:2/11: creat f0 x:0 0 0 2024-06-03T01:20:48.592 INFO:tasks.workunit.client.1.smithi044.stdout:9/7: creat f0 x:0 0 0 2024-06-03T01:20:48.592 INFO:tasks.workunit.client.1.smithi044.stdout:9/8: write f0 [746821,39166] 0 2024-06-03T01:20:48.592 INFO:tasks.workunit.client.1.smithi044.stdout:0/52: mknod d3/d9/cf 0 2024-06-03T01:20:48.594 INFO:tasks.workunit.client.1.smithi044.stdout:3/12: symlink l1 0 2024-06-03T01:20:48.594 INFO:tasks.workunit.client.1.smithi044.stdout:2/12: mknod c1 0 2024-06-03T01:20:48.594 INFO:tasks.workunit.client.1.smithi044.stdout:2/13: write f0 [563078,36021] 0 2024-06-03T01:20:48.595 INFO:tasks.workunit.client.1.smithi044.stdout:2/14: write f0 [479728,13134] 0 2024-06-03T01:20:48.595 INFO:tasks.workunit.client.1.smithi044.stdout:0/53: mknod d3/d9/c10 0 2024-06-03T01:20:48.595 INFO:tasks.workunit.client.1.smithi044.stdout:0/54: fsync d3/fc 0 2024-06-03T01:20:48.596 INFO:tasks.workunit.client.1.smithi044.stdout:3/13: link l1 l2 0 2024-06-03T01:20:48.596 INFO:tasks.workunit.client.1.smithi044.stdout:3/14: rename l2 to l3 0 2024-06-03T01:20:48.597 INFO:tasks.workunit.client.1.smithi044.stdout:3/15: fsync - no filename 2024-06-03T01:20:48.605 INFO:tasks.workunit.client.1.smithi044.stdout:2/15: dread f0 [0,4194304] 0 2024-06-03T01:20:48.610 INFO:tasks.workunit.client.1.smithi044.stdout:0/55: write d3/f4 [63293,55086] 0 2024-06-03T01:20:48.612 INFO:tasks.workunit.client.1.smithi044.stdout:3/16: rename l3 to l4 0 2024-06-03T01:20:48.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/17: mknod c5 0 2024-06-03T01:20:48.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/18: chown c5 844 1 2024-06-03T01:20:48.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/19: truncate - no filename 2024-06-03T01:20:48.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/20: fsync - no filename 2024-06-03T01:20:48.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/21: dread - no filename 2024-06-03T01:20:48.638 INFO:tasks.workunit.client.1.smithi044.stdout:3/22: getdents . 0 2024-06-03T01:20:48.640 INFO:tasks.workunit.client.1.smithi044.stdout:3/23: link c0 c6 0 2024-06-03T01:20:48.640 INFO:tasks.workunit.client.1.smithi044.stdout:3/24: write - no filename 2024-06-03T01:20:48.642 INFO:tasks.workunit.client.1.smithi044.stdout:3/25: unlink l1 0 2024-06-03T01:20:48.644 INFO:tasks.workunit.client.1.smithi044.stdout:3/26: creat f7 x:0 0 0 2024-06-03T01:20:48.644 INFO:tasks.workunit.client.1.smithi044.stdout:3/27: truncate f7 100533 0 2024-06-03T01:20:48.651 INFO:tasks.workunit.client.1.smithi044.stdout:5/9: dwrite d2/f3 [0,4194304] 0 2024-06-03T01:20:48.651 INFO:tasks.workunit.client.1.smithi044.stdout:5/10: creat d2/f4 x:0 0 0 2024-06-03T01:20:48.651 INFO:tasks.workunit.client.1.smithi044.stdout:2/16: dwrite f0 [0,4194304] 0 2024-06-03T01:20:48.652 INFO:tasks.workunit.client.1.smithi044.stdout:6/62: dwrite d2/fe [4194304,4194304] 0 2024-06-03T01:20:48.652 INFO:tasks.workunit.client.1.smithi044.stdout:6/63: creat d2/d6/db/df/f1a x:0 0 0 2024-06-03T01:20:48.652 INFO:tasks.workunit.client.1.smithi044.stdout:7/158: dwrite d0/d3/f3f [0,4194304] 0 2024-06-03T01:20:48.654 INFO:tasks.workunit.client.1.smithi044.stdout:1/85: dwrite d0/d3/d20/d7/d13/f24 [0,4194304] 0 2024-06-03T01:20:48.655 INFO:tasks.workunit.client.1.smithi044.stdout:8/24: dwrite f2 [0,4194304] 0 2024-06-03T01:20:48.658 INFO:tasks.workunit.client.1.smithi044.stdout:7/159: mknod d0/d3/d30/c40 0 2024-06-03T01:20:48.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/9: dread f0 [0,4194304] 0 2024-06-03T01:20:48.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/10: fsync f0 0 2024-06-03T01:20:48.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/11: fsync f0 0 2024-06-03T01:20:48.663 INFO:tasks.workunit.client.1.smithi044.stdout:9/12: write f0 [411648,82601] 0 2024-06-03T01:20:48.670 INFO:tasks.workunit.client.1.smithi044.stdout:1/86: rename d0/d3/d20/d7/c22 to d0/d3/d20/d7/d13/c25 0 2024-06-03T01:20:48.675 INFO:tasks.workunit.client.1.smithi044.stdout:1/87: getdents d0/d3/d20 0 2024-06-03T01:20:48.684 INFO:tasks.workunit.client.1.smithi044.stdout:6/64: dwrite d2/d6/f13 [4194304,4194304] 0 2024-06-03T01:20:48.686 INFO:tasks.workunit.client.1.smithi044.stdout:0/56: dwrite d3/f8 [0,4194304] 0 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/57: stat d3/f4 0 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/58: creat d3/d9/f11 x:0 0 0 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/65: symlink d2/d6/db/l1b 0 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/66: write d2/d6/f13 [9428026,58220] 0 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/67: dread - d2/d6/db/df/f19 zero size 2024-06-03T01:20:48.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/68: creat d2/d6/db/df/f1c x:0 0 0 2024-06-03T01:20:48.693 INFO:tasks.workunit.client.1.smithi044.stdout:0/59: rename d3/d9/f11 to d3/f12 0 2024-06-03T01:20:48.695 INFO:tasks.workunit.client.1.smithi044.stdout:5/11: dwrite f1 [0,4194304] 0 2024-06-03T01:20:48.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/60: rename d3/f6 to d3/d9/f13 0 2024-06-03T01:20:48.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/61: readlink d3/la 0 2024-06-03T01:20:48.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/62: creat d3/f14 x:0 0 0 2024-06-03T01:20:48.696 INFO:tasks.workunit.client.1.smithi044.stdout:2/17: truncate f0 3864368 0 2024-06-03T01:20:48.698 INFO:tasks.workunit.client.1.smithi044.stdout:7/160: dwrite d0/d3/f2c [0,4194304] 0 2024-06-03T01:20:48.701 INFO:tasks.workunit.client.1.smithi044.stdout:0/63: rename d3/d9/c10 to d3/d9/c15 0 2024-06-03T01:20:48.703 INFO:tasks.workunit.client.1.smithi044.stdout:9/13: dwrite f0 [0,4194304] 0 2024-06-03T01:20:48.707 INFO:tasks.workunit.client.1.smithi044.stdout:7/161: symlink d0/d3/d8/l41 0 2024-06-03T01:20:48.707 INFO:tasks.workunit.client.1.smithi044.stdout:7/162: readlink d0/d3/d8/d32/d33/l36 0 2024-06-03T01:20:48.707 INFO:tasks.workunit.client.1.smithi044.stdout:1/88: dwrite d0/d3/f19 [0,4194304] 0 2024-06-03T01:20:48.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/64: dread d3/d9/f13 [0,4194304] 0 2024-06-03T01:20:48.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/65: dread - d3/fc zero size 2024-06-03T01:20:48.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/66: creat d3/d9/f16 x:0 0 0 2024-06-03T01:20:48.715 INFO:tasks.workunit.client.1.smithi044.stdout:1/89: creat d0/d16/f26 x:0 0 0 2024-06-03T01:20:48.718 INFO:tasks.workunit.client.1.smithi044.stdout:1/90: mknod d0/d3/c27 0 2024-06-03T01:20:48.718 INFO:tasks.workunit.client.1.smithi044.stdout:1/91: truncate d0/d3/d20/f1e 76272 0 2024-06-03T01:20:48.718 INFO:tasks.workunit.client.1.smithi044.stdout:1/92: write d0/d3/d20/d7/f17 [1047762,18567] 0 2024-06-03T01:20:48.718 INFO:tasks.workunit.client.1.smithi044.stdout:1/93: chown d0/d3/l11 801604 1 2024-06-03T01:20:48.726 INFO:tasks.workunit.client.1.smithi044.stdout:2/18: link f0 f2 0 2024-06-03T01:20:48.726 INFO:tasks.workunit.client.1.smithi044.stdout:9/14: dwrite f0 [0,4194304] 0 2024-06-03T01:20:48.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/12: dwrite f1 [0,4194304] 0 2024-06-03T01:20:48.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/13: chown f1 32098820 1 2024-06-03T01:20:48.727 INFO:tasks.workunit.client.1.smithi044.stdout:2/19: mknod c3 0 2024-06-03T01:20:48.728 INFO:tasks.workunit.client.1.smithi044.stdout:9/15: symlink l1 0 2024-06-03T01:20:48.730 INFO:tasks.workunit.client.1.smithi044.stdout:2/20: symlink l4 0 2024-06-03T01:20:48.730 INFO:tasks.workunit.client.1.smithi044.stdout:2/21: fdatasync f2 0 2024-06-03T01:20:48.730 INFO:tasks.workunit.client.1.smithi044.stdout:2/22: creat f5 x:0 0 0 2024-06-03T01:20:48.741 INFO:tasks.workunit.client.1.smithi044.stdout:5/14: write f1 [3544749,22216] 0 2024-06-03T01:20:48.747 INFO:tasks.workunit.client.1.smithi044.stdout:6/69: dwrite d2/d6/f13 [0,4194304] 0 2024-06-03T01:20:48.749 INFO:tasks.workunit.client.1.smithi044.stdout:6/70: getdents d2/d6 0 2024-06-03T01:20:48.749 INFO:tasks.workunit.client.1.smithi044.stdout:6/71: creat d2/d6/db/df/f1d x:0 0 0 2024-06-03T01:20:48.751 INFO:tasks.workunit.client.1.smithi044.stdout:6/72: mkdir d2/d6/db/df/d1e 0 2024-06-03T01:20:48.763 INFO:tasks.workunit.client.1.smithi044.stdout:7/163: dwrite d0/d3/fa [0,4194304] 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/73: read d2/d6/f13 [5458070,10053] 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/74: write d2/d6/f13 [5195240,30931] 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/75: read - d2/d6/db/df/d17/f18 zero size 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/76: creat d2/d6/db/df/f1f x:0 0 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/77: write d2/f14 [845670,99392] 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/78: fdatasync d2/d6/db/df/f19 0 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/79: chown d2/d6/cd 93048 1 2024-06-03T01:20:48.764 INFO:tasks.workunit.client.1.smithi044.stdout:7/164: truncate d0/d3/f18 1056254 0 2024-06-03T01:20:48.765 INFO:tasks.workunit.client.1.smithi044.stdout:7/165: fdatasync d0/d3/f2c 0 2024-06-03T01:20:48.765 INFO:tasks.workunit.client.1.smithi044.stdout:7/166: fdatasync d0/d3/f2c 0 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/80: link d2/d4/c10 d2/d6/c20 0 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/81: truncate d2/f14 1415668 0 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/82: read - d2/d6/db/df/f1c zero size 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/83: truncate d2/fe 8631613 0 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/84: chown d2/f14 497036815 1 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:0/67: dwrite d3/f4 [4194304,4194304] 0 2024-06-03T01:20:48.766 INFO:tasks.workunit.client.1.smithi044.stdout:7/167: creat d0/d3/d8/d32/d33/f42 x:0 0 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:1/94: dwrite d0/d3/f19 [0,4194304] 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/23: dwrite f2 [0,4194304] 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/24: rmdir - no directory 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:1/95: stat d0/c1c 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/15: dwrite d2/f3 [0,4194304] 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/25: creat f6 x:0 0 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/26: creat f7 x:0 0 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/27: stat l4 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/28: dread - f5 zero size 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/16: write f1 [3569638,115465] 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/17: stat d2/f4 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/18: stat d2/f4 0 2024-06-03T01:20:48.768 INFO:tasks.workunit.client.1.smithi044.stdout:9/16: dwrite f0 [4194304,4194304] 0 2024-06-03T01:20:48.770 INFO:tasks.workunit.client.1.smithi044.stdout:0/68: link d3/f8 d3/f17 0 2024-06-03T01:20:48.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/168: mknod d0/d3/d8/c43 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/19: symlink d2/l5 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/20: creat d2/f6 x:0 0 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/21: stat l0 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/22: fdatasync d2/f6 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:0/69: write d3/f17 [3063912,104294] 0 2024-06-03T01:20:48.775 INFO:tasks.workunit.client.1.smithi044.stdout:0/70: creat d3/f18 x:0 0 0 2024-06-03T01:20:48.778 INFO:tasks.workunit.client.1.smithi044.stdout:9/17: write f0 [6249700,42574] 0 2024-06-03T01:20:48.778 INFO:tasks.workunit.client.1.smithi044.stdout:9/18: readlink l1 0 2024-06-03T01:20:48.778 INFO:tasks.workunit.client.1.smithi044.stdout:9/19: readlink l1 0 2024-06-03T01:20:48.778 INFO:tasks.workunit.client.1.smithi044.stdout:2/29: write f2 [1135745,64202] 0 2024-06-03T01:20:48.788 INFO:tasks.workunit.client.1.smithi044.stdout:1/96: dwrite d0/f15 [0,4194304] 0 2024-06-03T01:20:48.788 INFO:tasks.workunit.client.1.smithi044.stdout:1/97: creat d0/d3/f28 x:0 0 0 2024-06-03T01:20:48.788 INFO:tasks.workunit.client.1.smithi044.stdout:9/20: fsync f0 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:2/30: dread f0 [0,4194304] 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/169: symlink d0/d21/l44 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/170: truncate d0/d21/f28 321409 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/171: fsync d0/d3/f2c 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/172: write d0/d16/f22 [225167,73138] 0 2024-06-03T01:20:48.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/85: dwrite d2/f14 [0,4194304] 0 2024-06-03T01:20:48.792 INFO:tasks.workunit.client.1.smithi044.stdout:5/23: mknod d2/c7 0 2024-06-03T01:20:48.792 INFO:tasks.workunit.client.1.smithi044.stdout:5/24: dread - d2/f6 zero size 2024-06-03T01:20:48.792 INFO:tasks.workunit.client.1.smithi044.stdout:5/25: chown d2/c7 18008804 1 2024-06-03T01:20:48.792 INFO:tasks.workunit.client.1.smithi044.stdout:5/26: truncate d2/f4 592305 0 2024-06-03T01:20:48.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/71: link d3/f17 d3/f19 0 2024-06-03T01:20:48.796 INFO:tasks.workunit.client.1.smithi044.stdout:0/72: creat d3/f1a x:0 0 0 2024-06-03T01:20:48.796 INFO:tasks.workunit.client.1.smithi044.stdout:0/73: stat d3/d9 0 2024-06-03T01:20:48.797 INFO:tasks.workunit.client.1.smithi044.stdout:1/98: dread d0/f15 [0,4194304] 0 2024-06-03T01:20:48.800 INFO:tasks.workunit.client.1.smithi044.stdout:9/21: write f0 [2388904,43941] 0 2024-06-03T01:20:48.801 INFO:tasks.workunit.client.1.smithi044.stdout:7/173: symlink d0/l45 0 2024-06-03T01:20:48.805 INFO:tasks.workunit.client.1.smithi044.stdout:6/86: creat d2/d6/db/f21 x:0 0 0 2024-06-03T01:20:48.806 INFO:tasks.workunit.client.1.smithi044.stdout:0/74: link d3/d9/cf d3/d9/c1b 0 2024-06-03T01:20:48.806 INFO:tasks.workunit.client.1.smithi044.stdout:5/27: dwrite d2/f3 [4194304,4194304] 0 2024-06-03T01:20:48.806 INFO:tasks.workunit.client.1.smithi044.stdout:1/99: unlink d0/d3/d20/f1e 0 2024-06-03T01:20:48.807 INFO:tasks.workunit.client.1.smithi044.stdout:2/31: getdents . 0 2024-06-03T01:20:48.808 INFO:tasks.workunit.client.1.smithi044.stdout:7/174: unlink d0/d16/c1a 0 2024-06-03T01:20:48.809 INFO:tasks.workunit.client.1.smithi044.stdout:7/175: write d0/d16/f3c [188175,1973] 0 2024-06-03T01:20:48.809 INFO:tasks.workunit.client.1.smithi044.stdout:6/87: mknod d2/d6/c22 0 2024-06-03T01:20:48.809 INFO:tasks.workunit.client.1.smithi044.stdout:9/22: dread f0 [0,4194304] 0 2024-06-03T01:20:48.809 INFO:tasks.workunit.client.1.smithi044.stdout:9/23: chown l1 3 1 2024-06-03T01:20:48.810 INFO:tasks.workunit.client.1.smithi044.stdout:0/75: mkdir d3/d1c 0 2024-06-03T01:20:48.811 INFO:tasks.workunit.client.1.smithi044.stdout:3/28: sync 2024-06-03T01:20:48.811 INFO:tasks.workunit.client.1.smithi044.stdout:4/27: sync 2024-06-03T01:20:48.811 INFO:tasks.workunit.client.1.smithi044.stdout:8/25: sync 2024-06-03T01:20:48.812 INFO:tasks.workunit.client.1.smithi044.stdout:1/100: unlink d0/d16/c18 0 2024-06-03T01:20:48.814 INFO:tasks.workunit.client.1.smithi044.stdout:7/176: truncate d0/d3/d8/f10 3315049 0 2024-06-03T01:20:48.818 INFO:tasks.workunit.client.1.smithi044.stdout:8/26: unlink f0 0 2024-06-03T01:20:48.820 INFO:tasks.workunit.client.1.smithi044.stdout:1/101: creat d0/f29 x:0 0 0 2024-06-03T01:20:48.822 INFO:tasks.workunit.client.1.smithi044.stdout:4/28: getdents . 0 2024-06-03T01:20:48.823 INFO:tasks.workunit.client.1.smithi044.stdout:1/102: symlink d0/l2a 0 2024-06-03T01:20:48.825 INFO:tasks.workunit.client.1.smithi044.stdout:3/29: rename c6 to c8 0 2024-06-03T01:20:48.825 INFO:tasks.workunit.client.1.smithi044.stdout:4/29: creat f4 x:0 0 0 2024-06-03T01:20:48.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/103: unlink d0/c1 0 2024-06-03T01:20:48.827 INFO:tasks.workunit.client.1.smithi044.stdout:3/30: fdatasync f7 0 2024-06-03T01:20:48.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/88: dread d2/d6/f13 [0,4194304] 0 2024-06-03T01:20:48.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/89: fdatasync d2/f14 0 2024-06-03T01:20:48.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/90: dread - d2/d6/db/df/f1a zero size 2024-06-03T01:20:48.828 INFO:tasks.workunit.client.1.smithi044.stdout:7/177: dread d0/d3/f2c [0,4194304] 0 2024-06-03T01:20:48.828 INFO:tasks.workunit.client.1.smithi044.stdout:4/30: unlink f4 0 2024-06-03T01:20:48.828 INFO:tasks.workunit.client.1.smithi044.stdout:3/31: dread f7 [0,4194304] 0 2024-06-03T01:20:48.829 INFO:tasks.workunit.client.1.smithi044.stdout:1/104: truncate d0/d3/d20/f1d 1067590 0 2024-06-03T01:20:48.829 INFO:tasks.workunit.client.1.smithi044.stdout:1/105: write d0/f29 [122571,87489] 0 2024-06-03T01:20:48.831 INFO:tasks.workunit.client.1.smithi044.stdout:3/32: truncate f7 71064 0 2024-06-03T01:20:48.833 INFO:tasks.workunit.client.1.smithi044.stdout:3/33: readlink l4 0 2024-06-03T01:20:48.835 INFO:tasks.workunit.client.1.smithi044.stdout:3/34: creat f9 x:0 0 0 2024-06-03T01:20:48.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/27: dread f2 [0,4194304] 0 2024-06-03T01:20:48.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/28: creat d5/f7 x:0 0 0 2024-06-03T01:20:48.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/29: read - d5/f6 zero size 2024-06-03T01:20:48.839 INFO:tasks.workunit.client.1.smithi044.stdout:3/35: mknod ca 0 2024-06-03T01:20:48.839 INFO:tasks.workunit.client.1.smithi044.stdout:3/36: chown ca 58 1 2024-06-03T01:20:48.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/28: dwrite d2/f6 [0,4194304] 0 2024-06-03T01:20:48.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/29: creat d2/f8 x:0 0 0 2024-06-03T01:20:48.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/30: stat d2/l5 0 2024-06-03T01:20:48.849 INFO:tasks.workunit.client.1.smithi044.stdout:0/76: getdents d3 0 2024-06-03T01:20:48.849 INFO:tasks.workunit.client.1.smithi044.stdout:0/77: write d3/f1a [731839,38558] 0 2024-06-03T01:20:48.854 INFO:tasks.workunit.client.1.smithi044.stdout:9/24: dwrite f0 [8388608,4194304] 0 2024-06-03T01:20:48.855 INFO:tasks.workunit.client.1.smithi044.stdout:2/32: dwrite f2 [0,4194304] 0 2024-06-03T01:20:48.855 INFO:tasks.workunit.client.1.smithi044.stdout:7/178: dwrite d0/d16/d23/f24 [0,4194304] 0 2024-06-03T01:20:48.855 INFO:tasks.workunit.client.1.smithi044.stdout:4/31: dwrite f3 [0,4194304] 0 2024-06-03T01:20:48.856 INFO:tasks.workunit.client.1.smithi044.stdout:6/91: dwrite d2/d6/db/df/f19 [0,4194304] 0 2024-06-03T01:20:48.860 INFO:tasks.workunit.client.1.smithi044.stdout:2/33: mknod c8 0 2024-06-03T01:20:48.860 INFO:tasks.workunit.client.1.smithi044.stdout:0/78: dread d3/f17 [0,4194304] 0 2024-06-03T01:20:48.860 INFO:tasks.workunit.client.1.smithi044.stdout:0/79: creat d3/d9/f1d x:0 0 0 2024-06-03T01:20:48.860 INFO:tasks.workunit.client.1.smithi044.stdout:0/80: dread - d3/fe zero size 2024-06-03T01:20:48.861 INFO:tasks.workunit.client.1.smithi044.stdout:0/81: readlink d3/la 0 2024-06-03T01:20:48.863 INFO:tasks.workunit.client.1.smithi044.stdout:4/32: dread f3 [0,4194304] 0 2024-06-03T01:20:48.866 INFO:tasks.workunit.client.1.smithi044.stdout:3/37: dread f7 [0,4194304] 0 2024-06-03T01:20:48.866 INFO:tasks.workunit.client.1.smithi044.stdout:3/38: write f9 [449666,112542] 0 2024-06-03T01:20:48.870 INFO:tasks.workunit.client.1.smithi044.stdout:8/30: getdents d5 0 2024-06-03T01:20:48.870 INFO:tasks.workunit.client.1.smithi044.stdout:8/31: chown d5/f7 629124621 1 2024-06-03T01:20:48.873 INFO:tasks.workunit.client.1.smithi044.stdout:8/32: read f2 [3548827,24185] 0 2024-06-03T01:20:48.877 INFO:tasks.workunit.client.1.smithi044.stdout:9/25: getdents . 0 2024-06-03T01:20:48.877 INFO:tasks.workunit.client.1.smithi044.stdout:9/26: readlink l1 0 2024-06-03T01:20:48.877 INFO:tasks.workunit.client.1.smithi044.stdout:4/33: dwrite f3 [0,4194304] 0 2024-06-03T01:20:48.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/34: rename c1 to c9 0 2024-06-03T01:20:48.880 INFO:tasks.workunit.client.1.smithi044.stdout:0/82: symlink d3/d9/l1e 0 2024-06-03T01:20:48.880 INFO:tasks.workunit.client.1.smithi044.stdout:0/83: creat d3/d9/f1f x:0 0 0 2024-06-03T01:20:48.880 INFO:tasks.workunit.client.1.smithi044.stdout:7/179: getdents d0/d21 0 2024-06-03T01:20:48.881 INFO:tasks.workunit.client.1.smithi044.stdout:7/180: read - d0/d3/d8/f2f zero size 2024-06-03T01:20:48.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/33: symlink d5/l8 0 2024-06-03T01:20:48.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/34: fdatasync f2 0 2024-06-03T01:20:48.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/35: write d5/f7 [858997,63854] 0 2024-06-03T01:20:48.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/36: stat f3 0 2024-06-03T01:20:48.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/27: creat f2 x:0 0 0 2024-06-03T01:20:48.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/28: dread - f2 zero size 2024-06-03T01:20:48.888 INFO:tasks.workunit.client.1.smithi044.stdout:4/34: creat f5 x:0 0 0 2024-06-03T01:20:48.888 INFO:tasks.workunit.client.1.smithi044.stdout:4/35: chown f5 91 1 2024-06-03T01:20:48.889 INFO:tasks.workunit.client.1.smithi044.stdout:2/35: rename f5 to fa 0 2024-06-03T01:20:48.891 INFO:tasks.workunit.client.1.smithi044.stdout:0/84: mkdir d3/d1c/d20 0 2024-06-03T01:20:48.891 INFO:tasks.workunit.client.1.smithi044.stdout:0/85: write d3/f1a [401395,92030] 0 2024-06-03T01:20:48.891 INFO:tasks.workunit.client.1.smithi044.stdout:3/39: symlink lb 0 2024-06-03T01:20:48.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/37: truncate f2 3205328 0 2024-06-03T01:20:48.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/92: rename d2/d6/db/df/d17 to d2/d23 0 2024-06-03T01:20:48.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/93: chown d2/d23/f18 832 1 2024-06-03T01:20:48.898 INFO:tasks.workunit.client.1.smithi044.stdout:2/36: mkdir db 0 2024-06-03T01:20:48.898 INFO:tasks.workunit.client.1.smithi044.stdout:2/37: stat f0 0 2024-06-03T01:20:48.898 INFO:tasks.workunit.client.1.smithi044.stdout:2/38: dread - fa zero size 2024-06-03T01:20:48.902 INFO:tasks.workunit.client.1.smithi044.stdout:7/181: rename d0/f2 to d0/d21/f46 0 2024-06-03T01:20:48.902 INFO:tasks.workunit.client.1.smithi044.stdout:7/182: fsync d0/d3/d8/f39 0 2024-06-03T01:20:48.904 INFO:tasks.workunit.client.1.smithi044.stdout:4/36: mknod c6 0 2024-06-03T01:20:48.909 INFO:tasks.workunit.client.1.smithi044.stdout:0/86: dwrite d3/f14 [0,4194304] 0 2024-06-03T01:20:48.909 INFO:tasks.workunit.client.1.smithi044.stdout:2/39: link c3 db/cc 0 2024-06-03T01:20:48.909 INFO:tasks.workunit.client.1.smithi044.stdout:2/40: stat f7 0 2024-06-03T01:20:48.909 INFO:tasks.workunit.client.1.smithi044.stdout:3/40: link lb lc 0 2024-06-03T01:20:48.910 INFO:tasks.workunit.client.1.smithi044.stdout:7/183: creat d0/d3/f47 x:0 0 0 2024-06-03T01:20:48.912 INFO:tasks.workunit.client.1.smithi044.stdout:9/29: rename l1 to l3 0 2024-06-03T01:20:48.912 INFO:tasks.workunit.client.1.smithi044.stdout:4/37: chown c0 70442 1 2024-06-03T01:20:48.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/41: creat fd x:0 0 0 2024-06-03T01:20:48.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/42: rmdir - no directory 2024-06-03T01:20:48.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/43: chown fd 126232 1 2024-06-03T01:20:48.917 INFO:tasks.workunit.client.1.smithi044.stdout:9/30: fdatasync f0 0 2024-06-03T01:20:48.919 INFO:tasks.workunit.client.1.smithi044.stdout:7/184: getdents d0 0 2024-06-03T01:20:48.919 INFO:tasks.workunit.client.1.smithi044.stdout:7/185: creat d0/d3/d8/d32/f48 x:0 0 0 2024-06-03T01:20:48.919 INFO:tasks.workunit.client.1.smithi044.stdout:7/186: chown d0/d16/f22 20515382 1 2024-06-03T01:20:48.920 INFO:tasks.workunit.client.1.smithi044.stdout:9/31: creat f4 x:0 0 0 2024-06-03T01:20:48.921 INFO:tasks.workunit.client.1.smithi044.stdout:4/38: mknod c7 0 2024-06-03T01:20:48.921 INFO:tasks.workunit.client.1.smithi044.stdout:0/87: dread d3/f4 [4194304,4194304] 0 2024-06-03T01:20:48.923 INFO:tasks.workunit.client.1.smithi044.stdout:7/187: symlink d0/d16/d23/l49 0 2024-06-03T01:20:48.923 INFO:tasks.workunit.client.1.smithi044.stdout:7/188: creat d0/d3/d8/d32/f4a x:0 0 0 2024-06-03T01:20:48.924 INFO:tasks.workunit.client.1.smithi044.stdout:7/189: truncate d0/d3/d8/d32/f4a 131642 0 2024-06-03T01:20:48.926 INFO:tasks.workunit.client.1.smithi044.stdout:0/88: dread d3/f1a [0,4194304] 0 2024-06-03T01:20:48.927 INFO:tasks.workunit.client.1.smithi044.stdout:7/190: symlink d0/d16/d23/l4b 0 2024-06-03T01:20:48.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/39: mknod c8 0 2024-06-03T01:20:48.928 INFO:tasks.workunit.client.1.smithi044.stdout:9/32: dread f0 [4194304,4194304] 0 2024-06-03T01:20:48.935 INFO:tasks.workunit.client.1.smithi044.stdout:7/191: dread d0/f4 [0,4194304] 0 2024-06-03T01:20:48.941 INFO:tasks.workunit.client.1.smithi044.stdout:4/40: write f3 [3802039,117195] 0 2024-06-03T01:20:48.942 INFO:tasks.workunit.client.1.smithi044.stdout:0/89: dread d3/d9/fb [0,4194304] 0 2024-06-03T01:20:48.945 INFO:tasks.workunit.client.1.smithi044.stdout:8/38: dwrite f2 [0,4194304] 0 2024-06-03T01:20:48.945 INFO:tasks.workunit.client.1.smithi044.stdout:8/39: rename d5 to d5/d9 22 2024-06-03T01:20:48.945 INFO:tasks.workunit.client.1.smithi044.stdout:3/44: dwrite f9 [0,4194304] 0 2024-06-03T01:20:48.946 INFO:tasks.workunit.client.1.smithi044.stdout:2/41: dwrite f0 [0,4194304] 0 2024-06-03T01:20:48.947 INFO:tasks.workunit.client.1.smithi044.stdout:8/40: dread f3 [0,4194304] 0 2024-06-03T01:20:48.947 INFO:tasks.workunit.client.1.smithi044.stdout:0/90: dread d3/f8 [0,4194304] 0 2024-06-03T01:20:48.950 INFO:tasks.workunit.client.1.smithi044.stdout:0/91: mkdir d3/d1c/d21 0 2024-06-03T01:20:48.951 INFO:tasks.workunit.client.1.smithi044.stdout:3/45: link f7 fe 0 2024-06-03T01:20:48.951 INFO:tasks.workunit.client.1.smithi044.stdout:7/192: rmdir d0/d3 39 2024-06-03T01:20:48.952 INFO:tasks.workunit.client.1.smithi044.stdout:3/46: dread f7 [0,4194304] 0 2024-06-03T01:20:48.953 INFO:tasks.workunit.client.1.smithi044.stdout:7/193: mkdir d0/d21/d27/d4c 0 2024-06-03T01:20:48.954 INFO:tasks.workunit.client.1.smithi044.stdout:9/33: dwrite f4 [0,4194304] 0 2024-06-03T01:20:48.954 INFO:tasks.workunit.client.1.smithi044.stdout:7/194: dread - d0/d3/d8/f39 zero size 2024-06-03T01:20:48.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/92: dread d3/f17 [0,4194304] 0 2024-06-03T01:20:48.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/93: chown d3/d9/c1b 60074 1 2024-06-03T01:20:48.956 INFO:tasks.workunit.client.1.smithi044.stdout:3/47: mkdir df 0 2024-06-03T01:20:48.957 INFO:tasks.workunit.client.1.smithi044.stdout:2/42: read f0 [3885994,13517] 0 2024-06-03T01:20:48.957 INFO:tasks.workunit.client.1.smithi044.stdout:2/43: write f0 [700457,94484] 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:0/94: unlink d3/f1a 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:0/95: getdents d3/d1c/d21 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:0/96: write d3/d9/f13 [1256718,40334] 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:0/97: creat d3/d9/f22 x:0 0 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:4/41: dread f3 [0,4194304] 0 2024-06-03T01:20:48.959 INFO:tasks.workunit.client.1.smithi044.stdout:8/41: write f3 [357440,88336] 0 2024-06-03T01:20:48.960 INFO:tasks.workunit.client.1.smithi044.stdout:3/48: creat df/f10 x:0 0 0 2024-06-03T01:20:48.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/44: mkdir db/dd 0 2024-06-03T01:20:48.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/45: write f7 [155968,111995] 0 2024-06-03T01:20:48.965 INFO:tasks.workunit.client.1.smithi044.stdout:9/34: read f4 [923970,63364] 0 2024-06-03T01:20:48.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/98: unlink d3/d9/f16 0 2024-06-03T01:20:48.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/99: chown d3/f4 311457058 1 2024-06-03T01:20:48.965 INFO:tasks.workunit.client.1.smithi044.stdout:4/42: fdatasync f2 0 2024-06-03T01:20:48.965 INFO:tasks.workunit.client.1.smithi044.stdout:4/43: readlink - no filename 2024-06-03T01:20:48.966 INFO:tasks.workunit.client.1.smithi044.stdout:8/42: symlink d5/la 0 2024-06-03T01:20:48.966 INFO:tasks.workunit.client.1.smithi044.stdout:8/43: chown c4 993679 1 2024-06-03T01:20:48.966 INFO:tasks.workunit.client.1.smithi044.stdout:8/44: write f3 [985256,68893] 0 2024-06-03T01:20:48.968 INFO:tasks.workunit.client.1.smithi044.stdout:0/100: creat d3/f23 x:0 0 0 2024-06-03T01:20:48.968 INFO:tasks.workunit.client.1.smithi044.stdout:0/101: read - d3/f23 zero size 2024-06-03T01:20:48.968 INFO:tasks.workunit.client.1.smithi044.stdout:0/102: chown d3/f4 13402 1 2024-06-03T01:20:48.969 INFO:tasks.workunit.client.1.smithi044.stdout:3/49: write f9 [3903116,76861] 0 2024-06-03T01:20:48.970 INFO:tasks.workunit.client.1.smithi044.stdout:9/35: mknod c5 0 2024-06-03T01:20:48.971 INFO:tasks.workunit.client.1.smithi044.stdout:0/103: creat d3/d1c/f24 x:0 0 0 2024-06-03T01:20:48.972 INFO:tasks.workunit.client.1.smithi044.stdout:4/44: symlink l9 0 2024-06-03T01:20:48.974 INFO:tasks.workunit.client.1.smithi044.stdout:0/104: mkdir d3/d1c/d20/d25 0 2024-06-03T01:20:48.975 INFO:tasks.workunit.client.1.smithi044.stdout:9/36: dread f4 [0,4194304] 0 2024-06-03T01:20:48.976 INFO:tasks.workunit.client.1.smithi044.stdout:0/105: rename d3/f12 to d3/d1c/d21/f26 0 2024-06-03T01:20:48.976 INFO:tasks.workunit.client.1.smithi044.stdout:0/106: chown d3/d1c/d20/d25 0 1 2024-06-03T01:20:48.976 INFO:tasks.workunit.client.1.smithi044.stdout:0/107: chown d3/d9/f13 6958 1 2024-06-03T01:20:48.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/45: mknod ca 0 2024-06-03T01:20:48.977 INFO:tasks.workunit.client.1.smithi044.stdout:9/37: truncate f0 9833742 0 2024-06-03T01:20:48.977 INFO:tasks.workunit.client.1.smithi044.stdout:9/38: write f2 [350286,118268] 0 2024-06-03T01:20:48.978 INFO:tasks.workunit.client.1.smithi044.stdout:4/46: dread f2 [0,4194304] 0 2024-06-03T01:20:48.979 INFO:tasks.workunit.client.1.smithi044.stdout:4/47: dread - f5 zero size 2024-06-03T01:20:48.979 INFO:tasks.workunit.client.1.smithi044.stdout:4/48: rmdir - no directory 2024-06-03T01:20:48.979 INFO:tasks.workunit.client.1.smithi044.stdout:3/50: dread f9 [0,4194304] 0 2024-06-03T01:20:48.979 INFO:tasks.workunit.client.1.smithi044.stdout:0/108: mkdir d3/d1c/d27 0 2024-06-03T01:20:48.979 INFO:tasks.workunit.client.1.smithi044.stdout:0/109: creat d3/d9/f28 x:0 0 0 2024-06-03T01:20:48.980 INFO:tasks.workunit.client.1.smithi044.stdout:3/51: stat c8 0 2024-06-03T01:20:48.980 INFO:tasks.workunit.client.1.smithi044.stdout:3/52: dread - df/f10 zero size 2024-06-03T01:20:48.981 INFO:tasks.workunit.client.1.smithi044.stdout:0/110: mkdir d3/d1c/d27/d29 0 2024-06-03T01:20:48.982 INFO:tasks.workunit.client.1.smithi044.stdout:0/111: creat d3/d1c/f2a x:0 0 0 2024-06-03T01:20:48.982 INFO:tasks.workunit.client.1.smithi044.stdout:9/39: mkdir d6 0 2024-06-03T01:20:48.984 INFO:tasks.workunit.client.1.smithi044.stdout:0/112: link d3/d9/l1e d3/d9/l2b 0 2024-06-03T01:20:48.984 INFO:tasks.workunit.client.1.smithi044.stdout:7/195: dwrite d0/d3/d8/f14 [0,4194304] 0 2024-06-03T01:20:48.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/196: truncate d0/d21/f28 369863 0 2024-06-03T01:20:48.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/197: chown d0/d3/f47 29394 1 2024-06-03T01:20:48.985 INFO:tasks.workunit.client.1.smithi044.stdout:9/40: mknod d6/c7 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:0/113: unlink d3/fc 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:2/46: dwrite f6 [0,4194304] 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:2/47: stat f0 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:2/48: getdents db/dd 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:2/49: stat f6 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:2/50: dread - fa zero size 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:0/114: creat d3/f2c x:0 0 0 2024-06-03T01:20:48.988 INFO:tasks.workunit.client.1.smithi044.stdout:0/115: dread - d3/d9/f1d zero size 2024-06-03T01:20:48.991 INFO:tasks.workunit.client.1.smithi044.stdout:9/41: mkdir d6/d8 0 2024-06-03T01:20:48.999 INFO:tasks.workunit.client.1.smithi044.stdout:3/53: dwrite f9 [0,4194304] 0 2024-06-03T01:20:49.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/54: chown c8 5 1 2024-06-03T01:20:49.000 INFO:tasks.workunit.client.1.smithi044.stdout:2/51: dread f6 [0,4194304] 0 2024-06-03T01:20:49.003 INFO:tasks.workunit.client.1.smithi044.stdout:8/45: truncate f3 155253 0 2024-06-03T01:20:49.003 INFO:tasks.workunit.client.1.smithi044.stdout:8/46: fsync d5/f6 0 2024-06-03T01:20:49.003 INFO:tasks.workunit.client.1.smithi044.stdout:7/198: dwrite d0/d21/f28 [0,4194304] 0 2024-06-03T01:20:49.004 INFO:tasks.workunit.client.1.smithi044.stdout:7/199: chown d0/d16/d23/c3a 0 1 2024-06-03T01:20:49.007 INFO:tasks.workunit.client.1.smithi044.stdout:3/55: rename f7 to df/f11 0 2024-06-03T01:20:49.007 INFO:tasks.workunit.client.1.smithi044.stdout:3/56: fdatasync fe 0 2024-06-03T01:20:49.010 INFO:tasks.workunit.client.1.smithi044.stdout:9/42: dwrite f4 [0,4194304] 0 2024-06-03T01:20:49.010 INFO:tasks.workunit.client.1.smithi044.stdout:4/49: getdents . 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:4/50: fdatasync f3 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:2/52: creat db/dd/fe x:0 0 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/47: creat d5/fb x:0 0 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/48: fdatasync d5/f7 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/49: write d5/fb [728980,102756] 0 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/50: chown c4 132466 1 2024-06-03T01:20:49.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/51: write f2 [5028063,69573] 0 2024-06-03T01:20:49.014 INFO:tasks.workunit.client.1.smithi044.stdout:3/57: chown c5 1058615 1 2024-06-03T01:20:49.017 INFO:tasks.workunit.client.1.smithi044.stdout:4/51: creat fb x:0 0 0 2024-06-03T01:20:49.018 INFO:tasks.workunit.client.1.smithi044.stdout:7/200: dwrite d0/d3/fa [4194304,4194304] 0 2024-06-03T01:20:49.018 INFO:tasks.workunit.client.1.smithi044.stdout:8/52: link d5/la d5/lc 0 2024-06-03T01:20:49.019 INFO:tasks.workunit.client.1.smithi044.stdout:8/53: write f2 [4220924,16178] 0 2024-06-03T01:20:49.019 INFO:tasks.workunit.client.1.smithi044.stdout:8/54: readlink d5/l8 0 2024-06-03T01:20:49.019 INFO:tasks.workunit.client.1.smithi044.stdout:8/55: stat d5/l8 0 2024-06-03T01:20:49.019 INFO:tasks.workunit.client.1.smithi044.stdout:8/56: write d5/f6 [378708,38775] 0 2024-06-03T01:20:49.020 INFO:tasks.workunit.client.1.smithi044.stdout:3/58: mknod df/c12 0 2024-06-03T01:20:49.020 INFO:tasks.workunit.client.1.smithi044.stdout:3/59: write df/f10 [25906,107284] 0 2024-06-03T01:20:49.021 INFO:tasks.workunit.client.1.smithi044.stdout:9/43: rename f0 to d6/d8/f9 0 2024-06-03T01:20:49.022 INFO:tasks.workunit.client.1.smithi044.stdout:9/44: write f2 [227792,305] 0 2024-06-03T01:20:49.023 INFO:tasks.workunit.client.1.smithi044.stdout:4/52: rename f3 to fc 0 2024-06-03T01:20:49.025 INFO:tasks.workunit.client.1.smithi044.stdout:0/116: truncate d3/d9/f13 2786510 0 2024-06-03T01:20:49.028 INFO:tasks.workunit.client.1.smithi044.stdout:7/201: getdents d0/d21 0 2024-06-03T01:20:49.030 INFO:tasks.workunit.client.1.smithi044.stdout:3/60: mkdir df/d13 0 2024-06-03T01:20:49.031 INFO:tasks.workunit.client.1.smithi044.stdout:2/53: rename c9 to db/cf 0 2024-06-03T01:20:49.031 INFO:tasks.workunit.client.1.smithi044.stdout:2/54: write f2 [231996,70406] 0 2024-06-03T01:20:49.034 INFO:tasks.workunit.client.1.smithi044.stdout:2/55: dread f6 [0,4194304] 0 2024-06-03T01:20:49.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/53: link fb fd 0 2024-06-03T01:20:49.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/54: write fd [995338,89130] 0 2024-06-03T01:20:49.037 INFO:tasks.workunit.client.1.smithi044.stdout:7/202: chown d0/d3/c1c 55 1 2024-06-03T01:20:49.037 INFO:tasks.workunit.client.1.smithi044.stdout:7/203: truncate d0/d3/d8/f10 3339036 0 2024-06-03T01:20:49.037 INFO:tasks.workunit.client.1.smithi044.stdout:7/204: creat d0/d16/f4d x:0 0 0 2024-06-03T01:20:49.038 INFO:tasks.workunit.client.1.smithi044.stdout:3/61: mknod df/d13/c14 0 2024-06-03T01:20:49.039 INFO:tasks.workunit.client.1.smithi044.stdout:3/62: read df/f11 [61837,27396] 0 2024-06-03T01:20:49.039 INFO:tasks.workunit.client.1.smithi044.stdout:3/63: readlink l4 0 2024-06-03T01:20:49.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/45: rename l3 to d6/la 0 2024-06-03T01:20:49.040 INFO:tasks.workunit.client.1.smithi044.stdout:8/57: rmdir d5 39 2024-06-03T01:20:49.041 INFO:tasks.workunit.client.1.smithi044.stdout:2/56: rmdir db 39 2024-06-03T01:20:49.042 INFO:tasks.workunit.client.1.smithi044.stdout:4/55: symlink le 0 2024-06-03T01:20:49.042 INFO:tasks.workunit.client.1.smithi044.stdout:4/56: chown c6 28768 1 2024-06-03T01:20:49.042 INFO:tasks.workunit.client.1.smithi044.stdout:4/57: creat ff x:0 0 0 2024-06-03T01:20:49.044 INFO:tasks.workunit.client.1.smithi044.stdout:7/205: symlink d0/d3/d8/l4e 0 2024-06-03T01:20:49.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/117: rename d3/d9/fb to d3/d1c/d27/d29/f2d 0 2024-06-03T01:20:49.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/118: stat d3/f19 0 2024-06-03T01:20:49.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/119: write d3/d9/f13 [49458,60156] 0 2024-06-03T01:20:49.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/120: creat d3/d1c/f2e x:0 0 0 2024-06-03T01:20:49.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/121: creat d3/d9/f2f x:0 0 0 2024-06-03T01:20:49.048 INFO:tasks.workunit.client.1.smithi044.stdout:9/46: mkdir d6/d8/db 0 2024-06-03T01:20:49.048 INFO:tasks.workunit.client.1.smithi044.stdout:9/47: chown d6/d8/db 9410 1 2024-06-03T01:20:49.048 INFO:tasks.workunit.client.1.smithi044.stdout:2/57: truncate f6 2141699 0 2024-06-03T01:20:49.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/58: dread f2 [0,4194304] 0 2024-06-03T01:20:49.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/59: truncate d5/f7 1043014 0 2024-06-03T01:20:49.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/60: chown d5/f6 9 1 2024-06-03T01:20:49.050 INFO:tasks.workunit.client.1.smithi044.stdout:4/58: symlink l10 0 2024-06-03T01:20:49.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/206: unlink d0/d3/d8/c19 0 2024-06-03T01:20:49.057 INFO:tasks.workunit.client.1.smithi044.stdout:2/58: dread f2 [0,4194304] 0 2024-06-03T01:20:49.059 INFO:tasks.workunit.client.1.smithi044.stdout:7/207: write d0/d3/f3f [2859913,5007] 0 2024-06-03T01:20:49.068 INFO:tasks.workunit.client.1.smithi044.stdout:5/31: sync 2024-06-03T01:20:49.068 INFO:tasks.workunit.client.1.smithi044.stdout:1/106: sync 2024-06-03T01:20:49.068 INFO:tasks.workunit.client.1.smithi044.stdout:5/32: chown d2/f6 7273235 1 2024-06-03T01:20:49.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/33: dread d2/f6 [0,4194304] 0 2024-06-03T01:20:49.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/34: chown d2 115450 1 2024-06-03T01:20:49.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/35: stat d2/c7 0 2024-06-03T01:20:49.077 INFO:tasks.workunit.client.1.smithi044.stdout:5/36: dread d2/f4 [0,4194304] 0 2024-06-03T01:20:49.099 INFO:tasks.workunit.client.1.smithi044.stdout:6/94: sync 2024-06-03T01:20:49.099 INFO:tasks.workunit.client.1.smithi044.stdout:6/95: chown d2/d4/c10 3907518 1 2024-06-03T01:20:49.100 INFO:tasks.workunit.client.1.smithi044.stdout:0/122: creat d3/d1c/d20/d25/f30 x:0 0 0 2024-06-03T01:20:49.100 INFO:tasks.workunit.client.1.smithi044.stdout:2/59: dwrite fa [0,4194304] 0 2024-06-03T01:20:49.100 INFO:tasks.workunit.client.1.smithi044.stdout:2/60: chown f6 111399828 1 2024-06-03T01:20:49.101 INFO:tasks.workunit.client.1.smithi044.stdout:2/61: write f7 [928713,121373] 0 2024-06-03T01:20:49.101 INFO:tasks.workunit.client.1.smithi044.stdout:9/48: mknod d6/d8/cc 0 2024-06-03T01:20:49.103 INFO:tasks.workunit.client.1.smithi044.stdout:6/96: dread d2/d4/f7 [4194304,4194304] 0 2024-06-03T01:20:49.103 INFO:tasks.workunit.client.1.smithi044.stdout:6/97: fdatasync d2/d6/db/df/f1c 0 2024-06-03T01:20:49.103 INFO:tasks.workunit.client.1.smithi044.stdout:6/98: chown d2/f14 1796986 1 2024-06-03T01:20:49.103 INFO:tasks.workunit.client.1.smithi044.stdout:8/61: creat d5/fd x:0 0 0 2024-06-03T01:20:49.103 INFO:tasks.workunit.client.1.smithi044.stdout:8/62: creat d5/fe x:0 0 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:8/63: read d5/fb [521374,16326] 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:8/64: readlink l1 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/64: getdents df/d13 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/65: creat df/f15 x:0 0 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/66: readlink lc 0 2024-06-03T01:20:49.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/67: write df/f11 [676653,101011] 0 2024-06-03T01:20:49.110 INFO:tasks.workunit.client.1.smithi044.stdout:0/123: symlink d3/d1c/d27/d29/l31 0 2024-06-03T01:20:49.116 INFO:tasks.workunit.client.1.smithi044.stdout:6/99: mknod d2/d6/db/df/d1e/c24 0 2024-06-03T01:20:49.118 INFO:tasks.workunit.client.1.smithi044.stdout:4/59: getdents . 0 2024-06-03T01:20:49.121 INFO:tasks.workunit.client.1.smithi044.stdout:7/208: rename d0/d3/f2c to d0/d3/f4f 0 2024-06-03T01:20:49.121 INFO:tasks.workunit.client.1.smithi044.stdout:7/209: dread - d0/d3/d8/f37 zero size 2024-06-03T01:20:49.121 INFO:tasks.workunit.client.1.smithi044.stdout:7/210: chown d0/d21/l44 1 1 2024-06-03T01:20:49.121 INFO:tasks.workunit.client.1.smithi044.stdout:9/49: getdents d6/d8 0 2024-06-03T01:20:49.122 INFO:tasks.workunit.client.1.smithi044.stdout:1/107: creat d0/d3/d20/d7/f2b x:0 0 0 2024-06-03T01:20:49.123 INFO:tasks.workunit.client.1.smithi044.stdout:5/37: rmdir d2 39 2024-06-03T01:20:49.125 INFO:tasks.workunit.client.1.smithi044.stdout:0/124: unlink d3/d1c/f2a 0 2024-06-03T01:20:49.126 INFO:tasks.workunit.client.1.smithi044.stdout:2/62: getdents db/dd 0 2024-06-03T01:20:49.127 INFO:tasks.workunit.client.1.smithi044.stdout:6/100: mknod d2/d4/c25 0 2024-06-03T01:20:49.127 INFO:tasks.workunit.client.1.smithi044.stdout:6/101: dread - d2/d23/f18 zero size 2024-06-03T01:20:49.130 INFO:tasks.workunit.client.1.smithi044.stdout:5/38: getdents d2 0 2024-06-03T01:20:49.132 INFO:tasks.workunit.client.1.smithi044.stdout:6/102: truncate d2/d6/f13 3639508 0 2024-06-03T01:20:49.133 INFO:tasks.workunit.client.1.smithi044.stdout:6/103: write d2/d6/db/df/f19 [4502471,39645] 0 2024-06-03T01:20:49.133 INFO:tasks.workunit.client.1.smithi044.stdout:8/65: dwrite d5/fb [0,4194304] 0 2024-06-03T01:20:49.134 INFO:tasks.workunit.client.1.smithi044.stdout:7/211: dread d0/d3/fa [4194304,4194304] 0 2024-06-03T01:20:49.134 INFO:tasks.workunit.client.1.smithi044.stdout:3/68: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/69: getdents df/d13 0 2024-06-03T01:20:49.135 INFO:tasks.workunit.client.1.smithi044.stdout:5/39: creat d2/f9 x:0 0 0 2024-06-03T01:20:49.140 INFO:tasks.workunit.client.1.smithi044.stdout:6/104: rename d2/d6/c22 to d2/d6/db/df/c26 0 2024-06-03T01:20:49.145 INFO:tasks.workunit.client.1.smithi044.stdout:8/66: link d5/fd d5/ff 0 2024-06-03T01:20:49.146 INFO:tasks.workunit.client.1.smithi044.stdout:8/67: stat f3 0 2024-06-03T01:20:49.146 INFO:tasks.workunit.client.1.smithi044.stdout:8/68: write d5/f7 [753199,75932] 0 2024-06-03T01:20:49.147 INFO:tasks.workunit.client.1.smithi044.stdout:7/212: creat d0/d21/d27/f50 x:0 0 0 2024-06-03T01:20:49.151 INFO:tasks.workunit.client.1.smithi044.stdout:4/60: truncate fb 509017 0 2024-06-03T01:20:49.152 INFO:tasks.workunit.client.1.smithi044.stdout:5/40: creat d2/fa x:0 0 0 2024-06-03T01:20:49.153 INFO:tasks.workunit.client.1.smithi044.stdout:6/105: creat d2/f27 x:0 0 0 2024-06-03T01:20:49.153 INFO:tasks.workunit.client.1.smithi044.stdout:9/50: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:49.153 INFO:tasks.workunit.client.1.smithi044.stdout:9/51: chown f4 1 1 2024-06-03T01:20:49.154 INFO:tasks.workunit.client.1.smithi044.stdout:9/52: truncate d6/d8/f9 9839183 0 2024-06-03T01:20:49.154 INFO:tasks.workunit.client.1.smithi044.stdout:3/70: dwrite df/f10 [0,4194304] 0 2024-06-03T01:20:49.154 INFO:tasks.workunit.client.1.smithi044.stdout:0/125: truncate d3/f17 1671991 0 2024-06-03T01:20:49.154 INFO:tasks.workunit.client.1.smithi044.stdout:0/126: dread - d3/d1c/f2e zero size 2024-06-03T01:20:49.155 INFO:tasks.workunit.client.1.smithi044.stdout:0/127: fsync d3/d1c/f24 0 2024-06-03T01:20:49.155 INFO:tasks.workunit.client.1.smithi044.stdout:9/53: dread f2 [0,4194304] 0 2024-06-03T01:20:49.155 INFO:tasks.workunit.client.1.smithi044.stdout:2/63: truncate f2 1000804 0 2024-06-03T01:20:49.156 INFO:tasks.workunit.client.1.smithi044.stdout:8/69: mkdir d5/d10 0 2024-06-03T01:20:49.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/213: getdents d0/d3 0 2024-06-03T01:20:49.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/214: write d0/d16/f4d [799964,17829] 0 2024-06-03T01:20:49.159 INFO:tasks.workunit.client.1.smithi044.stdout:4/61: mknod c11 0 2024-06-03T01:20:49.161 INFO:tasks.workunit.client.1.smithi044.stdout:6/106: creat d2/d6/f28 x:0 0 0 2024-06-03T01:20:49.162 INFO:tasks.workunit.client.1.smithi044.stdout:3/71: dread f9 [0,4194304] 0 2024-06-03T01:20:49.164 INFO:tasks.workunit.client.1.smithi044.stdout:9/54: read f4 [4332887,72249] 0 2024-06-03T01:20:49.167 INFO:tasks.workunit.client.1.smithi044.stdout:0/128: mknod d3/d1c/d20/c32 0 2024-06-03T01:20:49.167 INFO:tasks.workunit.client.1.smithi044.stdout:0/129: creat d3/d1c/d20/d25/f33 x:0 0 0 2024-06-03T01:20:49.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/64: symlink db/l10 0 2024-06-03T01:20:49.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/65: write f7 [1729891,45515] 0 2024-06-03T01:20:49.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/66: chown db/cf 158 1 2024-06-03T01:20:49.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/67: write fa [4748723,40479] 0 2024-06-03T01:20:49.169 INFO:tasks.workunit.client.1.smithi044.stdout:8/70: link d5/f6 d5/f11 0 2024-06-03T01:20:49.170 INFO:tasks.workunit.client.1.smithi044.stdout:7/215: rmdir d0/d3/d30 39 2024-06-03T01:20:49.170 INFO:tasks.workunit.client.1.smithi044.stdout:7/216: dread - d0/d3/f47 zero size 2024-06-03T01:20:49.170 INFO:tasks.workunit.client.1.smithi044.stdout:3/72: dread fd [0,4194304] 0 2024-06-03T01:20:49.171 INFO:tasks.workunit.client.1.smithi044.stdout:4/62: mknod c12 0 2024-06-03T01:20:49.172 INFO:tasks.workunit.client.1.smithi044.stdout:5/41: mknod d2/cb 0 2024-06-03T01:20:49.174 INFO:tasks.workunit.client.1.smithi044.stdout:2/68: dread f6 [0,4194304] 0 2024-06-03T01:20:49.174 INFO:tasks.workunit.client.1.smithi044.stdout:6/107: unlink d2/d4/cc 0 2024-06-03T01:20:49.174 INFO:tasks.workunit.client.1.smithi044.stdout:6/108: dread - d2/d6/db/df/f1f zero size 2024-06-03T01:20:49.175 INFO:tasks.workunit.client.1.smithi044.stdout:8/71: read f2 [2611361,37835] 0 2024-06-03T01:20:49.176 INFO:tasks.workunit.client.1.smithi044.stdout:9/55: mkdir d6/dd 0 2024-06-03T01:20:49.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/56: write d6/d8/f9 [4998703,90932] 0 2024-06-03T01:20:49.182 INFO:tasks.workunit.client.1.smithi044.stdout:8/72: dread f3 [0,4194304] 0 2024-06-03T01:20:49.182 INFO:tasks.workunit.client.1.smithi044.stdout:8/73: chown d5/d10 7610051 1 2024-06-03T01:20:49.182 INFO:tasks.workunit.client.1.smithi044.stdout:8/74: getdents d5/d10 0 2024-06-03T01:20:49.182 INFO:tasks.workunit.client.1.smithi044.stdout:8/75: write d5/fd [89358,54184] 0 2024-06-03T01:20:49.186 INFO:tasks.workunit.client.1.smithi044.stdout:7/217: mkdir d0/d3/d8/d32/d51 0 2024-06-03T01:20:49.187 INFO:tasks.workunit.client.1.smithi044.stdout:3/73: unlink ca 0 2024-06-03T01:20:49.188 INFO:tasks.workunit.client.1.smithi044.stdout:5/42: write d2/f8 [129176,56007] 0 2024-06-03T01:20:49.190 INFO:tasks.workunit.client.1.smithi044.stdout:9/57: unlink d6/d8/cc 0 2024-06-03T01:20:49.191 INFO:tasks.workunit.client.1.smithi044.stdout:8/76: mknod d5/d10/c12 0 2024-06-03T01:20:49.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/218: chown d0/d21/d27/c31 26095 1 2024-06-03T01:20:49.193 INFO:tasks.workunit.client.1.smithi044.stdout:9/58: creat d6/d8/fe x:0 0 0 2024-06-03T01:20:49.193 INFO:tasks.workunit.client.1.smithi044.stdout:9/59: write d6/d8/f9 [7207750,126512] 0 2024-06-03T01:20:49.193 INFO:tasks.workunit.client.1.smithi044.stdout:9/60: chown d6/dd 3 1 2024-06-03T01:20:49.194 INFO:tasks.workunit.client.1.smithi044.stdout:8/77: creat d5/d10/f13 x:0 0 0 2024-06-03T01:20:49.194 INFO:tasks.workunit.client.1.smithi044.stdout:8/78: readlink d5/lc 0 2024-06-03T01:20:49.195 INFO:tasks.workunit.client.1.smithi044.stdout:0/130: dwrite d3/d1c/d21/f26 [0,4194304] 0 2024-06-03T01:20:49.195 INFO:tasks.workunit.client.1.smithi044.stdout:7/219: symlink d0/d3/l52 0 2024-06-03T01:20:49.197 INFO:tasks.workunit.client.1.smithi044.stdout:2/69: dwrite f2 [0,4194304] 0 2024-06-03T01:20:49.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/61: creat d6/ff x:0 0 0 2024-06-03T01:20:49.198 INFO:tasks.workunit.client.1.smithi044.stdout:4/63: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.198 INFO:tasks.workunit.client.1.smithi044.stdout:4/64: rmdir - no directory 2024-06-03T01:20:49.198 INFO:tasks.workunit.client.1.smithi044.stdout:4/65: write f5 [1032911,45802] 0 2024-06-03T01:20:49.198 INFO:tasks.workunit.client.1.smithi044.stdout:8/79: symlink d5/l14 0 2024-06-03T01:20:49.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/131: rename d3/d1c/f2e to d3/d1c/d27/f34 0 2024-06-03T01:20:49.204 INFO:tasks.workunit.client.1.smithi044.stdout:3/74: dwrite df/f15 [0,4194304] 0 2024-06-03T01:20:49.205 INFO:tasks.workunit.client.1.smithi044.stdout:9/62: mknod d6/d8/db/c10 0 2024-06-03T01:20:49.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/43: getdents d2 0 2024-06-03T01:20:49.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/44: chown d2/f8 38 1 2024-06-03T01:20:49.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/45: write d2/f8 [341291,129357] 0 2024-06-03T01:20:49.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/46: truncate d2/f8 520810 0 2024-06-03T01:20:49.208 INFO:tasks.workunit.client.1.smithi044.stdout:6/109: getdents d2/d4 0 2024-06-03T01:20:49.208 INFO:tasks.workunit.client.1.smithi044.stdout:3/75: dread f9 [0,4194304] 0 2024-06-03T01:20:49.211 INFO:tasks.workunit.client.1.smithi044.stdout:4/66: mknod c13 0 2024-06-03T01:20:49.211 INFO:tasks.workunit.client.1.smithi044.stdout:5/47: dread d2/f6 [0,4194304] 0 2024-06-03T01:20:49.213 INFO:tasks.workunit.client.1.smithi044.stdout:2/70: dwrite f0 [0,4194304] 0 2024-06-03T01:20:49.213 INFO:tasks.workunit.client.1.smithi044.stdout:8/80: rename d5/fd to d5/f15 0 2024-06-03T01:20:49.221 INFO:tasks.workunit.client.1.smithi044.stdout:6/110: mknod d2/d6/c29 0 2024-06-03T01:20:49.222 INFO:tasks.workunit.client.1.smithi044.stdout:9/63: dwrite f2 [0,4194304] 0 2024-06-03T01:20:49.222 INFO:tasks.workunit.client.1.smithi044.stdout:9/64: truncate d6/d8/fe 59547 0 2024-06-03T01:20:49.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/67: unlink c7 0 2024-06-03T01:20:49.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/68: stat fd 0 2024-06-03T01:20:49.225 INFO:tasks.workunit.client.1.smithi044.stdout:5/48: rmdir d2 39 2024-06-03T01:20:49.228 INFO:tasks.workunit.client.1.smithi044.stdout:2/71: creat db/dd/f11 x:0 0 0 2024-06-03T01:20:49.228 INFO:tasks.workunit.client.1.smithi044.stdout:3/76: dwrite df/f15 [0,4194304] 0 2024-06-03T01:20:49.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/81: stat d5/la 0 2024-06-03T01:20:49.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/82: chown d5/f15 177514504 1 2024-06-03T01:20:49.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/83: chown f2 6773 1 2024-06-03T01:20:49.233 INFO:tasks.workunit.client.1.smithi044.stdout:4/69: symlink l14 0 2024-06-03T01:20:49.233 INFO:tasks.workunit.client.1.smithi044.stdout:4/70: write f2 [431337,88129] 0 2024-06-03T01:20:49.235 INFO:tasks.workunit.client.1.smithi044.stdout:2/72: creat db/f12 x:0 0 0 2024-06-03T01:20:49.237 INFO:tasks.workunit.client.1.smithi044.stdout:6/111: getdents d2/d6/db/df 0 2024-06-03T01:20:49.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/71: creat f15 x:0 0 0 2024-06-03T01:20:49.240 INFO:tasks.workunit.client.1.smithi044.stdout:2/73: unlink f6 0 2024-06-03T01:20:49.240 INFO:tasks.workunit.client.1.smithi044.stdout:2/74: stat db/l10 0 2024-06-03T01:20:49.240 INFO:tasks.workunit.client.1.smithi044.stdout:2/75: write db/dd/fe [246509,23456] 0 2024-06-03T01:20:49.241 INFO:tasks.workunit.client.1.smithi044.stdout:9/65: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:49.243 INFO:tasks.workunit.client.1.smithi044.stdout:6/112: rename d2/d6/f28 to d2/d6/db/df/f2a 0 2024-06-03T01:20:49.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/72: link c0 c16 0 2024-06-03T01:20:49.248 INFO:tasks.workunit.client.1.smithi044.stdout:6/113: truncate d2/d6/f13 3296049 0 2024-06-03T01:20:49.249 INFO:tasks.workunit.client.1.smithi044.stdout:4/73: mkdir d17 0 2024-06-03T01:20:49.251 INFO:tasks.workunit.client.1.smithi044.stdout:4/74: mkdir d17/d18 0 2024-06-03T01:20:49.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/75: creat d17/d18/f19 x:0 0 0 2024-06-03T01:20:49.274 INFO:tasks.workunit.client.1.smithi044.stdout:5/49: dwrite d2/f8 [0,4194304] 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:3/77: dwrite f9 [0,4194304] 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:8/84: dwrite d5/fb [0,4194304] 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/66: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/67: chown d6/c7 3949 1 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/68: chown d6/d8/db 3196 1 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/69: truncate d6/d8/fe 227953 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:4/76: dwrite fb [0,4194304] 0 2024-06-03T01:20:49.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/50: mkdir d2/dc 0 2024-06-03T01:20:49.277 INFO:tasks.workunit.client.1.smithi044.stdout:3/78: creat df/d13/f16 x:0 0 0 2024-06-03T01:20:49.278 INFO:tasks.workunit.client.1.smithi044.stdout:8/85: mkdir d5/d16 0 2024-06-03T01:20:49.278 INFO:tasks.workunit.client.1.smithi044.stdout:8/86: chown f3 76318 1 2024-06-03T01:20:49.279 INFO:tasks.workunit.client.1.smithi044.stdout:9/70: rename d6/ff to d6/f11 0 2024-06-03T01:20:49.280 INFO:tasks.workunit.client.1.smithi044.stdout:9/71: chown c5 229464788 1 2024-06-03T01:20:49.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/76: rmdir db 39 2024-06-03T01:20:49.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/77: chown fa 1606119 1 2024-06-03T01:20:49.288 INFO:tasks.workunit.client.1.smithi044.stdout:5/51: symlink d2/ld 0 2024-06-03T01:20:49.289 INFO:tasks.workunit.client.1.smithi044.stdout:5/52: getdents d2/dc 0 2024-06-03T01:20:49.290 INFO:tasks.workunit.client.1.smithi044.stdout:3/79: rename df/f15 to df/f17 0 2024-06-03T01:20:49.292 INFO:tasks.workunit.client.1.smithi044.stdout:4/77: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.293 INFO:tasks.workunit.client.1.smithi044.stdout:2/78: unlink f0 0 2024-06-03T01:20:49.293 INFO:tasks.workunit.client.1.smithi044.stdout:2/79: fsync f7 0 2024-06-03T01:20:49.294 INFO:tasks.workunit.client.1.smithi044.stdout:5/53: rmdir d2 39 2024-06-03T01:20:49.294 INFO:tasks.workunit.client.1.smithi044.stdout:3/80: link df/f10 df/d13/f18 0 2024-06-03T01:20:49.297 INFO:tasks.workunit.client.1.smithi044.stdout:3/81: dread df/f11 [0,4194304] 0 2024-06-03T01:20:49.297 INFO:tasks.workunit.client.1.smithi044.stdout:3/82: chown df/f10 33395053 1 2024-06-03T01:20:49.297 INFO:tasks.workunit.client.1.smithi044.stdout:3/83: dread - df/d13/f16 zero size 2024-06-03T01:20:49.298 INFO:tasks.workunit.client.1.smithi044.stdout:2/80: symlink db/l13 0 2024-06-03T01:20:49.300 INFO:tasks.workunit.client.1.smithi044.stdout:8/87: getdents d5 0 2024-06-03T01:20:49.302 INFO:tasks.workunit.client.1.smithi044.stdout:2/81: symlink db/l14 0 2024-06-03T01:20:49.304 INFO:tasks.workunit.client.1.smithi044.stdout:8/88: rename c4 to d5/c17 0 2024-06-03T01:20:49.304 INFO:tasks.workunit.client.1.smithi044.stdout:9/72: truncate d6/d8/fe 92329 0 2024-06-03T01:20:49.304 INFO:tasks.workunit.client.1.smithi044.stdout:9/73: dread - d6/f11 zero size 2024-06-03T01:20:49.306 INFO:tasks.workunit.client.1.smithi044.stdout:2/82: mkdir db/d15 0 2024-06-03T01:20:49.308 INFO:tasks.workunit.client.1.smithi044.stdout:2/83: symlink db/dd/l16 0 2024-06-03T01:20:49.310 INFO:tasks.workunit.client.1.smithi044.stdout:8/89: symlink d5/d16/l18 0 2024-06-03T01:20:49.311 INFO:tasks.workunit.client.1.smithi044.stdout:2/84: mkdir db/d17 0 2024-06-03T01:20:49.314 INFO:tasks.workunit.client.1.smithi044.stdout:2/85: mknod db/d15/c18 0 2024-06-03T01:20:49.314 INFO:tasks.workunit.client.1.smithi044.stdout:5/54: dwrite d2/f6 [4194304,4194304] 0 2024-06-03T01:20:49.314 INFO:tasks.workunit.client.1.smithi044.stdout:3/84: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.314 INFO:tasks.workunit.client.1.smithi044.stdout:2/86: read db/dd/fe [150724,70687] 0 2024-06-03T01:20:49.315 INFO:tasks.workunit.client.1.smithi044.stdout:2/87: dread - db/f12 zero size 2024-06-03T01:20:49.317 INFO:tasks.workunit.client.1.smithi044.stdout:5/55: write d2/f6 [309050,98970] 0 2024-06-03T01:20:49.318 INFO:tasks.workunit.client.1.smithi044.stdout:3/85: rename df/d13/f18 to df/f19 0 2024-06-03T01:20:49.320 INFO:tasks.workunit.client.1.smithi044.stdout:4/78: truncate fb 2382626 0 2024-06-03T01:20:49.320 INFO:tasks.workunit.client.1.smithi044.stdout:4/79: write ff [667948,6647] 0 2024-06-03T01:20:49.322 INFO:tasks.workunit.client.1.smithi044.stdout:8/90: write d5/fb [1792162,95924] 0 2024-06-03T01:20:49.323 INFO:tasks.workunit.client.1.smithi044.stdout:2/88: write fa [1878023,5914] 0 2024-06-03T01:20:49.325 INFO:tasks.workunit.client.1.smithi044.stdout:3/86: write f9 [4017213,51149] 0 2024-06-03T01:20:49.325 INFO:tasks.workunit.client.1.smithi044.stdout:3/87: write f9 [998184,94071] 0 2024-06-03T01:20:49.325 INFO:tasks.workunit.client.1.smithi044.stdout:3/88: creat df/d13/f1a x:0 0 0 2024-06-03T01:20:49.326 INFO:tasks.workunit.client.1.smithi044.stdout:5/56: symlink d2/dc/le 0 2024-06-03T01:20:49.327 INFO:tasks.workunit.client.1.smithi044.stdout:3/89: dread df/f11 [0,4194304] 0 2024-06-03T01:20:49.327 INFO:tasks.workunit.client.1.smithi044.stdout:3/90: write df/f11 [966029,49582] 0 2024-06-03T01:20:49.328 INFO:tasks.workunit.client.1.smithi044.stdout:4/80: unlink d17/d18/f19 0 2024-06-03T01:20:49.329 INFO:tasks.workunit.client.1.smithi044.stdout:9/74: link d6/d8/fe d6/f12 0 2024-06-03T01:20:49.330 INFO:tasks.workunit.client.1.smithi044.stdout:9/75: creat d6/d8/f13 x:0 0 0 2024-06-03T01:20:49.330 INFO:tasks.workunit.client.1.smithi044.stdout:9/76: write d6/f12 [868654,29125] 0 2024-06-03T01:20:49.330 INFO:tasks.workunit.client.1.smithi044.stdout:9/77: write f2 [4443743,65726] 0 2024-06-03T01:20:49.330 INFO:tasks.workunit.client.1.smithi044.stdout:9/78: fdatasync d6/d8/f9 0 2024-06-03T01:20:49.333 INFO:tasks.workunit.client.1.smithi044.stdout:2/89: dread db/dd/fe [0,4194304] 0 2024-06-03T01:20:49.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/81: mkdir d17/d18/d1a 0 2024-06-03T01:20:49.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/82: fdatasync f5 0 2024-06-03T01:20:49.342 INFO:tasks.workunit.client.1.smithi044.stdout:5/57: getdents d2/dc 0 2024-06-03T01:20:49.342 INFO:tasks.workunit.client.1.smithi044.stdout:5/58: fsync d2/f3 0 2024-06-03T01:20:49.344 INFO:tasks.workunit.client.1.smithi044.stdout:2/90: creat db/f19 x:0 0 0 2024-06-03T01:20:49.347 INFO:tasks.workunit.client.1.smithi044.stdout:9/79: getdents d6 0 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/91: dwrite df/f11 [0,4194304] 0 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/92: stat f9 0 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/132: sync 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/133: write d3/d9/f1d [624965,100715] 0 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/114: sync 2024-06-03T01:20:49.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/115: chown d2/c5 893930943 1 2024-06-03T01:20:49.353 INFO:tasks.workunit.client.1.smithi044.stdout:7/220: sync 2024-06-03T01:20:49.353 INFO:tasks.workunit.client.1.smithi044.stdout:1/108: sync 2024-06-03T01:20:49.353 INFO:tasks.workunit.client.1.smithi044.stdout:8/91: dwrite d5/fe [0,4194304] 0 2024-06-03T01:20:49.357 INFO:tasks.workunit.client.1.smithi044.stdout:1/109: creat d0/f2c x:0 0 0 2024-06-03T01:20:49.359 INFO:tasks.workunit.client.1.smithi044.stdout:7/221: creat d0/d21/f53 x:0 0 0 2024-06-03T01:20:49.359 INFO:tasks.workunit.client.1.smithi044.stdout:7/222: fdatasync d0/fd 0 2024-06-03T01:20:49.359 INFO:tasks.workunit.client.1.smithi044.stdout:9/80: write f2 [1049998,89795] 0 2024-06-03T01:20:49.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/91: rmdir db 39 2024-06-03T01:20:49.362 INFO:tasks.workunit.client.1.smithi044.stdout:5/59: truncate d2/f6 7096637 0 2024-06-03T01:20:49.362 INFO:tasks.workunit.client.1.smithi044.stdout:5/60: write d2/f9 [421325,114770] 0 2024-06-03T01:20:49.363 INFO:tasks.workunit.client.1.smithi044.stdout:8/92: symlink d5/d16/l19 0 2024-06-03T01:20:49.366 INFO:tasks.workunit.client.1.smithi044.stdout:7/223: dread d0/f4 [0,4194304] 0 2024-06-03T01:20:49.366 INFO:tasks.workunit.client.1.smithi044.stdout:0/134: rmdir d3/d1c 39 2024-06-03T01:20:49.367 INFO:tasks.workunit.client.1.smithi044.stdout:0/135: chown d3/d1c/d27/d29/f2d 5 1 2024-06-03T01:20:49.367 INFO:tasks.workunit.client.1.smithi044.stdout:0/136: fsync d3/f8 0 2024-06-03T01:20:49.369 INFO:tasks.workunit.client.1.smithi044.stdout:6/116: dwrite d2/d6/f13 [0,4194304] 0 2024-06-03T01:20:49.369 INFO:tasks.workunit.client.1.smithi044.stdout:6/117: chown d2/d4/f7 49924018 1 2024-06-03T01:20:49.369 INFO:tasks.workunit.client.1.smithi044.stdout:1/110: mkdir d0/d3/d20/d7/d13/d2d 0 2024-06-03T01:20:49.369 INFO:tasks.workunit.client.1.smithi044.stdout:1/111: chown d0/d3/d20/f8 733 1 2024-06-03T01:20:49.378 INFO:tasks.workunit.client.1.smithi044.stdout:9/81: dwrite d6/f12 [0,4194304] 0 2024-06-03T01:20:49.378 INFO:tasks.workunit.client.1.smithi044.stdout:5/61: symlink d2/lf 0 2024-06-03T01:20:49.378 INFO:tasks.workunit.client.1.smithi044.stdout:8/93: fsync d5/d10/f13 0 2024-06-03T01:20:49.379 INFO:tasks.workunit.client.1.smithi044.stdout:4/83: truncate fd 533098 0 2024-06-03T01:20:49.379 INFO:tasks.workunit.client.1.smithi044.stdout:4/84: readlink l14 0 2024-06-03T01:20:49.379 INFO:tasks.workunit.client.1.smithi044.stdout:0/137: symlink d3/d1c/d20/d25/l35 0 2024-06-03T01:20:49.380 INFO:tasks.workunit.client.1.smithi044.stdout:6/118: mkdir d2/d23/d2b 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/224: dwrite d0/d3/d8/f37 [0,4194304] 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/225: creat d0/d16/f54 x:0 0 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/226: read d0/f4 [2387839,77775] 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/227: dread - d0/d16/f54 zero size 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/228: write d0/f4 [3366701,60155] 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:4/85: dread ff [0,4194304] 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:1/112: truncate d0/d3/f9 2830080 0 2024-06-03T01:20:49.382 INFO:tasks.workunit.client.1.smithi044.stdout:1/113: dread - d0/f2c zero size 2024-06-03T01:20:49.383 INFO:tasks.workunit.client.1.smithi044.stdout:2/92: symlink db/d17/l1a 0 2024-06-03T01:20:49.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/86: dread ff [0,4194304] 0 2024-06-03T01:20:49.383 INFO:tasks.workunit.client.1.smithi044.stdout:2/93: dread db/dd/fe [0,4194304] 0 2024-06-03T01:20:49.383 INFO:tasks.workunit.client.1.smithi044.stdout:9/82: creat d6/dd/f14 x:0 0 0 2024-06-03T01:20:49.389 INFO:tasks.workunit.client.1.smithi044.stdout:0/138: mkdir d3/d9/d36 0 2024-06-03T01:20:49.390 INFO:tasks.workunit.client.1.smithi044.stdout:6/119: write d2/f14 [565047,76448] 0 2024-06-03T01:20:49.390 INFO:tasks.workunit.client.1.smithi044.stdout:6/120: readlink d2/d6/db/l1b 0 2024-06-03T01:20:49.390 INFO:tasks.workunit.client.1.smithi044.stdout:8/94: write d5/f6 [303623,60443] 0 2024-06-03T01:20:49.394 INFO:tasks.workunit.client.1.smithi044.stdout:7/229: symlink d0/l55 0 2024-06-03T01:20:49.394 INFO:tasks.workunit.client.1.smithi044.stdout:7/230: write d0/d3/d8/d32/d33/f42 [566472,96914] 0 2024-06-03T01:20:49.395 INFO:tasks.workunit.client.1.smithi044.stdout:5/62: getdents d2 0 2024-06-03T01:20:49.395 INFO:tasks.workunit.client.1.smithi044.stdout:5/63: getdents d2 0 2024-06-03T01:20:49.397 INFO:tasks.workunit.client.1.smithi044.stdout:4/87: creat d17/d18/d1a/f1b x:0 0 0 2024-06-03T01:20:49.397 INFO:tasks.workunit.client.1.smithi044.stdout:4/88: stat d17 0 2024-06-03T01:20:49.398 INFO:tasks.workunit.client.1.smithi044.stdout:1/114: dread d0/f29 [0,4194304] 0 2024-06-03T01:20:49.399 INFO:tasks.workunit.client.1.smithi044.stdout:6/121: read d2/fe [4382904,116008] 0 2024-06-03T01:20:49.399 INFO:tasks.workunit.client.1.smithi044.stdout:6/122: creat d2/f2c x:0 0 0 2024-06-03T01:20:49.399 INFO:tasks.workunit.client.1.smithi044.stdout:6/123: readlink d2/d6/db/l1b 0 2024-06-03T01:20:49.400 INFO:tasks.workunit.client.1.smithi044.stdout:0/139: rename d3/cd to d3/d9/d36/c37 0 2024-06-03T01:20:49.401 INFO:tasks.workunit.client.1.smithi044.stdout:9/83: dwrite d6/f12 [0,4194304] 0 2024-06-03T01:20:49.401 INFO:tasks.workunit.client.1.smithi044.stdout:8/95: truncate d5/fe 261355 0 2024-06-03T01:20:49.402 INFO:tasks.workunit.client.1.smithi044.stdout:7/231: symlink d0/d3/d8/d32/l56 0 2024-06-03T01:20:49.404 INFO:tasks.workunit.client.1.smithi044.stdout:5/64: link f1 d2/dc/f10 0 2024-06-03T01:20:49.405 INFO:tasks.workunit.client.1.smithi044.stdout:4/89: mkdir d17/d1c 0 2024-06-03T01:20:49.406 INFO:tasks.workunit.client.1.smithi044.stdout:4/90: dread - d17/d18/d1a/f1b zero size 2024-06-03T01:20:49.406 INFO:tasks.workunit.client.1.smithi044.stdout:4/91: truncate ff 1364013 0 2024-06-03T01:20:49.406 INFO:tasks.workunit.client.1.smithi044.stdout:4/92: dread - d17/d18/d1a/f1b zero size 2024-06-03T01:20:49.407 INFO:tasks.workunit.client.1.smithi044.stdout:1/115: link d0/d3/d20/f8 d0/d3/d20/f2e 0 2024-06-03T01:20:49.407 INFO:tasks.workunit.client.1.smithi044.stdout:1/116: getdents d0/d3/d14 0 2024-06-03T01:20:49.409 INFO:tasks.workunit.client.1.smithi044.stdout:6/124: mknod d2/c2d 0 2024-06-03T01:20:49.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/65: dread d2/dc/f10 [0,4194304] 0 2024-06-03T01:20:49.410 INFO:tasks.workunit.client.1.smithi044.stdout:0/140: rmdir d3/d1c/d21 39 2024-06-03T01:20:49.411 INFO:tasks.workunit.client.1.smithi044.stdout:0/141: fsync d3/f8 0 2024-06-03T01:20:49.413 INFO:tasks.workunit.client.1.smithi044.stdout:9/84: link d6/d8/f9 d6/d8/db/f15 0 2024-06-03T01:20:49.414 INFO:tasks.workunit.client.1.smithi044.stdout:9/85: write d6/dd/f14 [202520,92344] 0 2024-06-03T01:20:49.415 INFO:tasks.workunit.client.1.smithi044.stdout:8/96: mknod d5/d10/c1a 0 2024-06-03T01:20:49.417 INFO:tasks.workunit.client.1.smithi044.stdout:0/142: dread d3/d1c/d27/d29/f2d [0,4194304] 0 2024-06-03T01:20:49.417 INFO:tasks.workunit.client.1.smithi044.stdout:0/143: readlink d3/d1c/d20/d25/l35 0 2024-06-03T01:20:49.417 INFO:tasks.workunit.client.1.smithi044.stdout:0/144: chown d3/d1c/d27/d29/f2d 3 1 2024-06-03T01:20:49.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/66: mkdir d2/d11 0 2024-06-03T01:20:49.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/67: chown d2/lf 3244926 1 2024-06-03T01:20:49.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/68: chown l0 1038526807 1 2024-06-03T01:20:49.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/86: mkdir d6/d8/d16 0 2024-06-03T01:20:49.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/87: creat d6/dd/f17 x:0 0 0 2024-06-03T01:20:49.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/117: dread d0/f15 [0,4194304] 0 2024-06-03T01:20:49.422 INFO:tasks.workunit.client.1.smithi044.stdout:0/145: rename d3/f23 to d3/d1c/d27/f38 0 2024-06-03T01:20:49.422 INFO:tasks.workunit.client.1.smithi044.stdout:0/146: chown d3/f8 122 1 2024-06-03T01:20:49.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/118: mkdir d0/d3/d20/d7/d2f 0 2024-06-03T01:20:49.425 INFO:tasks.workunit.client.1.smithi044.stdout:8/97: truncate d5/fe 45483 0 2024-06-03T01:20:49.427 INFO:tasks.workunit.client.1.smithi044.stdout:0/147: rmdir d3/d1c/d20 39 2024-06-03T01:20:49.427 INFO:tasks.workunit.client.1.smithi044.stdout:5/69: write d2/f3 [6425784,91744] 0 2024-06-03T01:20:49.427 INFO:tasks.workunit.client.1.smithi044.stdout:5/70: stat d2/dc/le 0 2024-06-03T01:20:49.427 INFO:tasks.workunit.client.1.smithi044.stdout:5/71: read f1 [2283703,64188] 0 2024-06-03T01:20:49.427 INFO:tasks.workunit.client.1.smithi044.stdout:1/119: write d0/fd [5306306,104385] 0 2024-06-03T01:20:49.428 INFO:tasks.workunit.client.1.smithi044.stdout:8/98: readlink d5/l14 0 2024-06-03T01:20:49.429 INFO:tasks.workunit.client.1.smithi044.stdout:0/148: mknod d3/d1c/d21/c39 0 2024-06-03T01:20:49.430 INFO:tasks.workunit.client.1.smithi044.stdout:0/149: creat d3/f3a x:0 0 0 2024-06-03T01:20:49.431 INFO:tasks.workunit.client.1.smithi044.stdout:5/72: mkdir d2/d11/d12 0 2024-06-03T01:20:49.432 INFO:tasks.workunit.client.1.smithi044.stdout:1/120: mkdir d0/d3/d14/d30 0 2024-06-03T01:20:49.432 INFO:tasks.workunit.client.1.smithi044.stdout:8/99: readlink d5/l8 0 2024-06-03T01:20:49.436 INFO:tasks.workunit.client.1.smithi044.stdout:5/73: rename d2/f4 to d2/f13 0 2024-06-03T01:20:49.436 INFO:tasks.workunit.client.1.smithi044.stdout:9/88: dwrite d6/d8/fe [0,4194304] 0 2024-06-03T01:20:49.437 INFO:tasks.workunit.client.1.smithi044.stdout:1/121: mknod d0/d3/d20/d7/d2f/c31 0 2024-06-03T01:20:49.437 INFO:tasks.workunit.client.1.smithi044.stdout:1/122: chown d0/d3/d14/d30 16626 1 2024-06-03T01:20:49.439 INFO:tasks.workunit.client.1.smithi044.stdout:0/150: getdents d3/d1c/d27 0 2024-06-03T01:20:49.439 INFO:tasks.workunit.client.1.smithi044.stdout:0/151: fdatasync d3/f14 0 2024-06-03T01:20:49.440 INFO:tasks.workunit.client.1.smithi044.stdout:5/74: mknod d2/d11/d12/c14 0 2024-06-03T01:20:49.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/89: mknod d6/d8/db/c18 0 2024-06-03T01:20:49.443 INFO:tasks.workunit.client.1.smithi044.stdout:1/123: write d0/d16/f26 [150631,433] 0 2024-06-03T01:20:49.445 INFO:tasks.workunit.client.1.smithi044.stdout:0/152: rename d3/d1c/d20/d25/f33 to d3/d1c/f3b 0 2024-06-03T01:20:49.446 INFO:tasks.workunit.client.1.smithi044.stdout:0/153: write d3/f17 [1843416,117122] 0 2024-06-03T01:20:49.446 INFO:tasks.workunit.client.1.smithi044.stdout:9/90: symlink d6/l19 0 2024-06-03T01:20:49.448 INFO:tasks.workunit.client.1.smithi044.stdout:0/154: mknod d3/d9/d36/c3c 0 2024-06-03T01:20:49.450 INFO:tasks.workunit.client.1.smithi044.stdout:1/124: link d0/f2c d0/f32 0 2024-06-03T01:20:49.451 INFO:tasks.workunit.client.1.smithi044.stdout:0/155: rename d3/d1c/d20/d25/l35 to d3/d1c/d27/l3d 0 2024-06-03T01:20:49.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/100: dwrite f3 [0,4194304] 0 2024-06-03T01:20:49.460 INFO:tasks.workunit.client.1.smithi044.stdout:9/91: dread d6/d8/f9 [0,4194304] 0 2024-06-03T01:20:49.463 INFO:tasks.workunit.client.1.smithi044.stdout:5/75: dwrite d2/f6 [4194304,4194304] 0 2024-06-03T01:20:49.463 INFO:tasks.workunit.client.1.smithi044.stdout:5/76: dread - d2/fa zero size 2024-06-03T01:20:49.463 INFO:tasks.workunit.client.1.smithi044.stdout:5/77: stat d2 0 2024-06-03T01:20:49.463 INFO:tasks.workunit.client.1.smithi044.stdout:9/92: symlink d6/dd/l1a 0 2024-06-03T01:20:49.464 INFO:tasks.workunit.client.1.smithi044.stdout:8/101: symlink d5/l1b 0 2024-06-03T01:20:49.467 INFO:tasks.workunit.client.1.smithi044.stdout:5/78: creat d2/dc/f15 x:0 0 0 2024-06-03T01:20:49.468 INFO:tasks.workunit.client.1.smithi044.stdout:0/156: dwrite d3/d9/f1d [0,4194304] 0 2024-06-03T01:20:49.468 INFO:tasks.workunit.client.1.smithi044.stdout:8/102: symlink d5/d16/l1c 0 2024-06-03T01:20:49.472 INFO:tasks.workunit.client.1.smithi044.stdout:0/157: creat d3/d1c/f3e x:0 0 0 2024-06-03T01:20:49.473 INFO:tasks.workunit.client.1.smithi044.stdout:8/103: chown d5/f15 29 1 2024-06-03T01:20:49.473 INFO:tasks.workunit.client.1.smithi044.stdout:8/104: write d5/f15 [61304,82820] 0 2024-06-03T01:20:49.475 INFO:tasks.workunit.client.1.smithi044.stdout:5/79: fsync d2/f3 0 2024-06-03T01:20:49.475 INFO:tasks.workunit.client.1.smithi044.stdout:5/80: fdatasync d2/f6 0 2024-06-03T01:20:49.476 INFO:tasks.workunit.client.1.smithi044.stdout:8/105: getdents d5/d10 0 2024-06-03T01:20:49.476 INFO:tasks.workunit.client.1.smithi044.stdout:8/106: readlink d5/d16/l1c 0 2024-06-03T01:20:49.476 INFO:tasks.workunit.client.1.smithi044.stdout:9/93: dwrite d6/f11 [0,4194304] 0 2024-06-03T01:20:49.481 INFO:tasks.workunit.client.1.smithi044.stdout:5/81: unlink d2/f9 0 2024-06-03T01:20:49.482 INFO:tasks.workunit.client.1.smithi044.stdout:5/82: creat d2/dc/f16 x:0 0 0 2024-06-03T01:20:49.483 INFO:tasks.workunit.client.1.smithi044.stdout:8/107: mkdir d5/d16/d1d 0 2024-06-03T01:20:49.487 INFO:tasks.workunit.client.1.smithi044.stdout:9/94: creat d6/d8/d16/f1b x:0 0 0 2024-06-03T01:20:49.490 INFO:tasks.workunit.client.1.smithi044.stdout:5/83: rename d2/d11/d12/c14 to d2/d11/c17 0 2024-06-03T01:20:49.490 INFO:tasks.workunit.client.1.smithi044.stdout:5/84: chown d2/d11/d12 4566 1 2024-06-03T01:20:49.491 INFO:tasks.workunit.client.1.smithi044.stdout:0/158: dwrite d3/d1c/f24 [0,4194304] 0 2024-06-03T01:20:49.491 INFO:tasks.workunit.client.1.smithi044.stdout:1/125: dwrite d0/d3/f9 [0,4194304] 0 2024-06-03T01:20:49.492 INFO:tasks.workunit.client.1.smithi044.stdout:8/108: symlink d5/d16/l1e 0 2024-06-03T01:20:49.499 INFO:tasks.workunit.client.1.smithi044.stdout:5/85: mkdir d2/dc/d18 0 2024-06-03T01:20:49.500 INFO:tasks.workunit.client.1.smithi044.stdout:0/159: rmdir d3/d1c/d21 39 2024-06-03T01:20:49.502 INFO:tasks.workunit.client.1.smithi044.stdout:8/109: creat d5/d16/d1d/f1f x:0 0 0 2024-06-03T01:20:49.503 INFO:tasks.workunit.client.1.smithi044.stdout:5/86: mknod d2/c19 0 2024-06-03T01:20:49.503 INFO:tasks.workunit.client.1.smithi044.stdout:5/87: fdatasync d2/dc/f10 0 2024-06-03T01:20:49.503 INFO:tasks.workunit.client.1.smithi044.stdout:0/160: rmdir d3/d1c/d27 39 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:8/110: dread d5/f6 [0,4194304] 0 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:8/111: dread - d5/d16/d1d/f1f zero size 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:9/95: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:9/96: chown d6/d8/f13 91960213 1 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:9/97: readlink d6/l19 0 2024-06-03T01:20:49.506 INFO:tasks.workunit.client.1.smithi044.stdout:9/98: write d6/f11 [4528172,94270] 0 2024-06-03T01:20:49.507 INFO:tasks.workunit.client.1.smithi044.stdout:9/99: mknod d6/d8/d16/c1c 0 2024-06-03T01:20:49.507 INFO:tasks.workunit.client.1.smithi044.stdout:9/100: truncate d6/d8/fe 290683 0 2024-06-03T01:20:49.509 INFO:tasks.workunit.client.1.smithi044.stdout:9/101: symlink d6/d8/l1d 0 2024-06-03T01:20:49.509 INFO:tasks.workunit.client.1.smithi044.stdout:9/102: truncate d6/d8/d16/f1b 976479 0 2024-06-03T01:20:49.511 INFO:tasks.workunit.client.1.smithi044.stdout:1/126: dwrite d0/d3/f9 [0,4194304] 0 2024-06-03T01:20:49.512 INFO:tasks.workunit.client.1.smithi044.stdout:9/103: dread d6/d8/db/f15 [0,4194304] 0 2024-06-03T01:20:49.513 INFO:tasks.workunit.client.1.smithi044.stdout:8/112: write f3 [1471014,17239] 0 2024-06-03T01:20:49.514 INFO:tasks.workunit.client.1.smithi044.stdout:8/113: chown d5/d16/l1c 0 1 2024-06-03T01:20:49.514 INFO:tasks.workunit.client.1.smithi044.stdout:1/127: creat d0/d3/d20/d7/d2f/f33 x:0 0 0 2024-06-03T01:20:49.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/128: fdatasync d0/f32 0 2024-06-03T01:20:49.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/129: chown d0/d3/d14 523 1 2024-06-03T01:20:49.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/130: fsync d0/d3/f19 0 2024-06-03T01:20:49.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/131: creat d0/d3/d20/d7/f34 x:0 0 0 2024-06-03T01:20:49.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/132: fdatasync d0/d3/d20/d7/d13/f24 0 2024-06-03T01:20:49.516 INFO:tasks.workunit.client.1.smithi044.stdout:0/161: write d3/f14 [3741559,129283] 0 2024-06-03T01:20:49.516 INFO:tasks.workunit.client.1.smithi044.stdout:9/104: symlink d6/l1e 0 2024-06-03T01:20:49.520 INFO:tasks.workunit.client.1.smithi044.stdout:1/133: readlink d0/l10 0 2024-06-03T01:20:49.520 INFO:tasks.workunit.client.1.smithi044.stdout:1/134: dread - d0/f32 zero size 2024-06-03T01:20:49.522 INFO:tasks.workunit.client.1.smithi044.stdout:0/162: link d3/f17 d3/d1c/d27/f3f 0 2024-06-03T01:20:49.524 INFO:tasks.workunit.client.1.smithi044.stdout:9/105: mknod d6/d8/d16/c1f 0 2024-06-03T01:20:49.525 INFO:tasks.workunit.client.1.smithi044.stdout:8/114: mkdir d5/d20 0 2024-06-03T01:20:49.526 INFO:tasks.workunit.client.1.smithi044.stdout:1/135: mknod d0/d3/d20/d7/d2f/c35 0 2024-06-03T01:20:49.526 INFO:tasks.workunit.client.1.smithi044.stdout:1/136: chown d0/d3/c27 2355 1 2024-06-03T01:20:49.527 INFO:tasks.workunit.client.1.smithi044.stdout:1/137: chown d0/d3/d20/d7/d13/c25 98140422 1 2024-06-03T01:20:49.528 INFO:tasks.workunit.client.1.smithi044.stdout:0/163: creat d3/d1c/d20/d25/f40 x:0 0 0 2024-06-03T01:20:49.528 INFO:tasks.workunit.client.1.smithi044.stdout:5/88: rmdir d2/dc 39 2024-06-03T01:20:49.528 INFO:tasks.workunit.client.1.smithi044.stdout:5/89: rename d2 to d2/d11/d1a 22 2024-06-03T01:20:49.529 INFO:tasks.workunit.client.1.smithi044.stdout:9/106: mknod d6/d8/db/c20 0 2024-06-03T01:20:49.532 INFO:tasks.workunit.client.1.smithi044.stdout:0/164: unlink d3/fe 0 2024-06-03T01:20:49.535 INFO:tasks.workunit.client.1.smithi044.stdout:8/115: link d5/lc d5/d10/l21 0 2024-06-03T01:20:49.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/138: write d0/d3/f9 [3305243,13132] 0 2024-06-03T01:20:49.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/139: fdatasync d0/f2c 0 2024-06-03T01:20:49.539 INFO:tasks.workunit.client.1.smithi044.stdout:8/116: dread d5/ff [0,4194304] 0 2024-06-03T01:20:49.539 INFO:tasks.workunit.client.1.smithi044.stdout:9/107: dread f4 [4194304,4194304] 0 2024-06-03T01:20:49.539 INFO:tasks.workunit.client.1.smithi044.stdout:9/108: fdatasync f4 0 2024-06-03T01:20:49.539 INFO:tasks.workunit.client.1.smithi044.stdout:9/109: write d6/dd/f14 [574105,105920] 0 2024-06-03T01:20:49.544 INFO:tasks.workunit.client.1.smithi044.stdout:5/90: dwrite d2/f3 [4194304,4194304] 0 2024-06-03T01:20:49.548 INFO:tasks.workunit.client.1.smithi044.stdout:5/91: creat d2/d11/d12/f1b x:0 0 0 2024-06-03T01:20:49.548 INFO:tasks.workunit.client.1.smithi044.stdout:5/92: chown d2/f3 27 1 2024-06-03T01:20:49.552 INFO:tasks.workunit.client.1.smithi044.stdout:1/140: getdents d0/d3/d20/d7 0 2024-06-03T01:20:49.555 INFO:tasks.workunit.client.1.smithi044.stdout:9/110: getdents d6/d8/d16 0 2024-06-03T01:20:49.556 INFO:tasks.workunit.client.1.smithi044.stdout:8/117: dwrite d5/ff [0,4194304] 0 2024-06-03T01:20:49.556 INFO:tasks.workunit.client.1.smithi044.stdout:8/118: write d5/f6 [706488,56459] 0 2024-06-03T01:20:49.558 INFO:tasks.workunit.client.1.smithi044.stdout:9/111: link d6/f11 d6/dd/f21 0 2024-06-03T01:20:49.558 INFO:tasks.workunit.client.1.smithi044.stdout:1/141: dread d0/fd [4194304,4194304] 0 2024-06-03T01:20:49.558 INFO:tasks.workunit.client.1.smithi044.stdout:1/142: fsync d0/d3/d20/d7/d2f/f33 0 2024-06-03T01:20:49.558 INFO:tasks.workunit.client.1.smithi044.stdout:9/112: read d6/d8/db/f15 [6092456,65980] 0 2024-06-03T01:20:49.559 INFO:tasks.workunit.client.1.smithi044.stdout:9/113: fsync d6/d8/db/f15 0 2024-06-03T01:20:49.560 INFO:tasks.workunit.client.1.smithi044.stdout:1/143: read d0/d3/d20/d7/f17 [447885,98558] 0 2024-06-03T01:20:49.561 INFO:tasks.workunit.client.1.smithi044.stdout:9/114: dread d6/d8/f9 [8388608,4194304] 0 2024-06-03T01:20:49.562 INFO:tasks.workunit.client.1.smithi044.stdout:8/119: symlink d5/d10/l22 0 2024-06-03T01:20:49.562 INFO:tasks.workunit.client.1.smithi044.stdout:8/120: creat d5/d16/d1d/f23 x:0 0 0 2024-06-03T01:20:49.562 INFO:tasks.workunit.client.1.smithi044.stdout:8/121: write f2 [2348666,117158] 0 2024-06-03T01:20:49.563 INFO:tasks.workunit.client.1.smithi044.stdout:1/144: chown d0/c2 10563195 1 2024-06-03T01:20:49.564 INFO:tasks.workunit.client.1.smithi044.stdout:9/115: rmdir d6/d8/db 39 2024-06-03T01:20:49.565 INFO:tasks.workunit.client.1.smithi044.stdout:9/116: read d6/f12 [222794,112630] 0 2024-06-03T01:20:49.565 INFO:tasks.workunit.client.1.smithi044.stdout:5/93: dwrite f1 [4194304,4194304] 0 2024-06-03T01:20:49.568 INFO:tasks.workunit.client.1.smithi044.stdout:1/145: dread d0/d3/d20/f1d [0,4194304] 0 2024-06-03T01:20:49.568 INFO:tasks.workunit.client.1.smithi044.stdout:1/146: write d0/d3/f28 [649475,71084] 0 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/117: creat d6/d8/db/f22 x:0 0 0 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:8/122: dread d5/fb [0,4194304] 0 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:5/94: symlink d2/d11/l1c 0 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:8/123: chown d5/d16/d1d 1566123 1 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:8/124: stat d5/l8 0 2024-06-03T01:20:49.569 INFO:tasks.workunit.client.1.smithi044.stdout:8/125: chown d5/d16/l19 173919674 1 2024-06-03T01:20:49.570 INFO:tasks.workunit.client.1.smithi044.stdout:9/118: symlink d6/d8/db/l23 0 2024-06-03T01:20:49.571 INFO:tasks.workunit.client.1.smithi044.stdout:9/119: fdatasync d6/d8/db/f15 0 2024-06-03T01:20:49.571 INFO:tasks.workunit.client.1.smithi044.stdout:9/120: chown d6/d8/db/c10 245665 1 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/147: dread d0/f15 [0,4194304] 0 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/148: chown d0/d3/d20/d7/f17 45933 1 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/149: write d0/d16/f26 [1101260,30669] 0 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/150: dread - d0/f32 zero size 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/151: write d0/d3/f9 [1910596,45114] 0 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/152: chown d0/d3/c27 0 1 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/153: fdatasync d0/fd 0 2024-06-03T01:20:49.573 INFO:tasks.workunit.client.1.smithi044.stdout:9/121: rename d6/c7 to d6/d8/c24 0 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:2/94: sync 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:3/93: sync 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:3/94: stat fe 0 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:1/154: unlink d0/d3/d20/f21 0 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:1/155: chown d0/d3/d20/d7/d2f/c31 6863 1 2024-06-03T01:20:49.575 INFO:tasks.workunit.client.1.smithi044.stdout:1/156: truncate d0/d3/f28 1494314 0 2024-06-03T01:20:49.578 INFO:tasks.workunit.client.1.smithi044.stdout:2/95: rename db/d17 to db/d1b 0 2024-06-03T01:20:49.578 INFO:tasks.workunit.client.1.smithi044.stdout:2/96: read - db/f12 zero size 2024-06-03T01:20:49.578 INFO:tasks.workunit.client.1.smithi044.stdout:2/97: write db/f19 [850004,110442] 0 2024-06-03T01:20:49.580 INFO:tasks.workunit.client.1.smithi044.stdout:2/98: rmdir db/dd 39 2024-06-03T01:20:49.581 INFO:tasks.workunit.client.1.smithi044.stdout:2/99: symlink db/d1b/l1c 0 2024-06-03T01:20:49.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/100: chown f7 158702 1 2024-06-03T01:20:49.585 INFO:tasks.workunit.client.1.smithi044.stdout:1/157: dread d0/d3/d20/f8 [0,4194304] 0 2024-06-03T01:20:49.589 INFO:tasks.workunit.client.1.smithi044.stdout:1/158: link d0/f5 d0/d3/d20/d7/f36 0 2024-06-03T01:20:49.590 INFO:tasks.workunit.client.1.smithi044.stdout:1/159: getdents d0/d3/d14 0 2024-06-03T01:20:49.591 INFO:tasks.workunit.client.1.smithi044.stdout:2/101: dread f7 [0,4194304] 0 2024-06-03T01:20:49.593 INFO:tasks.workunit.client.1.smithi044.stdout:2/102: mknod db/c1d 0 2024-06-03T01:20:49.599 INFO:tasks.workunit.client.1.smithi044.stdout:3/95: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.600 INFO:tasks.workunit.client.1.smithi044.stdout:5/95: dwrite d2/dc/f16 [0,4194304] 0 2024-06-03T01:20:49.600 INFO:tasks.workunit.client.1.smithi044.stdout:9/122: dwrite f2 [0,4194304] 0 2024-06-03T01:20:49.600 INFO:tasks.workunit.client.1.smithi044.stdout:9/123: fdatasync d6/d8/db/f22 0 2024-06-03T01:20:49.600 INFO:tasks.workunit.client.1.smithi044.stdout:9/124: chown d6/d8/l1d 576 1 2024-06-03T01:20:49.603 INFO:tasks.workunit.client.1.smithi044.stdout:1/160: dread d0/d3/d20/d7/d13/f24 [0,4194304] 0 2024-06-03T01:20:49.605 INFO:tasks.workunit.client.1.smithi044.stdout:5/96: rename d2/cb to d2/c1d 0 2024-06-03T01:20:49.628 INFO:tasks.workunit.client.1.smithi044.stdout:6/125: sync 2024-06-03T01:20:49.628 INFO:tasks.workunit.client.1.smithi044.stdout:7/232: sync 2024-06-03T01:20:49.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/93: sync 2024-06-03T01:20:49.628 INFO:tasks.workunit.client.1.smithi044.stdout:7/233: fsync d0/d3/fa 0 2024-06-03T01:20:49.629 INFO:tasks.workunit.client.1.smithi044.stdout:4/94: symlink d17/d18/l1d 0 2024-06-03T01:20:49.630 INFO:tasks.workunit.client.1.smithi044.stdout:4/95: creat d17/f1e x:0 0 0 2024-06-03T01:20:49.631 INFO:tasks.workunit.client.1.smithi044.stdout:4/96: symlink d17/d18/l1f 0 2024-06-03T01:20:49.631 INFO:tasks.workunit.client.1.smithi044.stdout:4/97: dread - d17/f1e zero size 2024-06-03T01:20:49.633 INFO:tasks.workunit.client.1.smithi044.stdout:4/98: read fb [253316,54207] 0 2024-06-03T01:20:49.633 INFO:tasks.workunit.client.1.smithi044.stdout:4/99: creat d17/d18/f20 x:0 0 0 2024-06-03T01:20:49.634 INFO:tasks.workunit.client.1.smithi044.stdout:4/100: creat d17/d18/d1a/f21 x:0 0 0 2024-06-03T01:20:49.634 INFO:tasks.workunit.client.1.smithi044.stdout:2/103: dwrite fa [0,4194304] 0 2024-06-03T01:20:49.635 INFO:tasks.workunit.client.1.smithi044.stdout:3/96: dwrite fe [4194304,4194304] 0 2024-06-03T01:20:49.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/234: dread d0/f4 [0,4194304] 0 2024-06-03T01:20:49.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/235: stat d0/d16/d23/f24 0 2024-06-03T01:20:49.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/236: truncate d0/d16/d23/f3d 875495 0 2024-06-03T01:20:49.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/237: chown d0/d16/d23/l49 99843509 1 2024-06-03T01:20:49.636 INFO:tasks.workunit.client.1.smithi044.stdout:9/125: dwrite f4 [0,4194304] 0 2024-06-03T01:20:49.637 INFO:tasks.workunit.client.1.smithi044.stdout:5/97: dwrite d2/f8 [0,4194304] 0 2024-06-03T01:20:49.637 INFO:tasks.workunit.client.1.smithi044.stdout:5/98: write d2/fa [916808,31366] 0 2024-06-03T01:20:49.637 INFO:tasks.workunit.client.1.smithi044.stdout:4/101: truncate fc 1752603 0 2024-06-03T01:20:49.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/238: getdents d0/d3/d8/d32/d33 0 2024-06-03T01:20:49.640 INFO:tasks.workunit.client.1.smithi044.stdout:7/239: creat d0/d3/d8/d32/d33/f57 x:0 0 0 2024-06-03T01:20:49.643 INFO:tasks.workunit.client.1.smithi044.stdout:7/240: unlink d0/d16/d23/f3d 0 2024-06-03T01:20:49.643 INFO:tasks.workunit.client.1.smithi044.stdout:7/241: chown d0/d16/d23/l26 20073 1 2024-06-03T01:20:49.645 INFO:tasks.workunit.client.1.smithi044.stdout:7/242: getdents d0/d3/d8/d32 0 2024-06-03T01:20:49.645 INFO:tasks.workunit.client.1.smithi044.stdout:7/243: readlink d0/d16/d23/l26 0 2024-06-03T01:20:49.645 INFO:tasks.workunit.client.1.smithi044.stdout:7/244: fdatasync d0/d3/d8/f37 0 2024-06-03T01:20:49.647 INFO:tasks.workunit.client.1.smithi044.stdout:7/245: dread d0/d3/d8/d32/f4a [0,4194304] 0 2024-06-03T01:20:49.647 INFO:tasks.workunit.client.1.smithi044.stdout:7/246: getdents d0/d3/d8/d32 0 2024-06-03T01:20:49.647 INFO:tasks.workunit.client.1.smithi044.stdout:7/247: read - d0/d3/d8/d32/f38 zero size 2024-06-03T01:20:49.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/102: write fc [2798760,64475] 0 2024-06-03T01:20:49.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/103: creat d17/d18/d1a/f22 x:0 0 0 2024-06-03T01:20:49.650 INFO:tasks.workunit.client.1.smithi044.stdout:9/126: dread d6/dd/f21 [0,4194304] 0 2024-06-03T01:20:49.652 INFO:tasks.workunit.client.1.smithi044.stdout:9/127: symlink d6/d8/db/l25 0 2024-06-03T01:20:49.653 INFO:tasks.workunit.client.1.smithi044.stdout:9/128: mkdir d6/d26 0 2024-06-03T01:20:49.656 INFO:tasks.workunit.client.1.smithi044.stdout:2/104: dwrite f7 [0,4194304] 0 2024-06-03T01:20:49.676 INFO:tasks.workunit.client.1.smithi044.stdout:3/97: dwrite df/d13/f1a [0,4194304] 0 2024-06-03T01:20:49.679 INFO:tasks.workunit.client.1.smithi044.stdout:5/99: dwrite d2/f8 [0,4194304] 0 2024-06-03T01:20:49.679 INFO:tasks.workunit.client.1.smithi044.stdout:5/100: readlink d2/d11/l1c 0 2024-06-03T01:20:49.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/98: rmdir df 39 2024-06-03T01:20:49.680 INFO:tasks.workunit.client.1.smithi044.stdout:7/248: dwrite d0/d21/f3b [0,4194304] 0 2024-06-03T01:20:49.681 INFO:tasks.workunit.client.1.smithi044.stdout:5/101: getdents d2/dc 0 2024-06-03T01:20:49.682 INFO:tasks.workunit.client.1.smithi044.stdout:2/105: dwrite db/f12 [0,4194304] 0 2024-06-03T01:20:49.682 INFO:tasks.workunit.client.1.smithi044.stdout:9/129: dwrite d6/d8/d16/f1b [0,4194304] 0 2024-06-03T01:20:49.682 INFO:tasks.workunit.client.1.smithi044.stdout:9/130: chown d6/d8/fe 0 1 2024-06-03T01:20:49.682 INFO:tasks.workunit.client.1.smithi044.stdout:9/131: chown c5 0 1 2024-06-03T01:20:49.683 INFO:tasks.workunit.client.1.smithi044.stdout:3/99: truncate df/f19 10826 0 2024-06-03T01:20:49.683 INFO:tasks.workunit.client.1.smithi044.stdout:3/100: chown fe 2326520 1 2024-06-03T01:20:49.683 INFO:tasks.workunit.client.1.smithi044.stdout:3/101: truncate df/f17 4929792 0 2024-06-03T01:20:49.683 INFO:tasks.workunit.client.1.smithi044.stdout:4/104: fsync d17/d18/d1a/f22 0 2024-06-03T01:20:49.683 INFO:tasks.workunit.client.1.smithi044.stdout:4/105: write fc [208086,124692] 0 2024-06-03T01:20:49.686 INFO:tasks.workunit.client.1.smithi044.stdout:7/249: rename d0/d3/d8/f2f to d0/d3/d8/f58 0 2024-06-03T01:20:49.687 INFO:tasks.workunit.client.1.smithi044.stdout:5/102: mkdir d2/d11/d1e 0 2024-06-03T01:20:49.697 INFO:tasks.workunit.client.1.smithi044.stdout:2/106: link db/d1b/l1c db/d1b/l1e 0 2024-06-03T01:20:49.698 INFO:tasks.workunit.client.1.smithi044.stdout:4/106: dwrite fc [0,4194304] 0 2024-06-03T01:20:49.698 INFO:tasks.workunit.client.1.smithi044.stdout:9/132: mkdir d6/d8/d16/d27 0 2024-06-03T01:20:49.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/102: mkdir df/d13/d1b 0 2024-06-03T01:20:49.701 INFO:tasks.workunit.client.1.smithi044.stdout:7/250: link d0/d16/d23/l49 d0/d16/l59 0 2024-06-03T01:20:49.701 INFO:tasks.workunit.client.1.smithi044.stdout:7/251: truncate d0/d16/f3c 736143 0 2024-06-03T01:20:49.702 INFO:tasks.workunit.client.1.smithi044.stdout:5/103: symlink d2/d11/d12/l1f 0 2024-06-03T01:20:49.702 INFO:tasks.workunit.client.1.smithi044.stdout:0/165: sync 2024-06-03T01:20:49.705 INFO:tasks.workunit.client.1.smithi044.stdout:5/104: creat d2/dc/d18/f20 x:0 0 0 2024-06-03T01:20:49.706 INFO:tasks.workunit.client.1.smithi044.stdout:3/103: truncate df/f11 6707339 0 2024-06-03T01:20:49.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/166: getdents d3/d1c/d27 0 2024-06-03T01:20:49.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/167: write d3/d1c/f3e [1437,96322] 0 2024-06-03T01:20:49.707 INFO:tasks.workunit.client.1.smithi044.stdout:9/133: dread d6/d8/d16/f1b [0,4194304] 0 2024-06-03T01:20:49.712 INFO:tasks.workunit.client.1.smithi044.stdout:5/105: mkdir d2/d11/d1e/d21 0 2024-06-03T01:20:49.712 INFO:tasks.workunit.client.1.smithi044.stdout:5/106: fdatasync d2/f13 0 2024-06-03T01:20:49.715 INFO:tasks.workunit.client.1.smithi044.stdout:2/107: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:49.716 INFO:tasks.workunit.client.1.smithi044.stdout:9/134: truncate d6/d8/d16/f1b 1590857 0 2024-06-03T01:20:49.716 INFO:tasks.workunit.client.1.smithi044.stdout:9/135: rename d6 to d6/d8/d16/d28 22 2024-06-03T01:20:49.716 INFO:tasks.workunit.client.1.smithi044.stdout:5/107: truncate d2/dc/f16 1525834 0 2024-06-03T01:20:49.719 INFO:tasks.workunit.client.1.smithi044.stdout:0/168: rmdir d3 39 2024-06-03T01:20:49.719 INFO:tasks.workunit.client.1.smithi044.stdout:0/169: readlink d3/d1c/d27/d29/l31 0 2024-06-03T01:20:49.723 INFO:tasks.workunit.client.1.smithi044.stdout:5/108: truncate d2/f3 1635187 0 2024-06-03T01:20:49.724 INFO:tasks.workunit.client.1.smithi044.stdout:3/104: dread df/d13/f1a [0,4194304] 0 2024-06-03T01:20:49.724 INFO:tasks.workunit.client.1.smithi044.stdout:0/170: mknod d3/d1c/d27/d29/c41 0 2024-06-03T01:20:49.724 INFO:tasks.workunit.client.1.smithi044.stdout:0/171: write d3/f8 [2329336,124474] 0 2024-06-03T01:20:49.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/109: link d2/fa d2/dc/f22 0 2024-06-03T01:20:49.729 INFO:tasks.workunit.client.1.smithi044.stdout:3/105: creat df/f1c x:0 0 0 2024-06-03T01:20:49.729 INFO:tasks.workunit.client.1.smithi044.stdout:3/106: chown df/d13/f16 275905292 1 2024-06-03T01:20:49.732 INFO:tasks.workunit.client.1.smithi044.stdout:0/172: creat d3/d9/d36/f42 x:0 0 0 2024-06-03T01:20:49.732 INFO:tasks.workunit.client.1.smithi044.stdout:0/173: chown d3/d1c/d27/l3d 34496589 1 2024-06-03T01:20:49.732 INFO:tasks.workunit.client.1.smithi044.stdout:0/174: read - d3/f18 zero size 2024-06-03T01:20:49.733 INFO:tasks.workunit.client.1.smithi044.stdout:5/110: link d2/l5 d2/d11/d12/l23 0 2024-06-03T01:20:49.734 INFO:tasks.workunit.client.1.smithi044.stdout:0/175: rmdir d3/d9 39 2024-06-03T01:20:49.734 INFO:tasks.workunit.client.1.smithi044.stdout:0/176: fdatasync d3/f4 0 2024-06-03T01:20:49.734 INFO:tasks.workunit.client.1.smithi044.stdout:0/177: chown d3/d1c/d27/f38 38 1 2024-06-03T01:20:49.734 INFO:tasks.workunit.client.1.smithi044.stdout:0/178: truncate d3/d9/f22 10973 0 2024-06-03T01:20:49.740 INFO:tasks.workunit.client.1.smithi044.stdout:7/252: dwrite d0/d3/d8/f39 [0,4194304] 0 2024-06-03T01:20:49.741 INFO:tasks.workunit.client.1.smithi044.stdout:2/108: dwrite db/f19 [0,4194304] 0 2024-06-03T01:20:49.741 INFO:tasks.workunit.client.1.smithi044.stdout:4/107: dwrite f2 [0,4194304] 0 2024-06-03T01:20:49.742 INFO:tasks.workunit.client.1.smithi044.stdout:4/108: truncate d17/f1e 485352 0 2024-06-03T01:20:49.742 INFO:tasks.workunit.client.1.smithi044.stdout:4/109: chown c13 1539 1 2024-06-03T01:20:49.753 INFO:tasks.workunit.client.1.smithi044.stdout:2/109: creat db/f1f x:0 0 0 2024-06-03T01:20:49.753 INFO:tasks.workunit.client.1.smithi044.stdout:3/107: dwrite df/f1c [0,4194304] 0 2024-06-03T01:20:49.754 INFO:tasks.workunit.client.1.smithi044.stdout:5/111: dwrite d2/f13 [0,4194304] 0 2024-06-03T01:20:49.755 INFO:tasks.workunit.client.1.smithi044.stdout:2/110: symlink db/d15/l20 0 2024-06-03T01:20:49.757 INFO:tasks.workunit.client.1.smithi044.stdout:7/253: dwrite d0/d3/d8/f58 [0,4194304] 0 2024-06-03T01:20:49.757 INFO:tasks.workunit.client.1.smithi044.stdout:7/254: dread - d0/d21/f53 zero size 2024-06-03T01:20:49.757 INFO:tasks.workunit.client.1.smithi044.stdout:7/255: write d0/d3/d8/d32/f4a [880295,68181] 0 2024-06-03T01:20:49.758 INFO:tasks.workunit.client.1.smithi044.stdout:3/108: creat df/d13/f1d x:0 0 0 2024-06-03T01:20:49.758 INFO:tasks.workunit.client.1.smithi044.stdout:3/109: readlink l4 0 2024-06-03T01:20:49.760 INFO:tasks.workunit.client.1.smithi044.stdout:5/112: mkdir d2/d11/d1e/d21/d24 0 2024-06-03T01:20:49.763 INFO:tasks.workunit.client.1.smithi044.stdout:1/161: sync 2024-06-03T01:20:49.763 INFO:tasks.workunit.client.1.smithi044.stdout:1/162: stat d0/d3/f28 0 2024-06-03T01:20:49.766 INFO:tasks.workunit.client.1.smithi044.stdout:4/110: rename d17/d18 to d17/d23 0 2024-06-03T01:20:49.766 INFO:tasks.workunit.client.1.smithi044.stdout:4/111: chown c16 472522 1 2024-06-03T01:20:49.767 INFO:tasks.workunit.client.1.smithi044.stdout:1/163: mknod d0/d3/c37 0 2024-06-03T01:20:49.767 INFO:tasks.workunit.client.1.smithi044.stdout:1/164: chown d0/d3/d20/d7/lf 673909 1 2024-06-03T01:20:49.767 INFO:tasks.workunit.client.1.smithi044.stdout:1/165: read d0/d3/d20/d7/f17 [297935,63825] 0 2024-06-03T01:20:49.769 INFO:tasks.workunit.client.1.smithi044.stdout:4/112: unlink l14 0 2024-06-03T01:20:49.769 INFO:tasks.workunit.client.1.smithi044.stdout:2/111: fsync db/f1f 0 2024-06-03T01:20:49.770 INFO:tasks.workunit.client.1.smithi044.stdout:8/126: sync 2024-06-03T01:20:49.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/166: creat d0/d3/d14/f38 x:0 0 0 2024-06-03T01:20:49.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/167: fsync d0/f2c 0 2024-06-03T01:20:49.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/168: write d0/d3/f28 [1627038,3892] 0 2024-06-03T01:20:49.773 INFO:tasks.workunit.client.1.smithi044.stdout:4/113: creat d17/d23/d1a/f24 x:0 0 0 2024-06-03T01:20:49.773 INFO:tasks.workunit.client.1.smithi044.stdout:2/112: symlink db/d1b/l21 0 2024-06-03T01:20:49.775 INFO:tasks.workunit.client.1.smithi044.stdout:8/127: creat d5/d16/f24 x:0 0 0 2024-06-03T01:20:49.777 INFO:tasks.workunit.client.1.smithi044.stdout:5/113: read d2/f6 [5551976,8882] 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/114: write d2/dc/f10 [4118445,71251] 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/115: fsync d2/dc/d18/f20 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/110: dwrite df/f19 [0,4194304] 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/111: fsync df/f11 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/116: creat d2/d11/d12/f25 x:0 0 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:4/114: link l9 d17/d1c/l25 0 2024-06-03T01:20:49.778 INFO:tasks.workunit.client.1.smithi044.stdout:2/113: creat db/d1b/f22 x:0 0 0 2024-06-03T01:20:49.779 INFO:tasks.workunit.client.1.smithi044.stdout:7/256: dwrite d0/d3/d8/f39 [4194304,4194304] 0 2024-06-03T01:20:49.779 INFO:tasks.workunit.client.1.smithi044.stdout:7/257: readlink d0/l45 0 2024-06-03T01:20:49.786 INFO:tasks.workunit.client.1.smithi044.stdout:4/115: unlink le 0 2024-06-03T01:20:49.786 INFO:tasks.workunit.client.1.smithi044.stdout:4/116: read - d17/d23/f20 zero size 2024-06-03T01:20:49.788 INFO:tasks.workunit.client.1.smithi044.stdout:2/114: symlink db/d1b/l23 0 2024-06-03T01:20:49.791 INFO:tasks.workunit.client.1.smithi044.stdout:1/169: getdents d0/d3/d14 0 2024-06-03T01:20:49.794 INFO:tasks.workunit.client.1.smithi044.stdout:5/117: dwrite d2/dc/f22 [0,4194304] 0 2024-06-03T01:20:49.796 INFO:tasks.workunit.client.1.smithi044.stdout:8/128: rmdir d5/d16 39 2024-06-03T01:20:49.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/117: symlink d17/l26 0 2024-06-03T01:20:49.798 INFO:tasks.workunit.client.1.smithi044.stdout:2/115: mknod db/d15/c24 0 2024-06-03T01:20:49.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/258: dwrite d0/d3/d8/f58 [0,4194304] 0 2024-06-03T01:20:49.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/259: fdatasync d0/d3/d8/d32/f48 0 2024-06-03T01:20:49.802 INFO:tasks.workunit.client.1.smithi044.stdout:5/118: rename d2/d11/l1c to d2/dc/l26 0 2024-06-03T01:20:49.804 INFO:tasks.workunit.client.1.smithi044.stdout:8/129: creat d5/f25 x:0 0 0 2024-06-03T01:20:49.805 INFO:tasks.workunit.client.1.smithi044.stdout:4/118: mknod d17/d23/d1a/c27 0 2024-06-03T01:20:49.806 INFO:tasks.workunit.client.1.smithi044.stdout:4/119: write f15 [792652,114150] 0 2024-06-03T01:20:49.806 INFO:tasks.workunit.client.1.smithi044.stdout:4/120: write d17/d23/d1a/f22 [758646,68849] 0 2024-06-03T01:20:49.806 INFO:tasks.workunit.client.1.smithi044.stdout:2/116: mkdir db/d25 0 2024-06-03T01:20:49.808 INFO:tasks.workunit.client.1.smithi044.stdout:7/260: write d0/d3/d8/f14 [3150342,62598] 0 2024-06-03T01:20:49.809 INFO:tasks.workunit.client.1.smithi044.stdout:7/261: write d0/d21/f28 [4994133,75739] 0 2024-06-03T01:20:49.810 INFO:tasks.workunit.client.1.smithi044.stdout:1/170: rmdir d0 39 2024-06-03T01:20:49.813 INFO:tasks.workunit.client.1.smithi044.stdout:8/130: truncate f2 474556 0 2024-06-03T01:20:49.814 INFO:tasks.workunit.client.1.smithi044.stdout:8/131: write d5/d16/d1d/f23 [109853,117098] 0 2024-06-03T01:20:49.814 INFO:tasks.workunit.client.1.smithi044.stdout:4/121: mkdir d17/d23/d28 0 2024-06-03T01:20:49.814 INFO:tasks.workunit.client.1.smithi044.stdout:2/117: write db/f19 [2409055,68007] 0 2024-06-03T01:20:49.814 INFO:tasks.workunit.client.1.smithi044.stdout:7/262: mknod d0/d3/d8/d32/d51/c5a 0 2024-06-03T01:20:49.820 INFO:tasks.workunit.client.1.smithi044.stdout:5/119: link d2/dc/l26 d2/d11/l27 0 2024-06-03T01:20:49.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/126: sync 2024-06-03T01:20:49.821 INFO:tasks.workunit.client.1.smithi044.stdout:4/122: link d17/d23/l1f d17/d23/d1a/l29 0 2024-06-03T01:20:49.826 INFO:tasks.workunit.client.1.smithi044.stdout:5/120: dread d2/f6 [0,4194304] 0 2024-06-03T01:20:49.826 INFO:tasks.workunit.client.1.smithi044.stdout:8/132: dwrite d5/f25 [0,4194304] 0 2024-06-03T01:20:49.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/127: creat d2/d23/f2e x:0 0 0 2024-06-03T01:20:49.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/128: creat d2/d23/f2f x:0 0 0 2024-06-03T01:20:49.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/129: truncate d2/d6/db/df/f2a 88297 0 2024-06-03T01:20:49.828 INFO:tasks.workunit.client.1.smithi044.stdout:2/118: link f2 db/f26 0 2024-06-03T01:20:49.832 INFO:tasks.workunit.client.1.smithi044.stdout:2/119: mkdir db/d1b/d27 0 2024-06-03T01:20:49.833 INFO:tasks.workunit.client.1.smithi044.stdout:7/263: rename d0/d21/d27/f35 to d0/d3/d8/d32/d33/f5b 0 2024-06-03T01:20:49.834 INFO:tasks.workunit.client.1.smithi044.stdout:8/133: mkdir d5/d26 0 2024-06-03T01:20:49.835 INFO:tasks.workunit.client.1.smithi044.stdout:2/120: mkdir db/dd/d28 0 2024-06-03T01:20:49.837 INFO:tasks.workunit.client.1.smithi044.stdout:4/123: dwrite fd [0,4194304] 0 2024-06-03T01:20:49.837 INFO:tasks.workunit.client.1.smithi044.stdout:4/124: chown fc 762 1 2024-06-03T01:20:49.838 INFO:tasks.workunit.client.1.smithi044.stdout:6/130: rename d2/d6/db/df to d2/d23/d30 0 2024-06-03T01:20:49.838 INFO:tasks.workunit.client.1.smithi044.stdout:6/131: truncate d2/f2c 893887 0 2024-06-03T01:20:49.839 INFO:tasks.workunit.client.1.smithi044.stdout:8/134: creat d5/d16/d1d/f27 x:0 0 0 2024-06-03T01:20:49.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/121: mknod db/dd/c29 0 2024-06-03T01:20:49.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/122: truncate db/f1f 197674 0 2024-06-03T01:20:49.842 INFO:tasks.workunit.client.1.smithi044.stdout:4/125: mknod d17/d23/d28/c2a 0 2024-06-03T01:20:49.842 INFO:tasks.workunit.client.1.smithi044.stdout:7/264: read d0/d3/fa [9200577,44622] 0 2024-06-03T01:20:49.849 INFO:tasks.workunit.client.1.smithi044.stdout:8/135: write d5/f25 [1839649,63875] 0 2024-06-03T01:20:49.849 INFO:tasks.workunit.client.1.smithi044.stdout:8/136: fsync d5/f15 0 2024-06-03T01:20:49.849 INFO:tasks.workunit.client.1.smithi044.stdout:8/137: truncate f2 1048448 0 2024-06-03T01:20:49.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/132: link d2/d23/d30/f2a d2/f31 0 2024-06-03T01:20:49.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/133: readlink d2/d4/la 0 2024-06-03T01:20:49.854 INFO:tasks.workunit.client.1.smithi044.stdout:8/138: rmdir d5/d26 0 2024-06-03T01:20:49.854 INFO:tasks.workunit.client.1.smithi044.stdout:8/139: write d5/d16/d1d/f27 [112210,70460] 0 2024-06-03T01:20:49.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/134: mknod d2/d23/d30/d1e/c32 0 2024-06-03T01:20:49.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/135: stat d2/d23 0 2024-06-03T01:20:49.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/136: write d2/fe [6171778,47132] 0 2024-06-03T01:20:49.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/137: read - d2/d23/d30/f1f zero size 2024-06-03T01:20:49.856 INFO:tasks.workunit.client.1.smithi044.stdout:6/138: fdatasync d2/f31 0 2024-06-03T01:20:49.857 INFO:tasks.workunit.client.1.smithi044.stdout:8/140: rename d5/l1b to d5/d20/l28 0 2024-06-03T01:20:49.858 INFO:tasks.workunit.client.1.smithi044.stdout:8/141: symlink d5/d10/l29 0 2024-06-03T01:20:49.858 INFO:tasks.workunit.client.1.smithi044.stdout:2/123: dwrite db/dd/fe [0,4194304] 0 2024-06-03T01:20:49.859 INFO:tasks.workunit.client.1.smithi044.stdout:7/265: write d0/d21/d27/f50 [955466,66331] 0 2024-06-03T01:20:49.863 INFO:tasks.workunit.client.1.smithi044.stdout:8/142: dread d5/f7 [0,4194304] 0 2024-06-03T01:20:49.863 INFO:tasks.workunit.client.1.smithi044.stdout:8/143: write f2 [860251,60382] 0 2024-06-03T01:20:49.868 INFO:tasks.workunit.client.1.smithi044.stdout:4/126: dwrite d17/d23/d1a/f1b [0,4194304] 0 2024-06-03T01:20:49.868 INFO:tasks.workunit.client.1.smithi044.stdout:4/127: fsync fd 0 2024-06-03T01:20:49.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/139: dwrite d2/f14 [0,4194304] 0 2024-06-03T01:20:49.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/140: dread - d2/d23/d30/f1c zero size 2024-06-03T01:20:49.875 INFO:tasks.workunit.client.1.smithi044.stdout:4/128: rename d17/d23/d1a to d17/d23/d28/d2b 0 2024-06-03T01:20:49.877 INFO:tasks.workunit.client.1.smithi044.stdout:7/266: mkdir d0/d21/d27/d4c/d5c 0 2024-06-03T01:20:49.877 INFO:tasks.workunit.client.1.smithi044.stdout:7/267: chown d0/d16/d23/l4b 479 1 2024-06-03T01:20:49.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/124: dwrite db/f19 [4194304,4194304] 0 2024-06-03T01:20:49.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/125: fsync db/f1f 0 2024-06-03T01:20:49.879 INFO:tasks.workunit.client.1.smithi044.stdout:6/141: link d2/d4/f7 d2/d23/f33 0 2024-06-03T01:20:49.881 INFO:tasks.workunit.client.1.smithi044.stdout:7/268: symlink d0/d3/d8/d32/l5d 0 2024-06-03T01:20:49.883 INFO:tasks.workunit.client.1.smithi044.stdout:8/144: getdents d5/d10 0 2024-06-03T01:20:49.884 INFO:tasks.workunit.client.1.smithi044.stdout:8/145: dread - d5/d16/f24 zero size 2024-06-03T01:20:49.884 INFO:tasks.workunit.client.1.smithi044.stdout:8/146: write d5/ff [4830263,105608] 0 2024-06-03T01:20:49.886 INFO:tasks.workunit.client.1.smithi044.stdout:7/269: creat d0/d3/d8/d32/d51/f5e x:0 0 0 2024-06-03T01:20:49.886 INFO:tasks.workunit.client.1.smithi044.stdout:4/129: read f2 [3651397,81117] 0 2024-06-03T01:20:49.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/142: write d2/f31 [53677,26221] 0 2024-06-03T01:20:49.887 INFO:tasks.workunit.client.1.smithi044.stdout:8/147: fsync f2 0 2024-06-03T01:20:49.889 INFO:tasks.workunit.client.1.smithi044.stdout:4/130: link d17/d23/d28/d2b/f21 d17/d23/d28/f2c 0 2024-06-03T01:20:49.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/148: unlink l1 0 2024-06-03T01:20:49.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/149: chown d5/lc 6147 1 2024-06-03T01:20:49.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/150: fdatasync f2 0 2024-06-03T01:20:49.893 INFO:tasks.workunit.client.1.smithi044.stdout:6/143: mkdir d2/d23/d30/d1e/d34 0 2024-06-03T01:20:49.893 INFO:tasks.workunit.client.1.smithi044.stdout:6/144: creat d2/d23/f35 x:0 0 0 2024-06-03T01:20:49.894 INFO:tasks.workunit.client.1.smithi044.stdout:4/131: truncate fb 4057884 0 2024-06-03T01:20:49.894 INFO:tasks.workunit.client.1.smithi044.stdout:4/132: creat d17/f2d x:0 0 0 2024-06-03T01:20:49.895 INFO:tasks.workunit.client.1.smithi044.stdout:8/151: creat d5/d16/f2a x:0 0 0 2024-06-03T01:20:49.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/145: truncate d2/d23/d30/f19 3887720 0 2024-06-03T01:20:49.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/146: truncate d2/d6/db/f12 754908 0 2024-06-03T01:20:49.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/147: fdatasync d2/d4/f7 0 2024-06-03T01:20:49.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/148: dread - d2/d6/db/f21 zero size 2024-06-03T01:20:49.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/149: truncate d2/d23/d30/f1d 854799 0 2024-06-03T01:20:49.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/150: dread d2/f2c [0,4194304] 0 2024-06-03T01:20:49.901 INFO:tasks.workunit.client.1.smithi044.stdout:4/133: unlink c0 0 2024-06-03T01:20:49.902 INFO:tasks.workunit.client.1.smithi044.stdout:7/270: dwrite d0/d21/f53 [0,4194304] 0 2024-06-03T01:20:49.902 INFO:tasks.workunit.client.1.smithi044.stdout:7/271: write d0/d3/d8/d32/f48 [638819,20459] 0 2024-06-03T01:20:49.904 INFO:tasks.workunit.client.1.smithi044.stdout:4/134: link d17/d23/d28/d2b/f21 d17/d23/d28/d2b/f2e 0 2024-06-03T01:20:49.907 INFO:tasks.workunit.client.1.smithi044.stdout:7/272: unlink d0/f3e 0 2024-06-03T01:20:49.907 INFO:tasks.workunit.client.1.smithi044.stdout:7/273: readlink d0/d21/l44 0 2024-06-03T01:20:49.907 INFO:tasks.workunit.client.1.smithi044.stdout:7/274: chown d0/d3/d8/l2b 10979309 1 2024-06-03T01:20:49.908 INFO:tasks.workunit.client.1.smithi044.stdout:4/135: symlink d17/d23/d28/l2f 0 2024-06-03T01:20:49.908 INFO:tasks.workunit.client.1.smithi044.stdout:4/136: read - d17/f2d zero size 2024-06-03T01:20:49.908 INFO:tasks.workunit.client.1.smithi044.stdout:4/137: chown fc 9898622 1 2024-06-03T01:20:49.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/138: getdents d17/d1c 0 2024-06-03T01:20:49.909 INFO:tasks.workunit.client.1.smithi044.stdout:2/126: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:49.909 INFO:tasks.workunit.client.1.smithi044.stdout:7/275: dread d0/d3/d8/f1b [0,4194304] 0 2024-06-03T01:20:49.910 INFO:tasks.workunit.client.1.smithi044.stdout:6/151: fsync d2/fe 0 2024-06-03T01:20:49.911 INFO:tasks.workunit.client.1.smithi044.stdout:8/152: dwrite d5/fe [0,4194304] 0 2024-06-03T01:20:49.911 INFO:tasks.workunit.client.1.smithi044.stdout:8/153: write f3 [1766849,128368] 0 2024-06-03T01:20:49.912 INFO:tasks.workunit.client.1.smithi044.stdout:2/127: mkdir db/d1b/d2a 0 2024-06-03T01:20:49.912 INFO:tasks.workunit.client.1.smithi044.stdout:2/128: stat db/f26 0 2024-06-03T01:20:49.912 INFO:tasks.workunit.client.1.smithi044.stdout:2/129: fdatasync db/d1b/f22 0 2024-06-03T01:20:49.912 INFO:tasks.workunit.client.1.smithi044.stdout:2/130: dread - db/d1b/f22 zero size 2024-06-03T01:20:49.913 INFO:tasks.workunit.client.1.smithi044.stdout:7/276: symlink d0/d3/d8/d32/l5f 0 2024-06-03T01:20:49.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/152: mknod d2/d23/c36 0 2024-06-03T01:20:49.914 INFO:tasks.workunit.client.1.smithi044.stdout:8/154: dread d5/d16/d1d/f27 [0,4194304] 0 2024-06-03T01:20:49.914 INFO:tasks.workunit.client.1.smithi044.stdout:8/155: write d5/f6 [1530997,74606] 0 2024-06-03T01:20:49.914 INFO:tasks.workunit.client.1.smithi044.stdout:8/156: truncate d5/d16/d1d/f27 185985 0 2024-06-03T01:20:49.916 INFO:tasks.workunit.client.1.smithi044.stdout:2/131: getdents db/d15 0 2024-06-03T01:20:49.919 INFO:tasks.workunit.client.1.smithi044.stdout:6/153: rename d2/c2d to d2/d4/c37 0 2024-06-03T01:20:49.919 INFO:tasks.workunit.client.1.smithi044.stdout:6/154: chown d2/f27 215154347 1 2024-06-03T01:20:49.919 INFO:tasks.workunit.client.1.smithi044.stdout:6/155: chown d2/f31 8 1 2024-06-03T01:20:49.921 INFO:tasks.workunit.client.1.smithi044.stdout:2/132: creat db/d1b/d2a/f2b x:0 0 0 2024-06-03T01:20:49.921 INFO:tasks.workunit.client.1.smithi044.stdout:2/133: truncate db/f19 8546939 0 2024-06-03T01:20:49.924 INFO:tasks.workunit.client.1.smithi044.stdout:7/277: dread d0/d3/fa [4194304,4194304] 0 2024-06-03T01:20:49.924 INFO:tasks.workunit.client.1.smithi044.stdout:7/278: dread - d0/d3/d8/d32/f38 zero size 2024-06-03T01:20:49.924 INFO:tasks.workunit.client.1.smithi044.stdout:7/279: truncate d0/d16/f22 691964 0 2024-06-03T01:20:49.927 INFO:tasks.workunit.client.1.smithi044.stdout:2/134: dread db/f19 [0,4194304] 0 2024-06-03T01:20:49.929 INFO:tasks.workunit.client.1.smithi044.stdout:8/157: dwrite d5/fb [0,4194304] 0 2024-06-03T01:20:49.929 INFO:tasks.workunit.client.1.smithi044.stdout:8/158: write d5/d16/f24 [484087,40845] 0 2024-06-03T01:20:49.931 INFO:tasks.workunit.client.1.smithi044.stdout:2/135: rmdir db/d25 0 2024-06-03T01:20:49.932 INFO:tasks.workunit.client.1.smithi044.stdout:7/280: symlink d0/d21/d27/l60 0 2024-06-03T01:20:49.937 INFO:tasks.workunit.client.1.smithi044.stdout:6/156: dwrite d2/f14 [0,4194304] 0 2024-06-03T01:20:49.942 INFO:tasks.workunit.client.1.smithi044.stdout:8/159: mkdir d5/d20/d2b 0 2024-06-03T01:20:49.944 INFO:tasks.workunit.client.1.smithi044.stdout:2/136: link db/d15/c18 db/d1b/d27/c2c 0 2024-06-03T01:20:49.946 INFO:tasks.workunit.client.1.smithi044.stdout:7/281: symlink d0/d21/l61 0 2024-06-03T01:20:49.946 INFO:tasks.workunit.client.1.smithi044.stdout:7/282: fsync d0/d3/d8/f10 0 2024-06-03T01:20:49.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/283: creat d0/f62 x:0 0 0 2024-06-03T01:20:49.950 INFO:tasks.workunit.client.1.smithi044.stdout:8/160: dread f3 [0,4194304] 0 2024-06-03T01:20:49.950 INFO:tasks.workunit.client.1.smithi044.stdout:6/157: dwrite d2/d23/f2e [0,4194304] 0 2024-06-03T01:20:49.950 INFO:tasks.workunit.client.1.smithi044.stdout:6/158: dread - d2/f27 zero size 2024-06-03T01:20:49.955 INFO:tasks.workunit.client.1.smithi044.stdout:9/136: sync 2024-06-03T01:20:49.955 INFO:tasks.workunit.client.1.smithi044.stdout:0/179: sync 2024-06-03T01:20:49.955 INFO:tasks.workunit.client.1.smithi044.stdout:3/112: sync 2024-06-03T01:20:49.956 INFO:tasks.workunit.client.1.smithi044.stdout:0/180: read - d3/d1c/f3b zero size 2024-06-03T01:20:49.956 INFO:tasks.workunit.client.1.smithi044.stdout:9/137: write d6/d8/db/f15 [1019382,39468] 0 2024-06-03T01:20:49.956 INFO:tasks.workunit.client.1.smithi044.stdout:7/284: dread d0/d3/d8/f14 [0,4194304] 0 2024-06-03T01:20:49.956 INFO:tasks.workunit.client.1.smithi044.stdout:7/285: readlink d0/l55 0 2024-06-03T01:20:49.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/138: symlink d6/l29 0 2024-06-03T01:20:49.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/139: truncate d6/f12 773828 0 2024-06-03T01:20:49.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/140: fsync f2 0 2024-06-03T01:20:49.967 INFO:tasks.workunit.client.1.smithi044.stdout:8/161: dwrite f2 [0,4194304] 0 2024-06-03T01:20:49.967 INFO:tasks.workunit.client.1.smithi044.stdout:8/162: chown f2 255115532 1 2024-06-03T01:20:49.990 INFO:tasks.workunit.client.1.smithi044.stdout:6/159: dwrite d2/f14 [4194304,4194304] 0 2024-06-03T01:20:49.991 INFO:tasks.workunit.client.1.smithi044.stdout:9/141: dwrite d6/d8/d16/f1b [0,4194304] 0 2024-06-03T01:20:49.992 INFO:tasks.workunit.client.1.smithi044.stdout:8/163: dwrite d5/f7 [0,4194304] 0 2024-06-03T01:20:49.992 INFO:tasks.workunit.client.1.smithi044.stdout:8/164: truncate d5/d16/f24 929964 0 2024-06-03T01:20:49.993 INFO:tasks.workunit.client.1.smithi044.stdout:8/165: write d5/d16/f2a [934080,83304] 0 2024-06-03T01:20:49.995 INFO:tasks.workunit.client.1.smithi044.stdout:3/113: dwrite df/f19 [4194304,4194304] 0 2024-06-03T01:20:49.996 INFO:tasks.workunit.client.1.smithi044.stdout:8/166: mkdir d5/d20/d2c 0 2024-06-03T01:20:50.002 INFO:tasks.workunit.client.1.smithi044.stdout:8/167: link d5/d16/d1d/f27 d5/f2d 0 2024-06-03T01:20:50.004 INFO:tasks.workunit.client.1.smithi044.stdout:9/142: write f4 [261629,35491] 0 2024-06-03T01:20:50.005 INFO:tasks.workunit.client.1.smithi044.stdout:8/168: unlink d5/d20/l28 0 2024-06-03T01:20:50.007 INFO:tasks.workunit.client.1.smithi044.stdout:9/143: symlink d6/dd/l2a 0 2024-06-03T01:20:50.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/144: mknod d6/c2b 0 2024-06-03T01:20:50.010 INFO:tasks.workunit.client.1.smithi044.stdout:6/160: dwrite d2/f2c [0,4194304] 0 2024-06-03T01:20:50.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/145: truncate d6/dd/f21 4441323 0 2024-06-03T01:20:50.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/146: chown f4 13935052 1 2024-06-03T01:20:50.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/147: chown d6/d8/fe 6 1 2024-06-03T01:20:50.019 INFO:tasks.workunit.client.1.smithi044.stdout:3/114: dwrite f9 [4194304,4194304] 0 2024-06-03T01:20:50.023 INFO:tasks.workunit.client.1.smithi044.stdout:3/115: mkdir df/d1e 0 2024-06-03T01:20:50.023 INFO:tasks.workunit.client.1.smithi044.stdout:6/161: write d2/d23/f33 [2057056,14287] 0 2024-06-03T01:20:50.024 INFO:tasks.workunit.client.1.smithi044.stdout:6/162: mkdir d2/d23/d30/d1e/d38 0 2024-06-03T01:20:50.032 INFO:tasks.workunit.client.1.smithi044.stdout:3/116: write df/f11 [671443,92424] 0 2024-06-03T01:20:50.034 INFO:tasks.workunit.client.1.smithi044.stdout:8/169: dwrite d5/f11 [0,4194304] 0 2024-06-03T01:20:50.034 INFO:tasks.workunit.client.1.smithi044.stdout:8/170: creat d5/d16/f2e x:0 0 0 2024-06-03T01:20:50.037 INFO:tasks.workunit.client.1.smithi044.stdout:1/171: sync 2024-06-03T01:20:50.037 INFO:tasks.workunit.client.1.smithi044.stdout:5/121: sync 2024-06-03T01:20:50.037 INFO:tasks.workunit.client.1.smithi044.stdout:8/171: mknod d5/d20/c2f 0 2024-06-03T01:20:50.039 INFO:tasks.workunit.client.1.smithi044.stdout:1/172: truncate d0/d3/d20/d7/f36 396939 0 2024-06-03T01:20:50.041 INFO:tasks.workunit.client.1.smithi044.stdout:5/122: dread d2/f3 [0,4194304] 0 2024-06-03T01:20:50.041 INFO:tasks.workunit.client.1.smithi044.stdout:1/173: link d0/d3/d20/d7/d2f/c31 d0/d3/d14/d30/c39 0 2024-06-03T01:20:50.041 INFO:tasks.workunit.client.1.smithi044.stdout:8/172: rmdir d5/d16/d1d 39 2024-06-03T01:20:50.042 INFO:tasks.workunit.client.1.smithi044.stdout:8/173: getdents d5/d10 0 2024-06-03T01:20:50.042 INFO:tasks.workunit.client.1.smithi044.stdout:8/174: creat d5/d20/d2b/f30 x:0 0 0 2024-06-03T01:20:50.043 INFO:tasks.workunit.client.1.smithi044.stdout:5/123: dread d2/f3 [0,4194304] 0 2024-06-03T01:20:50.044 INFO:tasks.workunit.client.1.smithi044.stdout:5/124: mkdir d2/dc/d28 0 2024-06-03T01:20:50.047 INFO:tasks.workunit.client.1.smithi044.stdout:6/163: dwrite d2/d23/d30/f2a [0,4194304] 0 2024-06-03T01:20:50.049 INFO:tasks.workunit.client.1.smithi044.stdout:6/164: mkdir d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:50.051 INFO:tasks.workunit.client.1.smithi044.stdout:6/165: truncate d2/f14 3540136 0 2024-06-03T01:20:50.052 INFO:tasks.workunit.client.1.smithi044.stdout:6/166: creat d2/d23/d30/f3a x:0 0 0 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:6/167: truncate d2/d4/f7 2248020 0 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:6/168: readlink d2/d6/db/l1b 0 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:6/169: stat d2/d6/cd 0 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/175: rmdir d5 39 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/176: chown d5/d10/l29 26310767 1 2024-06-03T01:20:50.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/177: write d5/f15 [5417762,46945] 0 2024-06-03T01:20:50.055 INFO:tasks.workunit.client.1.smithi044.stdout:8/178: dread d5/d16/d1d/f27 [0,4194304] 0 2024-06-03T01:20:50.055 INFO:tasks.workunit.client.1.smithi044.stdout:8/179: fsync d5/fb 0 2024-06-03T01:20:50.056 INFO:tasks.workunit.client.1.smithi044.stdout:8/180: rename d5/fb to d5/d16/d1d/f31 0 2024-06-03T01:20:50.057 INFO:tasks.workunit.client.1.smithi044.stdout:8/181: mkdir d5/d16/d1d/d32 0 2024-06-03T01:20:50.057 INFO:tasks.workunit.client.1.smithi044.stdout:8/182: rmdir d5/d16/d1d 39 2024-06-03T01:20:50.057 INFO:tasks.workunit.client.1.smithi044.stdout:1/174: dread d0/d3/d20/d7/f36 [0,4194304] 0 2024-06-03T01:20:50.059 INFO:tasks.workunit.client.1.smithi044.stdout:8/183: mkdir d5/d20/d33 0 2024-06-03T01:20:50.059 INFO:tasks.workunit.client.1.smithi044.stdout:8/184: chown d5/d16/l18 10555 1 2024-06-03T01:20:50.060 INFO:tasks.workunit.client.1.smithi044.stdout:8/185: creat d5/d10/f34 x:0 0 0 2024-06-03T01:20:50.060 INFO:tasks.workunit.client.1.smithi044.stdout:8/186: truncate d5/d10/f34 309457 0 2024-06-03T01:20:50.060 INFO:tasks.workunit.client.1.smithi044.stdout:8/187: getdents d5/d16/d1d/d32 0 2024-06-03T01:20:50.063 INFO:tasks.workunit.client.1.smithi044.stdout:6/170: write d2/fe [3072720,34276] 0 2024-06-03T01:20:50.063 INFO:tasks.workunit.client.1.smithi044.stdout:6/171: chown d2/d6/f13 1 1 2024-06-03T01:20:50.064 INFO:tasks.workunit.client.1.smithi044.stdout:5/125: dwrite d2/d11/d12/f1b [0,4194304] 0 2024-06-03T01:20:50.064 INFO:tasks.workunit.client.1.smithi044.stdout:5/126: write d2/dc/f15 [921726,106130] 0 2024-06-03T01:20:50.064 INFO:tasks.workunit.client.1.smithi044.stdout:5/127: creat d2/d11/d12/f29 x:0 0 0 2024-06-03T01:20:50.066 INFO:tasks.workunit.client.1.smithi044.stdout:6/172: fsync d2/d23/f33 0 2024-06-03T01:20:50.068 INFO:tasks.workunit.client.1.smithi044.stdout:1/175: dread d0/f15 [0,4194304] 0 2024-06-03T01:20:50.070 INFO:tasks.workunit.client.1.smithi044.stdout:5/128: mknod d2/dc/c2a 0 2024-06-03T01:20:50.073 INFO:tasks.workunit.client.1.smithi044.stdout:1/176: creat d0/d16/f3a x:0 0 0 2024-06-03T01:20:50.073 INFO:tasks.workunit.client.1.smithi044.stdout:1/177: stat d0/f5 0 2024-06-03T01:20:50.074 INFO:tasks.workunit.client.1.smithi044.stdout:6/173: rmdir d2/d23/d30 39 2024-06-03T01:20:50.076 INFO:tasks.workunit.client.1.smithi044.stdout:6/174: rename d2/d23/f2f to d2/d6/db/f3b 0 2024-06-03T01:20:50.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/175: fdatasync d2/d23/d30/f1d 0 2024-06-03T01:20:50.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/176: fsync d2/d6/f13 0 2024-06-03T01:20:50.078 INFO:tasks.workunit.client.1.smithi044.stdout:1/178: link d0/c1c d0/d3/d14/d30/c3b 0 2024-06-03T01:20:50.078 INFO:tasks.workunit.client.1.smithi044.stdout:6/177: creat d2/d23/d30/d1e/d34/d39/f3c x:0 0 0 2024-06-03T01:20:50.079 INFO:tasks.workunit.client.1.smithi044.stdout:6/178: mknod d2/d23/d30/d1e/d34/c3d 0 2024-06-03T01:20:50.080 INFO:tasks.workunit.client.1.smithi044.stdout:6/179: fsync d2/d23/f35 0 2024-06-03T01:20:50.081 INFO:tasks.workunit.client.1.smithi044.stdout:1/179: write d0/d3/d20/d7/d13/f24 [78335,20603] 0 2024-06-03T01:20:50.082 INFO:tasks.workunit.client.1.smithi044.stdout:5/129: write d2/d11/d12/f1b [785345,104709] 0 2024-06-03T01:20:50.082 INFO:tasks.workunit.client.1.smithi044.stdout:8/188: dwrite d5/d16/d1d/f27 [0,4194304] 0 2024-06-03T01:20:50.084 INFO:tasks.workunit.client.1.smithi044.stdout:1/180: symlink d0/d3/d20/d7/d2f/l3c 0 2024-06-03T01:20:50.086 INFO:tasks.workunit.client.1.smithi044.stdout:5/130: truncate d2/fa 3657271 0 2024-06-03T01:20:50.088 INFO:tasks.workunit.client.1.smithi044.stdout:8/189: rmdir d5/d20/d2c 0 2024-06-03T01:20:50.089 INFO:tasks.workunit.client.1.smithi044.stdout:1/181: creat d0/d3/d20/d7/d13/f3d x:0 0 0 2024-06-03T01:20:50.091 INFO:tasks.workunit.client.1.smithi044.stdout:5/131: mkdir d2/d11/d12/d2b 0 2024-06-03T01:20:50.091 INFO:tasks.workunit.client.1.smithi044.stdout:5/132: stat d2/f3 0 2024-06-03T01:20:50.091 INFO:tasks.workunit.client.1.smithi044.stdout:5/133: chown d2/dc/c2a 582791045 1 2024-06-03T01:20:50.092 INFO:tasks.workunit.client.1.smithi044.stdout:8/190: creat d5/d20/d2b/f35 x:0 0 0 2024-06-03T01:20:50.092 INFO:tasks.workunit.client.1.smithi044.stdout:8/191: dread - d5/d20/d2b/f35 zero size 2024-06-03T01:20:50.093 INFO:tasks.workunit.client.1.smithi044.stdout:5/134: creat d2/dc/d28/f2c x:0 0 0 2024-06-03T01:20:50.093 INFO:tasks.workunit.client.1.smithi044.stdout:5/135: chown d2/f3 0 1 2024-06-03T01:20:50.094 INFO:tasks.workunit.client.1.smithi044.stdout:5/136: creat d2/d11/d12/f2d x:0 0 0 2024-06-03T01:20:50.094 INFO:tasks.workunit.client.1.smithi044.stdout:8/192: symlink d5/d10/l36 0 2024-06-03T01:20:50.101 INFO:tasks.workunit.client.1.smithi044.stdout:5/137: dread d2/f8 [0,4194304] 0 2024-06-03T01:20:50.101 INFO:tasks.workunit.client.1.smithi044.stdout:5/138: write f1 [2651912,22578] 0 2024-06-03T01:20:50.108 INFO:tasks.workunit.client.1.smithi044.stdout:1/182: dwrite d0/d3/d20/d7/f34 [0,4194304] 0 2024-06-03T01:20:50.109 INFO:tasks.workunit.client.1.smithi044.stdout:6/180: dwrite d2/d23/d30/f2a [0,4194304] 0 2024-06-03T01:20:50.111 INFO:tasks.workunit.client.1.smithi044.stdout:1/183: symlink d0/d16/l3e 0 2024-06-03T01:20:50.111 INFO:tasks.workunit.client.1.smithi044.stdout:1/184: chown d0/d3/l11 935938923 1 2024-06-03T01:20:50.117 INFO:tasks.workunit.client.1.smithi044.stdout:6/181: truncate d2/d23/f33 1739847 0 2024-06-03T01:20:50.117 INFO:tasks.workunit.client.1.smithi044.stdout:6/182: fdatasync d2/f2c 0 2024-06-03T01:20:50.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/185: readlink d0/l2a 0 2024-06-03T01:20:50.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/186: truncate d0/f32 781523 0 2024-06-03T01:20:50.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/187: chown d0/d3/d20/f1d 2 1 2024-06-03T01:20:50.122 INFO:tasks.workunit.client.1.smithi044.stdout:5/139: dwrite d2/dc/f16 [0,4194304] 0 2024-06-03T01:20:50.123 INFO:tasks.workunit.client.1.smithi044.stdout:5/140: fsync f1 0 2024-06-03T01:20:50.125 INFO:tasks.workunit.client.1.smithi044.stdout:8/193: dwrite d5/f15 [0,4194304] 0 2024-06-03T01:20:50.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/141: creat d2/f2e x:0 0 0 2024-06-03T01:20:50.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/142: truncate d2/d11/d12/f29 581724 0 2024-06-03T01:20:50.127 INFO:tasks.workunit.client.1.smithi044.stdout:8/194: unlink d5/d10/l22 0 2024-06-03T01:20:50.129 INFO:tasks.workunit.client.1.smithi044.stdout:5/143: mkdir d2/d11/d1e/d2f 0 2024-06-03T01:20:50.131 INFO:tasks.workunit.client.1.smithi044.stdout:8/195: rename d5/d16/l18 to d5/d16/d1d/d32/l37 0 2024-06-03T01:20:50.131 INFO:tasks.workunit.client.1.smithi044.stdout:8/196: readlink d5/d10/l29 0 2024-06-03T01:20:50.131 INFO:tasks.workunit.client.1.smithi044.stdout:6/183: dread d2/f2c [0,4194304] 0 2024-06-03T01:20:50.131 INFO:tasks.workunit.client.1.smithi044.stdout:6/184: stat d2/d4/l8 0 2024-06-03T01:20:50.131 INFO:tasks.workunit.client.1.smithi044.stdout:6/185: creat d2/d6/f3e x:0 0 0 2024-06-03T01:20:50.132 INFO:tasks.workunit.client.1.smithi044.stdout:8/197: creat d5/d16/d1d/f38 x:0 0 0 2024-06-03T01:20:50.134 INFO:tasks.workunit.client.1.smithi044.stdout:6/186: mkdir d2/d4/d3f 0 2024-06-03T01:20:50.135 INFO:tasks.workunit.client.1.smithi044.stdout:6/187: symlink d2/d4/l40 0 2024-06-03T01:20:50.137 INFO:tasks.workunit.client.1.smithi044.stdout:5/144: dread d2/dc/f10 [0,4194304] 0 2024-06-03T01:20:50.137 INFO:tasks.workunit.client.1.smithi044.stdout:1/188: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-03T01:20:50.137 INFO:tasks.workunit.client.1.smithi044.stdout:5/145: chown d2/d11/l27 784 1 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/146: mknod d2/dc/c30 0 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/147: chown d2/d11/d1e 0 1 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/148: dread - d2/dc/d28/f2c zero size 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/149: write d2/d11/d12/f2d [861864,129253] 0 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/150: fsync d2/dc/d18/f20 0 2024-06-03T01:20:50.140 INFO:tasks.workunit.client.1.smithi044.stdout:1/189: creat d0/f3f x:0 0 0 2024-06-03T01:20:50.142 INFO:tasks.workunit.client.1.smithi044.stdout:6/188: write d2/d23/f2e [3535006,1354] 0 2024-06-03T01:20:50.143 INFO:tasks.workunit.client.1.smithi044.stdout:6/189: dread - d2/d6/db/f21 zero size 2024-06-03T01:20:50.143 INFO:tasks.workunit.client.1.smithi044.stdout:5/151: rename d2/dc/c30 to d2/d11/d1e/c31 0 2024-06-03T01:20:50.143 INFO:tasks.workunit.client.1.smithi044.stdout:5/152: truncate d2/d11/d12/f25 803058 0 2024-06-03T01:20:50.145 INFO:tasks.workunit.client.1.smithi044.stdout:6/190: symlink d2/d6/l41 0 2024-06-03T01:20:50.145 INFO:tasks.workunit.client.1.smithi044.stdout:6/191: readlink d2/d6/db/l1b 0 2024-06-03T01:20:50.147 INFO:tasks.workunit.client.1.smithi044.stdout:5/153: rename d2/f6 to d2/dc/f32 0 2024-06-03T01:20:50.149 INFO:tasks.workunit.client.1.smithi044.stdout:6/192: link d2/d23/d30/f1f d2/d23/d30/d1e/d34/d39/f42 0 2024-06-03T01:20:50.149 INFO:tasks.workunit.client.1.smithi044.stdout:6/193: truncate d2/d23/d30/f3a 484393 0 2024-06-03T01:20:50.149 INFO:tasks.workunit.client.1.smithi044.stdout:6/194: mkdir d2/d23/d2b/d43 0 2024-06-03T01:20:50.150 INFO:tasks.workunit.client.1.smithi044.stdout:6/195: chown d2/d4/c10 3278341 1 2024-06-03T01:20:50.157 INFO:tasks.workunit.client.1.smithi044.stdout:6/196: rename d2/d23/d2b to d2/d23/d44 0 2024-06-03T01:20:50.158 INFO:tasks.workunit.client.1.smithi044.stdout:6/197: getdents d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:50.159 INFO:tasks.workunit.client.1.smithi044.stdout:6/198: truncate d2/d23/f33 234064 0 2024-06-03T01:20:50.159 INFO:tasks.workunit.client.1.smithi044.stdout:6/199: stat d2/d4/f7 0 2024-06-03T01:20:50.161 INFO:tasks.workunit.client.1.smithi044.stdout:6/200: rmdir d2/d23/d44/d43 0 2024-06-03T01:20:50.161 INFO:tasks.workunit.client.1.smithi044.stdout:5/154: dwrite d2/d11/d12/f1b [0,4194304] 0 2024-06-03T01:20:50.161 INFO:tasks.workunit.client.1.smithi044.stdout:5/155: truncate d2/f8 4942478 0 2024-06-03T01:20:50.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/156: rename d2/d11/d1e/c31 to d2/c33 0 2024-06-03T01:20:50.166 INFO:tasks.workunit.client.1.smithi044.stdout:4/139: sync 2024-06-03T01:20:50.166 INFO:tasks.workunit.client.1.smithi044.stdout:5/157: rename d2/dc/d18/f20 to d2/d11/d12/d2b/f34 0 2024-06-03T01:20:50.169 INFO:tasks.workunit.client.1.smithi044.stdout:5/158: truncate d2/fa 378114 0 2024-06-03T01:20:50.171 INFO:tasks.workunit.client.1.smithi044.stdout:5/159: unlink d2/f3 0 2024-06-03T01:20:50.172 INFO:tasks.workunit.client.1.smithi044.stdout:5/160: mknod d2/d11/d1e/d21/c35 0 2024-06-03T01:20:50.173 INFO:tasks.workunit.client.1.smithi044.stdout:5/161: symlink d2/d11/d1e/l36 0 2024-06-03T01:20:50.186 INFO:tasks.workunit.client.1.smithi044.stdout:4/140: dwrite f5 [0,4194304] 0 2024-06-03T01:20:50.187 INFO:tasks.workunit.client.1.smithi044.stdout:6/201: dwrite d2/d23/d30/f1c [0,4194304] 0 2024-06-03T01:20:50.194 INFO:tasks.workunit.client.1.smithi044.stdout:6/202: creat d2/d23/d30/f45 x:0 0 0 2024-06-03T01:20:50.195 INFO:tasks.workunit.client.1.smithi044.stdout:6/203: dread - d2/d6/f3e zero size 2024-06-03T01:20:50.197 INFO:tasks.workunit.client.1.smithi044.stdout:6/204: link d2/d23/d30/f1c d2/d4/d3f/f46 0 2024-06-03T01:20:50.200 INFO:tasks.workunit.client.1.smithi044.stdout:4/141: rename fb to d17/d23/f30 0 2024-06-03T01:20:50.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/162: dwrite d2/fa [0,4194304] 0 2024-06-03T01:20:50.200 INFO:tasks.workunit.client.1.smithi044.stdout:4/142: creat d17/f31 x:0 0 0 2024-06-03T01:20:50.200 INFO:tasks.workunit.client.1.smithi044.stdout:4/143: fdatasync d17/d23/d28/d2b/f1b 0 2024-06-03T01:20:50.201 INFO:tasks.workunit.client.1.smithi044.stdout:6/205: rmdir d2/d23/d30/d1e/d34/d39 39 2024-06-03T01:20:50.201 INFO:tasks.workunit.client.1.smithi044.stdout:6/206: write d2/d4/f7 [1271338,106660] 0 2024-06-03T01:20:50.205 INFO:tasks.workunit.client.1.smithi044.stdout:4/144: mkdir d17/d23/d28/d32 0 2024-06-03T01:20:50.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/163: getdents d2/d11/d12 0 2024-06-03T01:20:50.213 INFO:tasks.workunit.client.1.smithi044.stdout:4/145: rename d17/f1e to d17/d23/d28/f33 0 2024-06-03T01:20:50.215 INFO:tasks.workunit.client.1.smithi044.stdout:4/146: creat d17/d23/d28/d2b/f34 x:0 0 0 2024-06-03T01:20:50.221 INFO:tasks.workunit.client.1.smithi044.stdout:6/207: truncate d2/d6/f13 677828 0 2024-06-03T01:20:50.225 INFO:tasks.workunit.client.1.smithi044.stdout:5/164: dwrite d2/d11/d12/f1b [0,4194304] 0 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:0/181: sync 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/286: sync 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/137: sync 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/138: fsync f7 0 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/139: creat db/d1b/d2a/f2d x:0 0 0 2024-06-03T01:20:50.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/140: fdatasync f2 0 2024-06-03T01:20:50.228 INFO:tasks.workunit.client.1.smithi044.stdout:2/141: write db/f1f [1097230,32929] 0 2024-06-03T01:20:50.229 INFO:tasks.workunit.client.1.smithi044.stdout:0/182: rmdir d3/d1c/d27 39 2024-06-03T01:20:50.229 INFO:tasks.workunit.client.1.smithi044.stdout:0/183: creat d3/d9/d36/f43 x:0 0 0 2024-06-03T01:20:50.230 INFO:tasks.workunit.client.1.smithi044.stdout:7/287: mkdir d0/d21/d63 0 2024-06-03T01:20:50.230 INFO:tasks.workunit.client.1.smithi044.stdout:7/288: chown d0/d3/d30 1315 1 2024-06-03T01:20:50.230 INFO:tasks.workunit.client.1.smithi044.stdout:7/289: fdatasync d0/d3/d8/f1b 0 2024-06-03T01:20:50.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/165: dread f1 [0,4194304] 0 2024-06-03T01:20:50.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/166: read d2/d11/d12/f2d [874201,23100] 0 2024-06-03T01:20:50.234 INFO:tasks.workunit.client.1.smithi044.stdout:0/184: creat d3/d9/f44 x:0 0 0 2024-06-03T01:20:50.234 INFO:tasks.workunit.client.1.smithi044.stdout:0/185: readlink d3/d9/l2b 0 2024-06-03T01:20:50.234 INFO:tasks.workunit.client.1.smithi044.stdout:4/147: dwrite d17/f2d [0,4194304] 0 2024-06-03T01:20:50.235 INFO:tasks.workunit.client.1.smithi044.stdout:5/167: creat d2/d11/d12/f37 x:0 0 0 2024-06-03T01:20:50.240 INFO:tasks.workunit.client.1.smithi044.stdout:5/168: dread d2/dc/f32 [0,4194304] 0 2024-06-03T01:20:50.242 INFO:tasks.workunit.client.1.smithi044.stdout:2/142: dread db/f19 [4194304,4194304] 0 2024-06-03T01:20:50.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/208: dwrite d2/d23/d30/d1e/d34/d39/f42 [0,4194304] 0 2024-06-03T01:20:50.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/169: creat d2/d11/d1e/d21/d24/f38 x:0 0 0 2024-06-03T01:20:50.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/170: fdatasync d2/fa 0 2024-06-03T01:20:50.245 INFO:tasks.workunit.client.1.smithi044.stdout:6/209: creat d2/d23/d30/d1e/f47 x:0 0 0 2024-06-03T01:20:50.248 INFO:tasks.workunit.client.1.smithi044.stdout:6/210: link d2/f14 d2/d23/d30/d1e/f48 0 2024-06-03T01:20:50.248 INFO:tasks.workunit.client.1.smithi044.stdout:6/211: creat d2/d23/d30/d1e/f49 x:0 0 0 2024-06-03T01:20:50.250 INFO:tasks.workunit.client.1.smithi044.stdout:6/212: symlink d2/d4/l4a 0 2024-06-03T01:20:50.255 INFO:tasks.workunit.client.1.smithi044.stdout:7/290: dwrite d0/d21/f46 [0,4194304] 0 2024-06-03T01:20:50.256 INFO:tasks.workunit.client.1.smithi044.stdout:7/291: mknod d0/d16/d23/c64 0 2024-06-03T01:20:50.256 INFO:tasks.workunit.client.1.smithi044.stdout:7/292: fsync d0/f1d 0 2024-06-03T01:20:50.256 INFO:tasks.workunit.client.1.smithi044.stdout:7/293: chown d0/fd 351774 1 2024-06-03T01:20:50.259 INFO:tasks.workunit.client.1.smithi044.stdout:7/294: dread d0/d16/f22 [0,4194304] 0 2024-06-03T01:20:50.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/186: dread d3/d1c/d21/f26 [0,4194304] 0 2024-06-03T01:20:50.264 INFO:tasks.workunit.client.1.smithi044.stdout:5/171: dwrite d2/dc/f32 [0,4194304] 0 2024-06-03T01:20:50.266 INFO:tasks.workunit.client.1.smithi044.stdout:0/187: creat d3/f45 x:0 0 0 2024-06-03T01:20:50.266 INFO:tasks.workunit.client.1.smithi044.stdout:0/188: write d3/d1c/d20/d25/f30 [890857,119059] 0 2024-06-03T01:20:50.266 INFO:tasks.workunit.client.1.smithi044.stdout:6/213: write d2/d23/d30/f1c [617184,51024] 0 2024-06-03T01:20:50.267 INFO:tasks.workunit.client.1.smithi044.stdout:6/214: read d2/d4/f7 [276766,40578] 0 2024-06-03T01:20:50.269 INFO:tasks.workunit.client.1.smithi044.stdout:0/189: mkdir d3/d9/d36/d46 0 2024-06-03T01:20:50.269 INFO:tasks.workunit.client.1.smithi044.stdout:0/190: chown d3/c7 30050 1 2024-06-03T01:20:50.270 INFO:tasks.workunit.client.1.smithi044.stdout:7/295: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:50.270 INFO:tasks.workunit.client.1.smithi044.stdout:7/296: creat d0/d21/f65 x:0 0 0 2024-06-03T01:20:50.272 INFO:tasks.workunit.client.1.smithi044.stdout:2/143: dwrite fa [0,4194304] 0 2024-06-03T01:20:50.273 INFO:tasks.workunit.client.1.smithi044.stdout:6/215: creat d2/d4/f4b x:0 0 0 2024-06-03T01:20:50.274 INFO:tasks.workunit.client.1.smithi044.stdout:5/172: write d2/f13 [606240,30302] 0 2024-06-03T01:20:50.275 INFO:tasks.workunit.client.1.smithi044.stdout:5/173: creat d2/d11/d12/f39 x:0 0 0 2024-06-03T01:20:50.277 INFO:tasks.workunit.client.1.smithi044.stdout:7/297: rmdir d0/d21/d27 39 2024-06-03T01:20:50.282 INFO:tasks.workunit.client.1.smithi044.stdout:6/216: mkdir d2/d6/d4c 0 2024-06-03T01:20:50.282 INFO:tasks.workunit.client.1.smithi044.stdout:6/217: truncate d2/d6/f3e 590029 0 2024-06-03T01:20:50.285 INFO:tasks.workunit.client.1.smithi044.stdout:2/144: dread f2 [0,4194304] 0 2024-06-03T01:20:50.286 INFO:tasks.workunit.client.1.smithi044.stdout:6/218: mkdir d2/d23/d44/d4d 0 2024-06-03T01:20:50.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/219: symlink d2/l4e 0 2024-06-03T01:20:50.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/220: write d2/d23/d30/f1c [1447016,23099] 0 2024-06-03T01:20:50.292 INFO:tasks.workunit.client.1.smithi044.stdout:2/145: write db/f19 [6634352,81299] 0 2024-06-03T01:20:50.299 INFO:tasks.workunit.client.1.smithi044.stdout:5/174: dwrite d2/d11/d12/f37 [0,4194304] 0 2024-06-03T01:20:50.299 INFO:tasks.workunit.client.1.smithi044.stdout:5/175: fdatasync d2/d11/d12/f2d 0 2024-06-03T01:20:50.299 INFO:tasks.workunit.client.1.smithi044.stdout:5/176: chown d2/f2e 1299 1 2024-06-03T01:20:50.301 INFO:tasks.workunit.client.1.smithi044.stdout:0/191: dwrite d3/d9/f44 [0,4194304] 0 2024-06-03T01:20:50.301 INFO:tasks.workunit.client.1.smithi044.stdout:0/192: chown d3/d1c/d27 5571940 1 2024-06-03T01:20:50.301 INFO:tasks.workunit.client.1.smithi044.stdout:0/193: readlink d3/d1c/d27/d29/l31 0 2024-06-03T01:20:50.302 INFO:tasks.workunit.client.1.smithi044.stdout:5/177: symlink d2/dc/d18/l3a 0 2024-06-03T01:20:50.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/117: sync 2024-06-03T01:20:50.303 INFO:tasks.workunit.client.1.smithi044.stdout:9/148: sync 2024-06-03T01:20:50.306 INFO:tasks.workunit.client.1.smithi044.stdout:0/194: unlink d3/d9/d36/c3c 0 2024-06-03T01:20:50.306 INFO:tasks.workunit.client.1.smithi044.stdout:0/195: write d3/d1c/d27/f3f [1069625,887] 0 2024-06-03T01:20:50.308 INFO:tasks.workunit.client.1.smithi044.stdout:9/149: dread d6/d8/db/f15 [0,4194304] 0 2024-06-03T01:20:50.308 INFO:tasks.workunit.client.1.smithi044.stdout:9/150: chown d6/d8/f13 48753633 1 2024-06-03T01:20:50.308 INFO:tasks.workunit.client.1.smithi044.stdout:9/151: creat d6/dd/f2c x:0 0 0 2024-06-03T01:20:50.312 INFO:tasks.workunit.client.1.smithi044.stdout:9/152: rename d6/c2b to d6/d8/d16/c2d 0 2024-06-03T01:20:50.313 INFO:tasks.workunit.client.1.smithi044.stdout:9/153: write f4 [4172173,84925] 0 2024-06-03T01:20:50.314 INFO:tasks.workunit.client.1.smithi044.stdout:8/198: sync 2024-06-03T01:20:50.315 INFO:tasks.workunit.client.1.smithi044.stdout:7/298: dwrite d0/d16/f3c [0,4194304] 0 2024-06-03T01:20:50.315 INFO:tasks.workunit.client.1.smithi044.stdout:2/146: dwrite db/d1b/d2a/f2d [0,4194304] 0 2024-06-03T01:20:50.316 INFO:tasks.workunit.client.1.smithi044.stdout:8/199: unlink d5/d16/f2a 0 2024-06-03T01:20:50.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/196: write d3/d9/f1d [2040936,41360] 0 2024-06-03T01:20:50.320 INFO:tasks.workunit.client.1.smithi044.stdout:3/118: dwrite fe [4194304,4194304] 0 2024-06-03T01:20:50.321 INFO:tasks.workunit.client.1.smithi044.stdout:7/299: link d0/d21/f28 d0/d3/d8/d32/d51/f66 0 2024-06-03T01:20:50.322 INFO:tasks.workunit.client.1.smithi044.stdout:8/200: dread f3 [0,4194304] 0 2024-06-03T01:20:50.322 INFO:tasks.workunit.client.1.smithi044.stdout:8/201: write d5/f7 [4241946,122056] 0 2024-06-03T01:20:50.323 INFO:tasks.workunit.client.1.smithi044.stdout:2/147: link db/dd/l16 db/d15/l2e 0 2024-06-03T01:20:50.326 INFO:tasks.workunit.client.1.smithi044.stdout:2/148: write f2 [1236985,300] 0 2024-06-03T01:20:50.328 INFO:tasks.workunit.client.1.smithi044.stdout:9/154: getdents d6/d8/d16 0 2024-06-03T01:20:50.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/197: symlink d3/d1c/d21/l47 0 2024-06-03T01:20:50.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/198: creat d3/d9/d36/f48 x:0 0 0 2024-06-03T01:20:50.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/199: write d3/d1c/f3b [89053,70386] 0 2024-06-03T01:20:50.331 INFO:tasks.workunit.client.1.smithi044.stdout:0/200: stat d3/d1c/d27/l3d 0 2024-06-03T01:20:50.331 INFO:tasks.workunit.client.1.smithi044.stdout:0/201: fsync d3/d1c/d20/d25/f40 0 2024-06-03T01:20:50.331 INFO:tasks.workunit.client.1.smithi044.stdout:8/202: dread d5/f15 [0,4194304] 0 2024-06-03T01:20:50.331 INFO:tasks.workunit.client.1.smithi044.stdout:8/203: creat d5/d16/d1d/f39 x:0 0 0 2024-06-03T01:20:50.332 INFO:tasks.workunit.client.1.smithi044.stdout:3/119: rename c5 to df/c1f 0 2024-06-03T01:20:50.333 INFO:tasks.workunit.client.1.smithi044.stdout:7/300: creat d0/d3/d8/d32/d33/f67 x:0 0 0 2024-06-03T01:20:50.334 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:49 smithi032 ceph-mon[29360]: pgmap v254: 65 pgs: 65 active+clean; 400 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 851 op/s 2024-06-03T01:20:50.335 INFO:tasks.workunit.client.1.smithi044.stdout:2/149: mkdir db/d1b/d2a/d2f 0 2024-06-03T01:20:50.336 INFO:tasks.workunit.client.1.smithi044.stdout:9/155: mknod d6/d8/d16/d27/c2e 0 2024-06-03T01:20:50.343 INFO:tasks.workunit.client.1.smithi044.stdout:2/150: creat db/d15/f30 x:0 0 0 2024-06-03T01:20:50.345 INFO:tasks.workunit.client.1.smithi044.stdout:9/156: mkdir d6/dd/d2f 0 2024-06-03T01:20:50.345 INFO:tasks.workunit.client.1.smithi044.stdout:9/157: read - d6/d8/f13 zero size 2024-06-03T01:20:50.347 INFO:tasks.workunit.client.1.smithi044.stdout:0/202: rename d3/d1c to d3/d49 0 2024-06-03T01:20:50.347 INFO:tasks.workunit.client.1.smithi044.stdout:0/203: dread - d3/d9/d36/f48 zero size 2024-06-03T01:20:50.348 INFO:tasks.workunit.client.1.smithi044.stdout:8/204: unlink d5/d16/d1d/f39 0 2024-06-03T01:20:50.352 INFO:tasks.workunit.client.1.smithi044.stdout:7/301: rename d0/d3/d8/f39 to d0/d21/d27/f68 0 2024-06-03T01:20:50.353 INFO:tasks.workunit.client.1.smithi044.stdout:2/151: mknod db/d1b/d2a/d2f/c31 0 2024-06-03T01:20:50.356 INFO:tasks.workunit.client.1.smithi044.stdout:9/158: symlink d6/d8/d16/d27/l30 0 2024-06-03T01:20:50.356 INFO:tasks.workunit.client.1.smithi044.stdout:9/159: fdatasync d6/dd/f14 0 2024-06-03T01:20:50.356 INFO:tasks.workunit.client.1.smithi044.stdout:9/160: write f2 [4600872,26963] 0 2024-06-03T01:20:50.356 INFO:tasks.workunit.client.1.smithi044.stdout:7/302: rmdir d0/d21/d27 39 2024-06-03T01:20:50.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/152: creat db/d1b/d27/f32 x:0 0 0 2024-06-03T01:20:50.359 INFO:tasks.workunit.client.1.smithi044.stdout:9/161: rmdir d6/dd/d2f 0 2024-06-03T01:20:50.361 INFO:tasks.workunit.client.1.smithi044.stdout:9/162: truncate d6/d8/fe 124038 0 2024-06-03T01:20:50.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/205: read d5/f6 [3707823,37580] 0 2024-06-03T01:20:50.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/206: fdatasync d5/f7 0 2024-06-03T01:20:50.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/207: stat d5/f7 0 2024-06-03T01:20:50.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/208: getdents d5 0 2024-06-03T01:20:50.367 INFO:tasks.workunit.client.1.smithi044.stdout:8/209: chown d5/f7 607 1 2024-06-03T01:20:50.367 INFO:tasks.workunit.client.1.smithi044.stdout:8/210: fdatasync d5/d10/f34 0 2024-06-03T01:20:50.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/211: creat d5/d16/f3a x:0 0 0 2024-06-03T01:20:50.371 INFO:tasks.workunit.client.1.smithi044.stdout:8/212: mknod d5/d20/d2b/c3b 0 2024-06-03T01:20:50.374 INFO:tasks.workunit.client.1.smithi044.stdout:8/213: getdents d5/d16/d1d 0 2024-06-03T01:20:50.390 INFO:tasks.workunit.client.1.smithi044.stdout:0/204: dwrite d3/d9/d36/f43 [0,4194304] 0 2024-06-03T01:20:50.390 INFO:tasks.workunit.client.1.smithi044.stdout:9/163: dread d6/d8/fe [0,4194304] 0 2024-06-03T01:20:50.390 INFO:tasks.workunit.client.1.smithi044.stdout:9/164: stat d6/dd/l1a 0 2024-06-03T01:20:50.393 INFO:tasks.workunit.client.1.smithi044.stdout:9/165: getdents d6/dd 0 2024-06-03T01:20:50.394 INFO:tasks.workunit.client.1.smithi044.stdout:9/166: write d6/dd/f17 [981555,92247] 0 2024-06-03T01:20:50.394 INFO:tasks.workunit.client.1.smithi044.stdout:9/167: chown d6/f12 18788 1 2024-06-03T01:20:50.394 INFO:tasks.workunit.client.1.smithi044.stdout:7/303: dwrite d0/d21/f53 [0,4194304] 0 2024-06-03T01:20:50.396 INFO:tasks.workunit.client.1.smithi044.stdout:2/153: dwrite db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:20:50.396 INFO:tasks.workunit.client.1.smithi044.stdout:8/214: dwrite d5/f6 [4194304,4194304] 0 2024-06-03T01:20:50.396 INFO:tasks.workunit.client.1.smithi044.stdout:8/215: creat d5/d16/f3c x:0 0 0 2024-06-03T01:20:50.400 INFO:tasks.workunit.client.1.smithi044.stdout:8/216: write d5/f15 [4041563,59458] 0 2024-06-03T01:20:50.400 INFO:tasks.workunit.client.1.smithi044.stdout:8/217: write d5/d20/d2b/f30 [204121,63077] 0 2024-06-03T01:20:50.400 INFO:tasks.workunit.client.1.smithi044.stdout:8/218: fsync d5/d20/d2b/f35 0 2024-06-03T01:20:50.403 INFO:tasks.workunit.client.1.smithi044.stdout:8/219: creat d5/d20/f3d x:0 0 0 2024-06-03T01:20:50.403 INFO:tasks.workunit.client.1.smithi044.stdout:2/154: rmdir db/d1b/d2a 39 2024-06-03T01:20:50.404 INFO:tasks.workunit.client.1.smithi044.stdout:8/220: mknod d5/d20/d33/c3e 0 2024-06-03T01:20:50.404 INFO:tasks.workunit.client.1.smithi044.stdout:1/190: sync 2024-06-03T01:20:50.405 INFO:tasks.workunit.client.1.smithi044.stdout:1/191: mkdir d0/d3/d20/d40 0 2024-06-03T01:20:50.417 INFO:tasks.workunit.client.1.smithi044.stdout:8/221: write d5/f2d [2056324,47053] 0 2024-06-03T01:20:50.418 INFO:tasks.workunit.client.1.smithi044.stdout:8/222: creat d5/d20/d33/f3f x:0 0 0 2024-06-03T01:20:50.419 INFO:tasks.workunit.client.1.smithi044.stdout:0/205: dwrite d3/d49/d20/d25/f40 [0,4194304] 0 2024-06-03T01:20:50.419 INFO:tasks.workunit.client.1.smithi044.stdout:8/223: getdents d5/d16/d1d/d32 0 2024-06-03T01:20:50.419 INFO:tasks.workunit.client.1.smithi044.stdout:8/224: dread - d5/d20/d33/f3f zero size 2024-06-03T01:20:50.421 INFO:tasks.workunit.client.1.smithi044.stdout:0/206: rename d3/d49/d27/d29/f2d to d3/d49/d21/f4a 0 2024-06-03T01:20:50.421 INFO:tasks.workunit.client.1.smithi044.stdout:8/225: rename d5 to d5/d16/d40 22 2024-06-03T01:20:50.421 INFO:tasks.workunit.client.1.smithi044.stdout:8/226: chown d5/la 746956 1 2024-06-03T01:20:50.421 INFO:tasks.workunit.client.1.smithi044.stdout:8/227: dread - d5/d16/f2e zero size 2024-06-03T01:20:50.422 INFO:tasks.workunit.client.1.smithi044.stdout:0/207: link d3/d9/d36/f43 d3/d49/d20/d25/f4b 0 2024-06-03T01:20:50.451 INFO:tasks.workunit.client.1.smithi044.stdout:8/228: dread d5/f15 [0,4194304] 0 2024-06-03T01:20:50.451 INFO:tasks.workunit.client.1.smithi044.stdout:8/229: dread - d5/d16/f2e zero size 2024-06-03T01:20:50.451 INFO:tasks.workunit.client.1.smithi044.stdout:8/230: read - d5/d20/d2b/f35 zero size 2024-06-03T01:20:50.451 INFO:tasks.workunit.client.1.smithi044.stdout:2/155: dwrite db/d1b/d2a/f2d [0,4194304] 0 2024-06-03T01:20:50.451 INFO:tasks.workunit.client.1.smithi044.stdout:0/208: dwrite d3/d9/f44 [0,4194304] 0 2024-06-03T01:20:50.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/231: symlink d5/d16/l41 0 2024-06-03T01:20:50.455 INFO:tasks.workunit.client.1.smithi044.stdout:1/192: getdents d0/d3/d20 0 2024-06-03T01:20:50.458 INFO:tasks.workunit.client.1.smithi044.stdout:0/209: symlink d3/d49/d21/l4c 0 2024-06-03T01:20:50.458 INFO:tasks.workunit.client.1.smithi044.stdout:0/210: write d3/f19 [575599,43454] 0 2024-06-03T01:20:50.459 INFO:tasks.workunit.client.1.smithi044.stdout:0/211: write d3/d49/d27/f38 [151100,24251] 0 2024-06-03T01:20:50.460 INFO:tasks.workunit.client.1.smithi044.stdout:8/232: creat d5/d16/d1d/f42 x:0 0 0 2024-06-03T01:20:50.462 INFO:tasks.workunit.client.1.smithi044.stdout:2/156: getdents db/d1b/d27 0 2024-06-03T01:20:50.462 INFO:tasks.workunit.client.1.smithi044.stdout:2/157: write db/f26 [3385893,45708] 0 2024-06-03T01:20:50.465 INFO:tasks.workunit.client.1.smithi044.stdout:8/233: truncate f3 2820475 0 2024-06-03T01:20:50.467 INFO:tasks.workunit.client.1.smithi044.stdout:2/158: creat db/d1b/d2a/f33 x:0 0 0 2024-06-03T01:20:50.467 INFO:tasks.workunit.client.1.smithi044.stdout:2/159: dread - db/d15/f30 zero size 2024-06-03T01:20:50.468 INFO:tasks.workunit.client.1.smithi044.stdout:0/212: read d3/d49/d20/d25/f4b [2457401,59519] 0 2024-06-03T01:20:50.469 INFO:tasks.workunit.client.1.smithi044.stdout:8/234: unlink d5/f6 0 2024-06-03T01:20:50.469 INFO:tasks.workunit.client.1.smithi044.stdout:8/235: fdatasync d5/d16/d1d/f1f 0 2024-06-03T01:20:50.469 INFO:tasks.workunit.client.1.smithi044.stdout:2/160: symlink db/dd/l34 0 2024-06-03T01:20:50.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/213: unlink d3/d9/c1b 0 2024-06-03T01:20:50.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/214: creat d3/d9/f4d x:0 0 0 2024-06-03T01:20:50.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/215: dread - d3/f45 zero size 2024-06-03T01:20:50.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/216: fsync d3/f18 0 2024-06-03T01:20:50.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/217: readlink d3/d49/d21/l47 0 2024-06-03T01:20:50.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:49 smithi044 ceph-mon[36956]: pgmap v254: 65 pgs: 65 active+clean; 400 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 851 op/s 2024-06-03T01:20:50.472 INFO:tasks.workunit.client.1.smithi044.stdout:8/236: symlink d5/d20/l43 0 2024-06-03T01:20:50.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/193: dwrite d0/d16/f26 [0,4194304] 0 2024-06-03T01:20:50.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/194: creat d0/d3/d20/d7/d13/f41 x:0 0 0 2024-06-03T01:20:50.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/195: stat d0/d3/d20/d7/f36 0 2024-06-03T01:20:50.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/196: readlink d0/d3/l11 0 2024-06-03T01:20:50.474 INFO:tasks.workunit.client.1.smithi044.stdout:8/237: symlink d5/d16/d1d/d32/l44 0 2024-06-03T01:20:50.476 INFO:tasks.workunit.client.1.smithi044.stdout:0/218: dread d3/d49/d20/d25/f4b [0,4194304] 0 2024-06-03T01:20:50.478 INFO:tasks.workunit.client.1.smithi044.stdout:8/238: creat d5/d16/d1d/d32/f45 x:0 0 0 2024-06-03T01:20:50.479 INFO:tasks.workunit.client.1.smithi044.stdout:0/219: symlink d3/d49/d27/d29/l4e 0 2024-06-03T01:20:50.481 INFO:tasks.workunit.client.1.smithi044.stdout:0/220: fdatasync d3/f19 0 2024-06-03T01:20:50.495 INFO:tasks.workunit.client.1.smithi044.stdout:1/197: dwrite d0/d3/d20/d7/d13/f41 [0,4194304] 0 2024-06-03T01:20:50.495 INFO:tasks.workunit.client.1.smithi044.stdout:1/198: read - d0/d3/d20/d7/d13/f3d zero size 2024-06-03T01:20:50.495 INFO:tasks.workunit.client.1.smithi044.stdout:2/161: dwrite db/f26 [0,4194304] 0 2024-06-03T01:20:50.495 INFO:tasks.workunit.client.1.smithi044.stdout:2/162: readlink db/d1b/l21 0 2024-06-03T01:20:50.496 INFO:tasks.workunit.client.1.smithi044.stdout:4/148: sync 2024-06-03T01:20:50.496 INFO:tasks.workunit.client.1.smithi044.stdout:4/149: chown d17 1 1 2024-06-03T01:20:50.499 INFO:tasks.workunit.client.1.smithi044.stdout:2/163: rename db/c1d to db/d15/c35 0 2024-06-03T01:20:50.499 INFO:tasks.workunit.client.1.smithi044.stdout:2/164: fdatasync db/d1b/d2a/f2b 0 2024-06-03T01:20:50.499 INFO:tasks.workunit.client.1.smithi044.stdout:2/165: write f2 [4424963,43209] 0 2024-06-03T01:20:50.499 INFO:tasks.workunit.client.1.smithi044.stdout:4/150: stat d17/d1c/l25 0 2024-06-03T01:20:50.499 INFO:tasks.workunit.client.1.smithi044.stdout:4/151: chown d17/d23/l1d 17 1 2024-06-03T01:20:50.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/152: mkdir d17/d23/d35 0 2024-06-03T01:20:50.502 INFO:tasks.workunit.client.1.smithi044.stdout:4/153: rename d17/d23/d28/l2f to d17/d23/d28/l36 0 2024-06-03T01:20:50.503 INFO:tasks.workunit.client.1.smithi044.stdout:4/154: symlink d17/d23/d28/d2b/l37 0 2024-06-03T01:20:50.504 INFO:tasks.workunit.client.1.smithi044.stdout:4/155: symlink d17/d23/d28/l38 0 2024-06-03T01:20:50.504 INFO:tasks.workunit.client.1.smithi044.stdout:4/156: dread - d17/d23/d28/d2b/f24 zero size 2024-06-03T01:20:50.504 INFO:tasks.workunit.client.1.smithi044.stdout:4/157: stat d17/d23/d28/f2c 0 2024-06-03T01:20:50.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/199: dwrite d0/d3/d20/d7/f2b [0,4194304] 0 2024-06-03T01:20:50.515 INFO:tasks.workunit.client.1.smithi044.stdout:0/221: dwrite d3/d49/d20/d25/f40 [0,4194304] 0 2024-06-03T01:20:50.515 INFO:tasks.workunit.client.1.smithi044.stdout:1/200: chown d0/d3/d20/d7/lf 15 1 2024-06-03T01:20:50.516 INFO:tasks.workunit.client.1.smithi044.stdout:4/158: dread f5 [0,4194304] 0 2024-06-03T01:20:50.517 INFO:tasks.workunit.client.1.smithi044.stdout:2/166: truncate f2 3636039 0 2024-06-03T01:20:50.519 INFO:tasks.workunit.client.1.smithi044.stdout:0/222: creat d3/d49/d27/d29/f4f x:0 0 0 2024-06-03T01:20:50.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/159: mkdir d17/d23/d28/d39 0 2024-06-03T01:20:50.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/160: write d17/d23/f20 [117723,85265] 0 2024-06-03T01:20:50.520 INFO:tasks.workunit.client.1.smithi044.stdout:1/201: dread d0/d3/f28 [0,4194304] 0 2024-06-03T01:20:50.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/161: write d17/d23/d28/d2b/f22 [223707,7896] 0 2024-06-03T01:20:50.522 INFO:tasks.workunit.client.1.smithi044.stdout:2/167: rename db/d15/f30 to db/d1b/f36 0 2024-06-03T01:20:50.522 INFO:tasks.workunit.client.1.smithi044.stdout:1/202: rmdir d0/d3 39 2024-06-03T01:20:50.525 INFO:tasks.workunit.client.1.smithi044.stdout:0/223: rename d3/f2c to d3/d9/d36/f50 0 2024-06-03T01:20:50.525 INFO:tasks.workunit.client.1.smithi044.stdout:0/224: fdatasync d3/d49/d21/f26 0 2024-06-03T01:20:50.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/168: symlink db/l37 0 2024-06-03T01:20:50.527 INFO:tasks.workunit.client.1.smithi044.stdout:1/203: mkdir d0/d3/d20/d7/d42 0 2024-06-03T01:20:50.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/162: getdents d17 0 2024-06-03T01:20:50.529 INFO:tasks.workunit.client.1.smithi044.stdout:2/169: symlink db/dd/l38 0 2024-06-03T01:20:50.529 INFO:tasks.workunit.client.1.smithi044.stdout:2/170: readlink db/d1b/l23 0 2024-06-03T01:20:50.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/163: creat d17/d1c/f3a x:0 0 0 2024-06-03T01:20:50.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/164: chown d17/f31 1456841 1 2024-06-03T01:20:50.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/165: stat d17/d23/f30 0 2024-06-03T01:20:50.532 INFO:tasks.workunit.client.1.smithi044.stdout:4/166: rmdir d17/d23/d28/d32 0 2024-06-03T01:20:50.533 INFO:tasks.workunit.client.1.smithi044.stdout:0/225: write d3/f4 [6673186,25255] 0 2024-06-03T01:20:50.533 INFO:tasks.workunit.client.1.smithi044.stdout:0/226: write d3/d49/d27/f3f [28767,86020] 0 2024-06-03T01:20:50.535 INFO:tasks.workunit.client.1.smithi044.stdout:0/227: symlink d3/d49/d27/l51 0 2024-06-03T01:20:50.537 INFO:tasks.workunit.client.1.smithi044.stdout:2/171: dread f2 [0,4194304] 0 2024-06-03T01:20:50.537 INFO:tasks.workunit.client.1.smithi044.stdout:0/228: symlink d3/d49/d27/d29/l52 0 2024-06-03T01:20:50.537 INFO:tasks.workunit.client.1.smithi044.stdout:0/229: stat d3/d9 0 2024-06-03T01:20:50.537 INFO:tasks.workunit.client.1.smithi044.stdout:0/230: creat d3/d9/f53 x:0 0 0 2024-06-03T01:20:50.537 INFO:tasks.workunit.client.1.smithi044.stdout:0/231: dread - d3/d49/d27/f34 zero size 2024-06-03T01:20:50.546 INFO:tasks.workunit.client.1.smithi044.stdout:6/221: sync 2024-06-03T01:20:50.548 INFO:tasks.workunit.client.1.smithi044.stdout:6/222: rename d2/d6/d4c to d2/d4f 0 2024-06-03T01:20:50.551 INFO:tasks.workunit.client.1.smithi044.stdout:1/204: dwrite d0/fd [0,4194304] 0 2024-06-03T01:20:50.551 INFO:tasks.workunit.client.1.smithi044.stdout:4/167: dwrite d17/d23/d28/d2b/f34 [0,4194304] 0 2024-06-03T01:20:50.552 INFO:tasks.workunit.client.1.smithi044.stdout:0/232: dwrite d3/d49/f3b [0,4194304] 0 2024-06-03T01:20:50.553 INFO:tasks.workunit.client.1.smithi044.stdout:1/205: truncate d0/d3/d20/d7/f34 1579247 0 2024-06-03T01:20:50.555 INFO:tasks.workunit.client.1.smithi044.stdout:6/223: dread d2/fe [0,4194304] 0 2024-06-03T01:20:50.555 INFO:tasks.workunit.client.1.smithi044.stdout:4/168: creat d17/d23/d28/d39/f3b x:0 0 0 2024-06-03T01:20:50.558 INFO:tasks.workunit.client.1.smithi044.stdout:6/224: mkdir d2/d23/d30/d1e/d38/d50 0 2024-06-03T01:20:50.560 INFO:tasks.workunit.client.1.smithi044.stdout:4/169: creat d17/d23/d35/f3c x:0 0 0 2024-06-03T01:20:50.560 INFO:tasks.workunit.client.1.smithi044.stdout:4/170: stat d17/d23/d28/f2c 0 2024-06-03T01:20:50.562 INFO:tasks.workunit.client.1.smithi044.stdout:6/225: mknod d2/d4f/c51 0 2024-06-03T01:20:50.562 INFO:tasks.workunit.client.1.smithi044.stdout:6/226: dread - d2/d6/db/f3b zero size 2024-06-03T01:20:50.565 INFO:tasks.workunit.client.1.smithi044.stdout:6/227: mknod d2/d23/d30/d1e/d34/c52 0 2024-06-03T01:20:50.565 INFO:tasks.workunit.client.1.smithi044.stdout:6/228: chown d2/d4/l8 2963184 1 2024-06-03T01:20:50.566 INFO:tasks.workunit.client.1.smithi044.stdout:1/206: dread d0/d16/f26 [0,4194304] 0 2024-06-03T01:20:50.566 INFO:tasks.workunit.client.1.smithi044.stdout:4/171: rename f15 to d17/d23/d28/d2b/f3d 0 2024-06-03T01:20:50.568 INFO:tasks.workunit.client.1.smithi044.stdout:1/207: mknod d0/d3/c43 0 2024-06-03T01:20:50.570 INFO:tasks.workunit.client.1.smithi044.stdout:1/208: rename d0/d3/d20/f2e to d0/d3/d20/d7/f44 0 2024-06-03T01:20:50.575 INFO:tasks.workunit.client.1.smithi044.stdout:4/172: dread f2 [0,4194304] 0 2024-06-03T01:20:50.575 INFO:tasks.workunit.client.1.smithi044.stdout:0/233: dwrite d3/d9/f22 [0,4194304] 0 2024-06-03T01:20:50.578 INFO:tasks.workunit.client.1.smithi044.stdout:6/229: read d2/d23/d30/f1f [2190141,74272] 0 2024-06-03T01:20:50.578 INFO:tasks.workunit.client.1.smithi044.stdout:6/230: chown d2 210567 1 2024-06-03T01:20:50.578 INFO:tasks.workunit.client.1.smithi044.stdout:0/234: mknod d3/d9/c54 0 2024-06-03T01:20:50.578 INFO:tasks.workunit.client.1.smithi044.stdout:0/235: readlink d3/d49/d27/l51 0 2024-06-03T01:20:50.579 INFO:tasks.workunit.client.1.smithi044.stdout:6/231: truncate d2/d6/f3e 464494 0 2024-06-03T01:20:50.580 INFO:tasks.workunit.client.1.smithi044.stdout:0/236: unlink d3/d9/c54 0 2024-06-03T01:20:50.580 INFO:tasks.workunit.client.1.smithi044.stdout:0/237: stat d3/d9/f13 0 2024-06-03T01:20:50.582 INFO:tasks.workunit.client.1.smithi044.stdout:0/238: truncate d3/f4 2491510 0 2024-06-03T01:20:50.583 INFO:tasks.workunit.client.1.smithi044.stdout:0/239: chown d3/d49/d20 352771 1 2024-06-03T01:20:50.583 INFO:tasks.workunit.client.1.smithi044.stdout:0/240: stat d3/f14 0 2024-06-03T01:20:50.587 INFO:tasks.workunit.client.1.smithi044.stdout:0/241: dread d3/d49/d21/f26 [0,4194304] 0 2024-06-03T01:20:50.587 INFO:tasks.workunit.client.1.smithi044.stdout:1/209: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-03T01:20:50.595 INFO:tasks.workunit.client.1.smithi044.stdout:6/232: dwrite d2/d23/f18 [0,4194304] 0 2024-06-03T01:20:50.596 INFO:tasks.workunit.client.1.smithi044.stdout:4/173: dwrite d17/f2d [4194304,4194304] 0 2024-06-03T01:20:50.603 INFO:tasks.workunit.client.1.smithi044.stdout:4/174: creat d17/d23/d28/f3e x:0 0 0 2024-06-03T01:20:50.604 INFO:tasks.workunit.client.1.smithi044.stdout:4/175: mkdir d17/d23/d35/d3f 0 2024-06-03T01:20:50.605 INFO:tasks.workunit.client.1.smithi044.stdout:4/176: rmdir d17/d23/d28/d39 39 2024-06-03T01:20:50.606 INFO:tasks.workunit.client.1.smithi044.stdout:4/177: write d17/f31 [286723,60106] 0 2024-06-03T01:20:50.607 INFO:tasks.workunit.client.1.smithi044.stdout:4/178: rename d17/d23/d28/d39/f3b to d17/d23/d28/d2b/f40 0 2024-06-03T01:20:50.609 INFO:tasks.workunit.client.1.smithi044.stdout:4/179: link f2 d17/d23/d35/f41 0 2024-06-03T01:20:50.617 INFO:tasks.workunit.client.1.smithi044.stdout:1/210: dwrite d0/d3/d14/f38 [4194304,4194304] 0 2024-06-03T01:20:50.619 INFO:tasks.workunit.client.1.smithi044.stdout:0/242: dwrite d3/d9/d36/f42 [0,4194304] 0 2024-06-03T01:20:50.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/211: mkdir d0/d3/d20/d40/d45 0 2024-06-03T01:20:50.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/212: chown d0/d3/d20/d40/d45 6 1 2024-06-03T01:20:50.626 INFO:tasks.workunit.client.1.smithi044.stdout:6/233: dwrite d2/d6/db/f12 [0,4194304] 0 2024-06-03T01:20:50.627 INFO:tasks.workunit.client.1.smithi044.stdout:1/213: rmdir d0/d3/d20/d7/d2f 39 2024-06-03T01:20:50.630 INFO:tasks.workunit.client.1.smithi044.stdout:0/243: getdents d3/d9/d36 0 2024-06-03T01:20:50.632 INFO:tasks.workunit.client.1.smithi044.stdout:1/214: rename d0/d3/c37 to d0/d3/d20/d40/d45/c46 0 2024-06-03T01:20:50.632 INFO:tasks.workunit.client.1.smithi044.stdout:1/215: truncate d0/d3/d20/d7/d2f/f33 485068 0 2024-06-03T01:20:50.633 INFO:tasks.workunit.client.1.smithi044.stdout:0/244: truncate d3/f17 680639 0 2024-06-03T01:20:50.633 INFO:tasks.workunit.client.1.smithi044.stdout:0/245: chown d3/d49/d27/f3f 0 1 2024-06-03T01:20:50.635 INFO:tasks.workunit.client.1.smithi044.stdout:1/216: dread d0/d3/f28 [0,4194304] 0 2024-06-03T01:20:50.635 INFO:tasks.workunit.client.1.smithi044.stdout:1/217: fdatasync d0/d16/f3a 0 2024-06-03T01:20:50.636 INFO:tasks.workunit.client.1.smithi044.stdout:6/234: write d2/d23/d30/f2a [4064268,55060] 0 2024-06-03T01:20:50.636 INFO:tasks.workunit.client.1.smithi044.stdout:1/218: mkdir d0/d3/d20/d7/d42/d47 0 2024-06-03T01:20:50.637 INFO:tasks.workunit.client.1.smithi044.stdout:4/180: dwrite d17/d23/d28/d2b/f1b [0,4194304] 0 2024-06-03T01:20:50.637 INFO:tasks.workunit.client.1.smithi044.stdout:4/181: fdatasync d17/d23/d28/d2b/f2e 0 2024-06-03T01:20:50.637 INFO:tasks.workunit.client.1.smithi044.stdout:6/235: mkdir d2/d23/d30/d1e/d38/d50/d53 0 2024-06-03T01:20:50.637 INFO:tasks.workunit.client.1.smithi044.stdout:6/236: dread - d2/d23/d30/f1a zero size 2024-06-03T01:20:50.638 INFO:tasks.workunit.client.1.smithi044.stdout:1/219: symlink d0/d3/d20/l48 0 2024-06-03T01:20:50.638 INFO:tasks.workunit.client.1.smithi044.stdout:6/237: dread d2/d6/f13 [0,4194304] 0 2024-06-03T01:20:50.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/182: mknod d17/c42 0 2024-06-03T01:20:50.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/183: chown f2 4 1 2024-06-03T01:20:50.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/184: read f2 [2499263,80192] 0 2024-06-03T01:20:50.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/185: dread - d17/d23/d28/d2b/f24 zero size 2024-06-03T01:20:50.640 INFO:tasks.workunit.client.1.smithi044.stdout:1/220: mknod d0/d3/c49 0 2024-06-03T01:20:50.640 INFO:tasks.workunit.client.1.smithi044.stdout:1/221: dread d0/d3/d20/d7/d2f/f33 [0,4194304] 0 2024-06-03T01:20:50.641 INFO:tasks.workunit.client.1.smithi044.stdout:6/238: creat d2/d23/d44/f54 x:0 0 0 2024-06-03T01:20:50.643 INFO:tasks.workunit.client.1.smithi044.stdout:0/246: truncate d3/f17 1039369 0 2024-06-03T01:20:50.645 INFO:tasks.workunit.client.1.smithi044.stdout:4/186: rmdir d17/d23/d35/d3f 0 2024-06-03T01:20:50.645 INFO:tasks.workunit.client.1.smithi044.stdout:4/187: truncate d17/d23/d28/d2b/f1b 4669981 0 2024-06-03T01:20:50.646 INFO:tasks.workunit.client.1.smithi044.stdout:0/247: mkdir d3/d9/d55 0 2024-06-03T01:20:50.646 INFO:tasks.workunit.client.1.smithi044.stdout:0/248: write d3/d9/d36/f48 [674152,67577] 0 2024-06-03T01:20:50.647 INFO:tasks.workunit.client.1.smithi044.stdout:4/188: creat d17/d23/d28/d39/f43 x:0 0 0 2024-06-03T01:20:50.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/249: unlink d3/d49/d20/d25/f30 0 2024-06-03T01:20:50.652 INFO:tasks.workunit.client.1.smithi044.stdout:4/189: rmdir d17/d23/d28/d2b 39 2024-06-03T01:20:50.661 INFO:tasks.workunit.client.1.smithi044.stdout:1/222: dwrite d0/d3/d20/d7/f36 [0,4194304] 0 2024-06-03T01:20:50.673 INFO:tasks.workunit.client.1.smithi044.stdout:6/239: dwrite d2/d6/db/f12 [0,4194304] 0 2024-06-03T01:20:50.673 INFO:tasks.workunit.client.1.smithi044.stdout:6/240: read - d2/d23/d30/d1e/d34/d39/f3c zero size 2024-06-03T01:20:50.674 INFO:tasks.workunit.client.1.smithi044.stdout:0/250: dwrite d3/d49/f3e [0,4194304] 0 2024-06-03T01:20:50.674 INFO:tasks.workunit.client.1.smithi044.stdout:0/251: write d3/f4 [2813316,58428] 0 2024-06-03T01:20:50.674 INFO:tasks.workunit.client.1.smithi044.stdout:0/252: chown d3/d9/f28 36593 1 2024-06-03T01:20:50.674 INFO:tasks.workunit.client.1.smithi044.stdout:4/190: dwrite d17/d23/d28/d2b/f22 [0,4194304] 0 2024-06-03T01:20:50.674 INFO:tasks.workunit.client.1.smithi044.stdout:4/191: chown d17/d23/d28/d39 12 1 2024-06-03T01:20:50.679 INFO:tasks.workunit.client.1.smithi044.stdout:0/253: mkdir d3/d49/d56 0 2024-06-03T01:20:50.679 INFO:tasks.workunit.client.1.smithi044.stdout:0/254: truncate d3/d9/f53 969796 0 2024-06-03T01:20:50.681 INFO:tasks.workunit.client.1.smithi044.stdout:6/241: rmdir d2/d23 39 2024-06-03T01:20:50.681 INFO:tasks.workunit.client.1.smithi044.stdout:6/242: write d2/f27 [134087,46663] 0 2024-06-03T01:20:50.682 INFO:tasks.workunit.client.1.smithi044.stdout:0/255: creat d3/d9/d36/f57 x:0 0 0 2024-06-03T01:20:50.682 INFO:tasks.workunit.client.1.smithi044.stdout:6/243: truncate d2/d23/d30/f19 749508 0 2024-06-03T01:20:50.698 INFO:tasks.workunit.client.1.smithi044.stdout:5/178: sync 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:8/239: sync 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:9/168: sync 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:5/179: stat d2/dc/f16 0 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/120: sync 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/121: chown lc 59837 1 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:9/169: getdents d6/dd 0 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/122: chown fd 113846 1 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/123: readlink lb 0 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:9/170: write d6/f12 [298196,77942] 0 2024-06-03T01:20:50.699 INFO:tasks.workunit.client.1.smithi044.stdout:7/304: sync 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:7/305: truncate d0/d16/d23/f24 4441953 0 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:2/172: sync 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:2/173: write db/f1f [1986483,107467] 0 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:8/240: fdatasync d5/d20/d33/f3f 0 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:9/171: mknod d6/d8/d16/d27/c31 0 2024-06-03T01:20:50.700 INFO:tasks.workunit.client.1.smithi044.stdout:9/172: creat d6/d8/db/f32 x:0 0 0 2024-06-03T01:20:50.701 INFO:tasks.workunit.client.1.smithi044.stdout:9/173: truncate d6/f12 622672 0 2024-06-03T01:20:50.701 INFO:tasks.workunit.client.1.smithi044.stdout:8/241: read d5/d10/f34 [105608,50146] 0 2024-06-03T01:20:50.702 INFO:tasks.workunit.client.1.smithi044.stdout:8/242: creat d5/d10/f46 x:0 0 0 2024-06-03T01:20:50.703 INFO:tasks.workunit.client.1.smithi044.stdout:9/174: rmdir d6 39 2024-06-03T01:20:50.706 INFO:tasks.workunit.client.1.smithi044.stdout:7/306: rename d0/d21/d27/c34 to d0/d3/c69 0 2024-06-03T01:20:50.706 INFO:tasks.workunit.client.1.smithi044.stdout:7/307: chown d0/d16/d23/c3a 1 1 2024-06-03T01:20:50.710 INFO:tasks.workunit.client.1.smithi044.stdout:7/308: mknod d0/d3/d30/c6a 0 2024-06-03T01:20:50.712 INFO:tasks.workunit.client.1.smithi044.stdout:7/309: unlink d0/d3/c1e 0 2024-06-03T01:20:50.713 INFO:tasks.workunit.client.1.smithi044.stdout:2/174: dread db/f12 [0,4194304] 0 2024-06-03T01:20:50.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/310: creat d0/d21/d63/f6b x:0 0 0 2024-06-03T01:20:50.716 INFO:tasks.workunit.client.1.smithi044.stdout:3/124: dread df/f1c [0,4194304] 0 2024-06-03T01:20:50.716 INFO:tasks.workunit.client.1.smithi044.stdout:3/125: getdents df/d13/d1b 0 2024-06-03T01:20:50.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/311: creat d0/d3/d8/d32/f6c x:0 0 0 2024-06-03T01:20:50.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/312: write d0/d16/f54 [875178,56569] 0 2024-06-03T01:20:50.718 INFO:tasks.workunit.client.1.smithi044.stdout:5/180: dwrite d2/d11/d12/f37 [0,4194304] 0 2024-06-03T01:20:50.718 INFO:tasks.workunit.client.1.smithi044.stdout:4/192: dwrite d17/d23/d28/d2b/f24 [0,4194304] 0 2024-06-03T01:20:50.718 INFO:tasks.workunit.client.1.smithi044.stdout:3/126: creat df/d13/d1b/f20 x:0 0 0 2024-06-03T01:20:50.721 INFO:tasks.workunit.client.1.smithi044.stdout:7/313: link d0/d3/d8/d32/f48 d0/d3/d8/f6d 0 2024-06-03T01:20:50.721 INFO:tasks.workunit.client.1.smithi044.stdout:4/193: dread d17/d23/d28/d2b/f3d [0,4194304] 0 2024-06-03T01:20:50.725 INFO:tasks.workunit.client.1.smithi044.stdout:0/256: truncate d3/d9/f53 247379 0 2024-06-03T01:20:50.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/181: mkdir d2/d11/d1e/d3b 0 2024-06-03T01:20:50.728 INFO:tasks.workunit.client.1.smithi044.stdout:2/175: dread db/f1f [0,4194304] 0 2024-06-03T01:20:50.729 INFO:tasks.workunit.client.1.smithi044.stdout:5/182: link d2/f13 d2/f3c 0 2024-06-03T01:20:50.730 INFO:tasks.workunit.client.1.smithi044.stdout:5/183: read - d2/d11/d12/f39 zero size 2024-06-03T01:20:50.730 INFO:tasks.workunit.client.1.smithi044.stdout:3/127: write fd [2824727,100306] 0 2024-06-03T01:20:50.730 INFO:tasks.workunit.client.1.smithi044.stdout:9/175: dwrite d6/dd/f21 [0,4194304] 0 2024-06-03T01:20:50.730 INFO:tasks.workunit.client.1.smithi044.stdout:3/128: creat df/f21 x:0 0 0 2024-06-03T01:20:50.731 INFO:tasks.workunit.client.1.smithi044.stdout:9/176: write d6/f12 [983946,96016] 0 2024-06-03T01:20:50.732 INFO:tasks.workunit.client.1.smithi044.stdout:4/194: dread ff [0,4194304] 0 2024-06-03T01:20:50.732 INFO:tasks.workunit.client.1.smithi044.stdout:7/314: mkdir d0/d21/d27/d6e 0 2024-06-03T01:20:50.732 INFO:tasks.workunit.client.1.smithi044.stdout:7/315: write d0/fd [3075952,82479] 0 2024-06-03T01:20:50.735 INFO:tasks.workunit.client.1.smithi044.stdout:0/257: dread d3/d9/d36/f43 [0,4194304] 0 2024-06-03T01:20:50.735 INFO:tasks.workunit.client.1.smithi044.stdout:0/258: read d3/f4 [873622,6520] 0 2024-06-03T01:20:50.738 INFO:tasks.workunit.client.1.smithi044.stdout:8/243: dwrite d5/f11 [0,4194304] 0 2024-06-03T01:20:50.743 INFO:tasks.workunit.client.1.smithi044.stdout:6/244: dwrite d2/d23/d30/f19 [0,4194304] 0 2024-06-03T01:20:50.743 INFO:tasks.workunit.client.1.smithi044.stdout:9/177: mkdir d6/d8/d33 0 2024-06-03T01:20:50.744 INFO:tasks.workunit.client.1.smithi044.stdout:7/316: write d0/f1d [783879,111018] 0 2024-06-03T01:20:50.744 INFO:tasks.workunit.client.1.smithi044.stdout:7/317: chown d0/d3/d8/l2b 2 1 2024-06-03T01:20:50.744 INFO:tasks.workunit.client.1.smithi044.stdout:7/318: chown d0/d21/f3b 781781 1 2024-06-03T01:20:50.745 INFO:tasks.workunit.client.1.smithi044.stdout:3/129: dread df/f17 [0,4194304] 0 2024-06-03T01:20:50.745 INFO:tasks.workunit.client.1.smithi044.stdout:4/195: mknod d17/d23/c44 0 2024-06-03T01:20:50.745 INFO:tasks.workunit.client.1.smithi044.stdout:3/130: chown df/f11 20 1 2024-06-03T01:20:50.745 INFO:tasks.workunit.client.1.smithi044.stdout:8/244: write d5/d20/d2b/f30 [172543,90450] 0 2024-06-03T01:20:50.745 INFO:tasks.workunit.client.1.smithi044.stdout:8/245: write d5/d20/f3d [959091,49973] 0 2024-06-03T01:20:50.746 INFO:tasks.workunit.client.1.smithi044.stdout:0/259: mkdir d3/d49/d27/d29/d58 0 2024-06-03T01:20:50.747 INFO:tasks.workunit.client.1.smithi044.stdout:9/178: dread f4 [4194304,4194304] 0 2024-06-03T01:20:50.748 INFO:tasks.workunit.client.1.smithi044.stdout:9/179: write d6/d8/db/f22 [987116,101214] 0 2024-06-03T01:20:50.748 INFO:tasks.workunit.client.1.smithi044.stdout:9/180: truncate f2 5464291 0 2024-06-03T01:20:50.748 INFO:tasks.workunit.client.1.smithi044.stdout:3/131: read df/f1c [1886000,100792] 0 2024-06-03T01:20:50.749 INFO:tasks.workunit.client.1.smithi044.stdout:6/245: mkdir d2/d23/d55 0 2024-06-03T01:20:50.751 INFO:tasks.workunit.client.1.smithi044.stdout:4/196: mknod d17/d23/c45 0 2024-06-03T01:20:50.751 INFO:tasks.workunit.client.1.smithi044.stdout:4/197: rename d17/d23/d28/d39 to d17/d23/d28/d39/d46 22 2024-06-03T01:20:50.751 INFO:tasks.workunit.client.1.smithi044.stdout:4/198: rename d17/d23 to d17/d23/d28/d2b/d47 22 2024-06-03T01:20:50.752 INFO:tasks.workunit.client.1.smithi044.stdout:7/319: dread d0/d3/d8/f14 [0,4194304] 0 2024-06-03T01:20:50.752 INFO:tasks.workunit.client.1.smithi044.stdout:2/176: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:50.752 INFO:tasks.workunit.client.1.smithi044.stdout:7/320: chown d0/d3/c69 1323276 1 2024-06-03T01:20:50.752 INFO:tasks.workunit.client.1.smithi044.stdout:0/260: rmdir d3/d49 39 2024-06-03T01:20:50.754 INFO:tasks.workunit.client.1.smithi044.stdout:3/132: creat df/d1e/f22 x:0 0 0 2024-06-03T01:20:50.754 INFO:tasks.workunit.client.1.smithi044.stdout:9/181: creat d6/d8/d16/f34 x:0 0 0 2024-06-03T01:20:50.754 INFO:tasks.workunit.client.1.smithi044.stdout:3/133: creat df/f23 x:0 0 0 2024-06-03T01:20:50.757 INFO:tasks.workunit.client.1.smithi044.stdout:6/246: creat d2/d23/d30/d1e/d34/d39/f56 x:0 0 0 2024-06-03T01:20:50.757 INFO:tasks.workunit.client.1.smithi044.stdout:6/247: chown d2/d4 61 1 2024-06-03T01:20:50.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/246: unlink d5/d16/d1d/d32/f45 0 2024-06-03T01:20:50.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/247: dread - d5/d16/d1d/f42 zero size 2024-06-03T01:20:50.762 INFO:tasks.workunit.client.1.smithi044.stdout:8/248: write f3 [2518772,39614] 0 2024-06-03T01:20:50.764 INFO:tasks.workunit.client.1.smithi044.stdout:2/177: creat db/d1b/d27/f39 x:0 0 0 2024-06-03T01:20:50.767 INFO:tasks.workunit.client.1.smithi044.stdout:7/321: symlink d0/d3/l6f 0 2024-06-03T01:20:50.768 INFO:tasks.workunit.client.1.smithi044.stdout:9/182: symlink d6/d8/l35 0 2024-06-03T01:20:50.772 INFO:tasks.workunit.client.1.smithi044.stdout:6/248: creat d2/d23/d30/d1e/d38/d50/d53/f57 x:0 0 0 2024-06-03T01:20:50.772 INFO:tasks.workunit.client.1.smithi044.stdout:6/249: chown d2/d6/db/f3b 0 1 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:0/261: dwrite d3/f8 [0,4194304] 0 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/134: dwrite df/f19 [4194304,4194304] 0 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/135: write df/f23 [620664,24973] 0 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/136: chown df/d1e 58040 1 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/137: creat df/d13/f24 x:0 0 0 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/138: creat df/d13/f25 x:0 0 0 2024-06-03T01:20:50.778 INFO:tasks.workunit.client.1.smithi044.stdout:4/199: getdents d17/d23 0 2024-06-03T01:20:50.784 INFO:tasks.workunit.client.1.smithi044.stdout:9/183: creat d6/dd/f36 x:0 0 0 2024-06-03T01:20:50.785 INFO:tasks.workunit.client.1.smithi044.stdout:9/184: creat d6/d8/d16/f37 x:0 0 0 2024-06-03T01:20:50.786 INFO:tasks.workunit.client.1.smithi044.stdout:6/250: mknod d2/d23/d44/d4d/c58 0 2024-06-03T01:20:50.787 INFO:tasks.workunit.client.1.smithi044.stdout:0/262: mknod d3/d49/c59 0 2024-06-03T01:20:50.787 INFO:tasks.workunit.client.1.smithi044.stdout:0/263: dread - d3/d9/d36/f50 zero size 2024-06-03T01:20:50.788 INFO:tasks.workunit.client.1.smithi044.stdout:4/200: dread d17/d23/d28/d2b/f1b [0,4194304] 0 2024-06-03T01:20:50.792 INFO:tasks.workunit.client.1.smithi044.stdout:6/251: dread d2/d23/d30/f1c [0,4194304] 0 2024-06-03T01:20:50.792 INFO:tasks.workunit.client.1.smithi044.stdout:8/249: dwrite d5/d16/f3c [0,4194304] 0 2024-06-03T01:20:50.793 INFO:tasks.workunit.client.1.smithi044.stdout:3/139: link df/f17 df/d13/f26 0 2024-06-03T01:20:50.793 INFO:tasks.workunit.client.1.smithi044.stdout:2/178: getdents db/d1b 0 2024-06-03T01:20:50.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/140: read - df/d13/f1d zero size 2024-06-03T01:20:50.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/141: stat df/f11 0 2024-06-03T01:20:50.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/142: write df/f1c [822771,63230] 0 2024-06-03T01:20:50.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/143: truncate df/d13/d1b/f20 614099 0 2024-06-03T01:20:50.795 INFO:tasks.workunit.client.1.smithi044.stdout:9/185: mkdir d6/d38 0 2024-06-03T01:20:50.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/201: rmdir d17/d23 39 2024-06-03T01:20:50.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/202: write d17/d23/d35/f41 [291333,105583] 0 2024-06-03T01:20:50.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/203: creat d17/d23/d28/d39/f48 x:0 0 0 2024-06-03T01:20:50.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/264: dread d3/d49/f24 [0,4194304] 0 2024-06-03T01:20:50.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/265: fdatasync d3/d9/d36/f43 0 2024-06-03T01:20:50.800 INFO:tasks.workunit.client.1.smithi044.stdout:6/252: creat d2/d23/d30/d1e/d38/d50/f59 x:0 0 0 2024-06-03T01:20:50.801 INFO:tasks.workunit.client.1.smithi044.stdout:2/179: truncate db/f19 3499425 0 2024-06-03T01:20:50.803 INFO:tasks.workunit.client.1.smithi044.stdout:3/144: write f9 [7455451,33119] 0 2024-06-03T01:20:50.806 INFO:tasks.workunit.client.1.smithi044.stdout:6/253: mkdir d2/d23/d30/d1e/d38/d50/d53/d5a 0 2024-06-03T01:20:50.806 INFO:tasks.workunit.client.1.smithi044.stdout:6/254: write d2/f2c [4675660,101937] 0 2024-06-03T01:20:50.806 INFO:tasks.workunit.client.1.smithi044.stdout:6/255: dread - d2/d23/d30/f45 zero size 2024-06-03T01:20:50.806 INFO:tasks.workunit.client.1.smithi044.stdout:6/256: chown d2/d4/l40 97 1 2024-06-03T01:20:50.806 INFO:tasks.workunit.client.1.smithi044.stdout:6/257: write d2/d23/d30/d1e/f49 [787323,35294] 0 2024-06-03T01:20:50.808 INFO:tasks.workunit.client.1.smithi044.stdout:2/180: symlink db/d15/l3a 0 2024-06-03T01:20:50.812 INFO:tasks.workunit.client.1.smithi044.stdout:0/266: rename d3/d49/d27 to d3/d9/d55/d5a 0 2024-06-03T01:20:50.815 INFO:tasks.workunit.client.1.smithi044.stdout:2/181: symlink db/d1b/d27/l3b 0 2024-06-03T01:20:50.817 INFO:tasks.workunit.client.1.smithi044.stdout:0/267: creat d3/d49/d20/f5b x:0 0 0 2024-06-03T01:20:50.818 INFO:tasks.workunit.client.1.smithi044.stdout:2/182: mknod db/c3c 0 2024-06-03T01:20:50.820 INFO:tasks.workunit.client.1.smithi044.stdout:0/268: rmdir d3/d49/d21 39 2024-06-03T01:20:50.822 INFO:tasks.workunit.client.1.smithi044.stdout:2/183: rename db/d1b/l1c to db/dd/d28/l3d 0 2024-06-03T01:20:50.822 INFO:tasks.workunit.client.1.smithi044.stdout:2/184: creat db/f3e x:0 0 0 2024-06-03T01:20:50.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/258: dwrite d2/d23/d30/d1e/d34/d39/f56 [0,4194304] 0 2024-06-03T01:20:50.827 INFO:tasks.workunit.client.1.smithi044.stdout:9/186: dwrite d6/dd/f36 [0,4194304] 0 2024-06-03T01:20:50.828 INFO:tasks.workunit.client.1.smithi044.stdout:4/204: dwrite d17/d23/f30 [0,4194304] 0 2024-06-03T01:20:50.832 INFO:tasks.workunit.client.1.smithi044.stdout:9/187: dread f4 [4194304,4194304] 0 2024-06-03T01:20:50.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/185: mkdir db/d1b/d3f 0 2024-06-03T01:20:50.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/186: chown db/dd/f11 9 1 2024-06-03T01:20:50.835 INFO:tasks.workunit.client.1.smithi044.stdout:2/187: stat db/dd/c29 0 2024-06-03T01:20:50.840 INFO:tasks.workunit.client.1.smithi044.stdout:6/259: dwrite d2/d23/f33 [0,4194304] 0 2024-06-03T01:20:50.844 INFO:tasks.workunit.client.1.smithi044.stdout:9/188: creat d6/d8/d16/d27/f39 x:0 0 0 2024-06-03T01:20:50.844 INFO:tasks.workunit.client.1.smithi044.stdout:9/189: chown d6/d8/db/f15 1626915 1 2024-06-03T01:20:50.845 INFO:tasks.workunit.client.1.smithi044.stdout:2/188: symlink db/d15/l40 0 2024-06-03T01:20:50.847 INFO:tasks.workunit.client.1.smithi044.stdout:6/260: symlink d2/d6/l5b 0 2024-06-03T01:20:50.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/205: truncate f2 767057 0 2024-06-03T01:20:50.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/206: dread - d17/d23/d28/d2b/f40 zero size 2024-06-03T01:20:50.850 INFO:tasks.workunit.client.1.smithi044.stdout:9/190: mknod d6/d8/d16/c3a 0 2024-06-03T01:20:50.851 INFO:tasks.workunit.client.1.smithi044.stdout:2/189: rename fa to db/d1b/d3f/f41 0 2024-06-03T01:20:50.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/191: mkdir d6/d8/d3b 0 2024-06-03T01:20:50.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/192: stat d6/d8/l35 0 2024-06-03T01:20:50.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/193: fsync d6/d8/db/f15 0 2024-06-03T01:20:50.854 INFO:tasks.workunit.client.1.smithi044.stdout:9/194: mknod d6/d8/c3c 0 2024-06-03T01:20:50.854 INFO:tasks.workunit.client.1.smithi044.stdout:9/195: read - d6/d8/d16/f37 zero size 2024-06-03T01:20:50.854 INFO:tasks.workunit.client.1.smithi044.stdout:5/184: sync 2024-06-03T01:20:50.854 INFO:tasks.workunit.client.1.smithi044.stdout:1/223: sync 2024-06-03T01:20:50.855 INFO:tasks.workunit.client.1.smithi044.stdout:5/185: symlink d2/d11/d1e/d2f/l3d 0 2024-06-03T01:20:50.857 INFO:tasks.workunit.client.1.smithi044.stdout:5/186: mkdir d2/d11/d3e 0 2024-06-03T01:20:50.858 INFO:tasks.workunit.client.1.smithi044.stdout:4/207: dread d17/d23/d28/d2b/f24 [0,4194304] 0 2024-06-03T01:20:50.858 INFO:tasks.workunit.client.1.smithi044.stdout:6/261: write d2/f2c [3062894,12952] 0 2024-06-03T01:20:50.858 INFO:tasks.workunit.client.1.smithi044.stdout:6/262: stat d2/d23 0 2024-06-03T01:20:50.859 INFO:tasks.workunit.client.1.smithi044.stdout:5/187: link d2/dc/le d2/d11/d1e/d21/l3f 0 2024-06-03T01:20:50.860 INFO:tasks.workunit.client.1.smithi044.stdout:2/190: dread db/d1b/d2a/f2d [0,4194304] 0 2024-06-03T01:20:50.862 INFO:tasks.workunit.client.1.smithi044.stdout:4/208: unlink d17/c42 0 2024-06-03T01:20:50.866 INFO:tasks.workunit.client.1.smithi044.stdout:5/188: rename d2/dc/f10 to d2/d11/d3e/f40 0 2024-06-03T01:20:50.869 INFO:tasks.workunit.client.1.smithi044.stdout:2/191: rename db/cf to db/d1b/d2a/c42 0 2024-06-03T01:20:50.869 INFO:tasks.workunit.client.1.smithi044.stdout:9/196: write d6/dd/f36 [16775,69628] 0 2024-06-03T01:20:50.870 INFO:tasks.workunit.client.1.smithi044.stdout:2/192: creat db/d1b/d2a/f43 x:0 0 0 2024-06-03T01:20:50.870 INFO:tasks.workunit.client.1.smithi044.stdout:9/197: write d6/dd/f36 [845635,38963] 0 2024-06-03T01:20:50.871 INFO:tasks.workunit.client.1.smithi044.stdout:5/189: mknod d2/dc/d18/c41 0 2024-06-03T01:20:50.871 INFO:tasks.workunit.client.1.smithi044.stdout:5/190: truncate d2/d11/d12/f25 1800102 0 2024-06-03T01:20:50.871 INFO:tasks.workunit.client.1.smithi044.stdout:5/191: readlink d2/d11/d1e/d2f/l3d 0 2024-06-03T01:20:50.871 INFO:tasks.workunit.client.1.smithi044.stdout:5/192: truncate d2/f2e 599545 0 2024-06-03T01:20:50.874 INFO:tasks.workunit.client.1.smithi044.stdout:1/224: dwrite d0/f29 [0,4194304] 0 2024-06-03T01:20:50.874 INFO:tasks.workunit.client.1.smithi044.stdout:1/225: dread - d0/f3f zero size 2024-06-03T01:20:50.874 INFO:tasks.workunit.client.1.smithi044.stdout:4/209: rename d17/d23/d28/d2b/f22 to d17/d1c/f49 0 2024-06-03T01:20:50.882 INFO:tasks.workunit.client.1.smithi044.stdout:6/263: dwrite d2/d23/d30/d1e/d38/d50/d53/f57 [0,4194304] 0 2024-06-03T01:20:50.887 INFO:tasks.workunit.client.1.smithi044.stdout:1/226: rename d0/fd to d0/d3/d20/d7/d13/f4a 0 2024-06-03T01:20:50.887 INFO:tasks.workunit.client.1.smithi044.stdout:1/227: chown d0/d3/c49 54987381 1 2024-06-03T01:20:50.889 INFO:tasks.workunit.client.1.smithi044.stdout:4/210: creat d17/d23/d28/d2b/f4a x:0 0 0 2024-06-03T01:20:50.893 INFO:tasks.workunit.client.1.smithi044.stdout:9/198: dwrite d6/dd/f2c [0,4194304] 0 2024-06-03T01:20:50.894 INFO:tasks.workunit.client.1.smithi044.stdout:1/228: mkdir d0/d3/d20/d40/d45/d4b 0 2024-06-03T01:20:50.895 INFO:tasks.workunit.client.1.smithi044.stdout:4/211: stat l9 0 2024-06-03T01:20:50.895 INFO:tasks.workunit.client.1.smithi044.stdout:2/193: dwrite db/d1b/d27/f32 [4194304,4194304] 0 2024-06-03T01:20:50.895 INFO:tasks.workunit.client.1.smithi044.stdout:4/212: creat d17/d23/d28/d39/f4b x:0 0 0 2024-06-03T01:20:50.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/199: link f2 d6/dd/f3d 0 2024-06-03T01:20:50.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/200: chown d6/l29 1016 1 2024-06-03T01:20:50.898 INFO:tasks.workunit.client.1.smithi044.stdout:9/201: chown d6/f11 19 1 2024-06-03T01:20:50.899 INFO:tasks.workunit.client.1.smithi044.stdout:1/229: link d0/d3/d20/d7/d13/f41 d0/d3/d20/d7/d42/d47/f4c 0 2024-06-03T01:20:50.903 INFO:tasks.workunit.client.1.smithi044.stdout:2/194: write db/f12 [2474144,108117] 0 2024-06-03T01:20:50.903 INFO:tasks.workunit.client.1.smithi044.stdout:4/213: write fc [4607628,27624] 0 2024-06-03T01:20:50.907 INFO:tasks.workunit.client.1.smithi044.stdout:1/230: rename d0/d3/d20/d7/d13/f41 to d0/d3/d20/d7/d13/d2d/f4d 0 2024-06-03T01:20:50.907 INFO:tasks.workunit.client.1.smithi044.stdout:1/231: stat d0/d3/d20/d40 0 2024-06-03T01:20:50.907 INFO:tasks.workunit.client.1.smithi044.stdout:5/193: dwrite f1 [4194304,4194304] 0 2024-06-03T01:20:50.909 INFO:tasks.workunit.client.1.smithi044.stdout:1/232: mkdir d0/d3/d20/d7/d4e 0 2024-06-03T01:20:50.913 INFO:tasks.workunit.client.1.smithi044.stdout:5/194: getdents d2/d11/d12/d2b 0 2024-06-03T01:20:50.914 INFO:tasks.workunit.client.1.smithi044.stdout:4/214: link l10 d17/d23/l4c 0 2024-06-03T01:20:50.915 INFO:tasks.workunit.client.1.smithi044.stdout:6/264: dwrite d2/f31 [0,4194304] 0 2024-06-03T01:20:50.915 INFO:tasks.workunit.client.1.smithi044.stdout:6/265: dread - d2/d23/d30/f45 zero size 2024-06-03T01:20:50.915 INFO:tasks.workunit.client.1.smithi044.stdout:6/266: fsync d2/d23/f18 0 2024-06-03T01:20:50.916 INFO:tasks.workunit.client.1.smithi044.stdout:1/233: mkdir d0/d3/d20/d7/d13/d4f 0 2024-06-03T01:20:50.916 INFO:tasks.workunit.client.1.smithi044.stdout:5/195: creat d2/d11/d1e/d2f/f42 x:0 0 0 2024-06-03T01:20:50.919 INFO:tasks.workunit.client.1.smithi044.stdout:6/267: rename d2/d4f to d2/d6/db/d5c 0 2024-06-03T01:20:50.921 INFO:tasks.workunit.client.1.smithi044.stdout:1/234: creat d0/d3/d14/f50 x:0 0 0 2024-06-03T01:20:50.922 INFO:tasks.workunit.client.1.smithi044.stdout:1/235: write d0/d3/d20/d7/f17 [719033,70810] 0 2024-06-03T01:20:50.924 INFO:tasks.workunit.client.1.smithi044.stdout:5/196: link d2/lf d2/d11/d1e/d21/d24/l43 0 2024-06-03T01:20:50.924 INFO:tasks.workunit.client.1.smithi044.stdout:5/197: chown d2/d11/d1e/d2f/l3d 31506 1 2024-06-03T01:20:50.924 INFO:tasks.workunit.client.1.smithi044.stdout:5/198: dread - d2/d11/d12/f39 zero size 2024-06-03T01:20:50.927 INFO:tasks.workunit.client.1.smithi044.stdout:8/250: sync 2024-06-03T01:20:50.927 INFO:tasks.workunit.client.1.smithi044.stdout:7/322: sync 2024-06-03T01:20:50.930 INFO:tasks.workunit.client.1.smithi044.stdout:5/199: rename d2/dc/d18 to d2/d11/d12/d2b/d44 0 2024-06-03T01:20:50.930 INFO:tasks.workunit.client.1.smithi044.stdout:7/323: mknod d0/d16/d23/c70 0 2024-06-03T01:20:50.931 INFO:tasks.workunit.client.1.smithi044.stdout:5/200: mkdir d2/d11/d1e/d45 0 2024-06-03T01:20:50.931 INFO:tasks.workunit.client.1.smithi044.stdout:5/201: chown d2/d11/d1e/d21 3711202 1 2024-06-03T01:20:50.931 INFO:tasks.workunit.client.1.smithi044.stdout:1/236: dread d0/d16/f26 [0,4194304] 0 2024-06-03T01:20:50.933 INFO:tasks.workunit.client.1.smithi044.stdout:5/202: symlink d2/d11/d12/d2b/d44/l46 0 2024-06-03T01:20:50.933 INFO:tasks.workunit.client.1.smithi044.stdout:5/203: truncate d2/dc/f15 1600159 0 2024-06-03T01:20:50.934 INFO:tasks.workunit.client.1.smithi044.stdout:1/237: symlink d0/d3/d20/d7/d42/d47/l51 0 2024-06-03T01:20:50.935 INFO:tasks.workunit.client.1.smithi044.stdout:1/238: dread d0/f32 [0,4194304] 0 2024-06-03T01:20:50.935 INFO:tasks.workunit.client.1.smithi044.stdout:1/239: read d0/d3/d20/d7/d2f/f33 [205689,127310] 0 2024-06-03T01:20:50.936 INFO:tasks.workunit.client.1.smithi044.stdout:1/240: chown d0/d3/d20/d40 981427 1 2024-06-03T01:20:50.936 INFO:tasks.workunit.client.1.smithi044.stdout:0/269: sync 2024-06-03T01:20:50.936 INFO:tasks.workunit.client.1.smithi044.stdout:7/324: rename d0/d3/cb to d0/d3/d8/d32/d33/c71 0 2024-06-03T01:20:50.937 INFO:tasks.workunit.client.1.smithi044.stdout:1/241: mkdir d0/d3/d20/d7/d42/d52 0 2024-06-03T01:20:50.938 INFO:tasks.workunit.client.1.smithi044.stdout:3/145: sync 2024-06-03T01:20:50.938 INFO:tasks.workunit.client.1.smithi044.stdout:3/146: write df/d13/f16 [356041,119802] 0 2024-06-03T01:20:50.938 INFO:tasks.workunit.client.1.smithi044.stdout:0/270: mkdir d3/d5c 0 2024-06-03T01:20:50.939 INFO:tasks.workunit.client.1.smithi044.stdout:6/268: dwrite d2/d4/f7 [0,4194304] 0 2024-06-03T01:20:50.939 INFO:tasks.workunit.client.1.smithi044.stdout:7/325: write d0/d3/f7 [2867038,55329] 0 2024-06-03T01:20:50.940 INFO:tasks.workunit.client.1.smithi044.stdout:1/242: rename d0/d3/d20/d7/d13/c25 to d0/d3/d20/d40/d45/c53 0 2024-06-03T01:20:50.941 INFO:tasks.workunit.client.1.smithi044.stdout:3/147: symlink df/l27 0 2024-06-03T01:20:50.941 INFO:tasks.workunit.client.1.smithi044.stdout:3/148: dread - df/d13/f1d zero size 2024-06-03T01:20:50.941 INFO:tasks.workunit.client.1.smithi044.stdout:0/271: creat d3/d49/d56/f5d x:0 0 0 2024-06-03T01:20:50.941 INFO:tasks.workunit.client.1.smithi044.stdout:0/272: chown d3/d9/d55/d5a/d29 379 1 2024-06-03T01:20:50.942 INFO:tasks.workunit.client.1.smithi044.stdout:0/273: fsync d3/d9/f44 0 2024-06-03T01:20:50.943 INFO:tasks.workunit.client.1.smithi044.stdout:5/204: dread d2/dc/f16 [0,4194304] 0 2024-06-03T01:20:50.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/215: dwrite d17/d23/f30 [0,4194304] 0 2024-06-03T01:20:50.944 INFO:tasks.workunit.client.1.smithi044.stdout:7/326: rename d0/d21/d27/d4c to d0/d16/d72 0 2024-06-03T01:20:50.944 INFO:tasks.workunit.client.1.smithi044.stdout:4/216: dread d17/d23/d28/f33 [0,4194304] 0 2024-06-03T01:20:50.944 INFO:tasks.workunit.client.1.smithi044.stdout:3/149: link df/c1f df/c28 0 2024-06-03T01:20:50.945 INFO:tasks.workunit.client.1.smithi044.stdout:0/274: mknod d3/d49/d56/c5e 0 2024-06-03T01:20:50.945 INFO:tasks.workunit.client.1.smithi044.stdout:0/275: fdatasync d3/d9/f28 0 2024-06-03T01:20:50.946 INFO:tasks.workunit.client.1.smithi044.stdout:6/269: dread d2/d23/d30/f1c [0,4194304] 0 2024-06-03T01:20:50.946 INFO:tasks.workunit.client.1.smithi044.stdout:3/150: dread df/d13/d1b/f20 [0,4194304] 0 2024-06-03T01:20:50.947 INFO:tasks.workunit.client.1.smithi044.stdout:5/205: link d2/d11/c17 d2/d11/c47 0 2024-06-03T01:20:50.948 INFO:tasks.workunit.client.1.smithi044.stdout:4/217: creat d17/d23/f4d x:0 0 0 2024-06-03T01:20:50.950 INFO:tasks.workunit.client.1.smithi044.stdout:0/276: rename d3/d49/d21/l4c to d3/d49/d20/d25/l5f 0 2024-06-03T01:20:50.952 INFO:tasks.workunit.client.1.smithi044.stdout:6/270: link d2/d23/d30/d1e/d34/d39/f42 d2/d6/f5d 0 2024-06-03T01:20:50.952 INFO:tasks.workunit.client.1.smithi044.stdout:6/271: readlink d2/d4/l8 0 2024-06-03T01:20:50.954 INFO:tasks.workunit.client.1.smithi044.stdout:3/151: unlink df/d1e/f22 0 2024-06-03T01:20:50.954 INFO:tasks.workunit.client.1.smithi044.stdout:3/152: dread - df/f21 zero size 2024-06-03T01:20:50.955 INFO:tasks.workunit.client.1.smithi044.stdout:7/327: rmdir d0/d16/d23 39 2024-06-03T01:20:50.957 INFO:tasks.workunit.client.1.smithi044.stdout:4/218: dread d17/d23/d28/d2b/f34 [0,4194304] 0 2024-06-03T01:20:50.957 INFO:tasks.workunit.client.1.smithi044.stdout:5/206: rename d2/dc/l26 to d2/d11/d12/l48 0 2024-06-03T01:20:50.959 INFO:tasks.workunit.client.1.smithi044.stdout:4/219: dread d17/d23/f20 [0,4194304] 0 2024-06-03T01:20:50.959 INFO:tasks.workunit.client.1.smithi044.stdout:6/272: read d2/d23/f33 [2132853,15913] 0 2024-06-03T01:20:50.960 INFO:tasks.workunit.client.1.smithi044.stdout:6/273: stat d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:50.960 INFO:tasks.workunit.client.1.smithi044.stdout:6/274: creat d2/d4/f5e x:0 0 0 2024-06-03T01:20:50.960 INFO:tasks.workunit.client.1.smithi044.stdout:7/328: creat d0/d3/d30/f73 x:0 0 0 2024-06-03T01:20:50.961 INFO:tasks.workunit.client.1.smithi044.stdout:5/207: creat d2/dc/f49 x:0 0 0 2024-06-03T01:20:50.973 INFO:tasks.workunit.client.1.smithi044.stdout:5/208: dread d2/f3c [0,4194304] 0 2024-06-03T01:20:50.973 INFO:tasks.workunit.client.1.smithi044.stdout:5/209: write d2/d11/d3e/f40 [2346261,28128] 0 2024-06-03T01:20:50.973 INFO:tasks.workunit.client.1.smithi044.stdout:5/210: chown d2/f3c 0 1 2024-06-03T01:20:50.975 INFO:tasks.workunit.client.1.smithi044.stdout:5/211: mkdir d2/d11/d12/d2b/d4a 0 2024-06-03T01:20:50.976 INFO:tasks.workunit.client.1.smithi044.stdout:3/153: getdents df/d1e 0 2024-06-03T01:20:50.976 INFO:tasks.workunit.client.1.smithi044.stdout:3/154: fsync df/f17 0 2024-06-03T01:20:50.976 INFO:tasks.workunit.client.1.smithi044.stdout:3/155: chown df/c28 1 1 2024-06-03T01:20:50.977 INFO:tasks.workunit.client.1.smithi044.stdout:3/156: write df/d13/f26 [4813240,25218] 0 2024-06-03T01:20:50.977 INFO:tasks.workunit.client.1.smithi044.stdout:3/157: creat df/d13/f29 x:0 0 0 2024-06-03T01:20:50.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/212: dread d2/f13 [0,4194304] 0 2024-06-03T01:20:50.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/213: chown d2/d11/d1e/d2f 16719565 1 2024-06-03T01:20:50.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/214: write d2/d11/d12/f1b [4268494,91251] 0 2024-06-03T01:20:50.987 INFO:tasks.workunit.client.1.smithi044.stdout:5/215: unlink d2/d11/l27 0 2024-06-03T01:20:50.988 INFO:tasks.workunit.client.1.smithi044.stdout:5/216: chown d2/l5 26 1 2024-06-03T01:20:50.988 INFO:tasks.workunit.client.1.smithi044.stdout:5/217: write d2/d11/d1e/d21/d24/f38 [541002,37368] 0 2024-06-03T01:20:50.988 INFO:tasks.workunit.client.1.smithi044.stdout:5/218: mknod d2/dc/c4b 0 2024-06-03T01:20:50.989 INFO:tasks.workunit.client.1.smithi044.stdout:5/219: mkdir d2/dc/d28/d4c 0 2024-06-03T01:20:50.989 INFO:tasks.workunit.client.1.smithi044.stdout:5/220: readlink d2/d11/d1e/l36 0 2024-06-03T01:20:50.991 INFO:tasks.workunit.client.1.smithi044.stdout:7/329: dwrite d0/d3/d8/d32/d33/f5b [0,4194304] 0 2024-06-03T01:20:50.991 INFO:tasks.workunit.client.1.smithi044.stdout:7/330: write d0/d21/f53 [5207646,129329] 0 2024-06-03T01:20:50.992 INFO:tasks.workunit.client.1.smithi044.stdout:6/275: dwrite d2/fe [4194304,4194304] 0 2024-06-03T01:20:50.992 INFO:tasks.workunit.client.1.smithi044.stdout:4/220: dwrite fd [0,4194304] 0 2024-06-03T01:20:50.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/221: symlink d17/d1c/l4e 0 2024-06-03T01:20:50.996 INFO:tasks.workunit.client.1.smithi044.stdout:7/331: getdents d0/d3/d8 0 2024-06-03T01:20:50.997 INFO:tasks.workunit.client.1.smithi044.stdout:7/332: truncate d0/d21/f3b 4966651 0 2024-06-03T01:20:50.997 INFO:tasks.workunit.client.1.smithi044.stdout:7/333: creat d0/f74 x:0 0 0 2024-06-03T01:20:50.999 INFO:tasks.workunit.client.1.smithi044.stdout:7/334: mknod d0/d16/c75 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/158: dwrite df/d13/d1b/f20 [0,4194304] 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/159: fsync df/d13/d1b/f20 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/276: rename d2/d23/d30/d1e/d38/d50 to d2/d23/d55/d5f 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/277: fdatasync d2/d6/f3e 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/278: read d2/d23/f33 [2274827,92605] 0 2024-06-03T01:20:51.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/279: chown d2/d23/d30/f1c 1253 1 2024-06-03T01:20:51.001 INFO:tasks.workunit.client.1.smithi044.stdout:6/280: write d2/d23/f2e [782275,60248] 0 2024-06-03T01:20:51.002 INFO:tasks.workunit.client.1.smithi044.stdout:3/160: mkdir df/d1e/d2a 0 2024-06-03T01:20:51.003 INFO:tasks.workunit.client.1.smithi044.stdout:6/281: mknod d2/d23/d55/c60 0 2024-06-03T01:20:51.003 INFO:tasks.workunit.client.1.smithi044.stdout:6/282: dread - d2/d6/db/f3b zero size 2024-06-03T01:20:51.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/283: creat d2/d23/d55/d5f/d53/f61 x:0 0 0 2024-06-03T01:20:51.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/284: stat d2/d23/d30/d1e/d34/c3d 0 2024-06-03T01:20:51.007 INFO:tasks.workunit.client.1.smithi044.stdout:6/285: creat d2/d23/d55/d5f/f62 x:0 0 0 2024-06-03T01:20:51.007 INFO:tasks.workunit.client.1.smithi044.stdout:6/286: write d2/d23/d30/d1e/f48 [3644922,94632] 0 2024-06-03T01:20:51.007 INFO:tasks.workunit.client.1.smithi044.stdout:6/287: fdatasync d2/d23/d55/d5f/d53/f57 0 2024-06-03T01:20:51.008 INFO:tasks.workunit.client.1.smithi044.stdout:4/222: read d17/d1c/f49 [3008876,129512] 0 2024-06-03T01:20:51.011 INFO:tasks.workunit.client.1.smithi044.stdout:6/288: link d2/d23/d55/d5f/d53/f61 d2/d23/d55/d5f/f63 0 2024-06-03T01:20:51.014 INFO:tasks.workunit.client.1.smithi044.stdout:4/223: link d17/d1c/f3a d17/d23/d35/f4f 0 2024-06-03T01:20:51.015 INFO:tasks.workunit.client.1.smithi044.stdout:6/289: mkdir d2/d23/d30/d1e/d34/d39/d64 0 2024-06-03T01:20:51.016 INFO:tasks.workunit.client.1.smithi044.stdout:4/224: truncate d17/f2d 2840553 0 2024-06-03T01:20:51.016 INFO:tasks.workunit.client.1.smithi044.stdout:3/161: dread fe [0,4194304] 0 2024-06-03T01:20:51.017 INFO:tasks.workunit.client.1.smithi044.stdout:6/290: mknod d2/d23/c65 0 2024-06-03T01:20:51.019 INFO:tasks.workunit.client.1.smithi044.stdout:4/225: unlink d17/d23/d28/d2b/c27 0 2024-06-03T01:20:51.019 INFO:tasks.workunit.client.1.smithi044.stdout:4/226: truncate d17/d23/d28/d2b/f21 197200 0 2024-06-03T01:20:51.019 INFO:tasks.workunit.client.1.smithi044.stdout:4/227: readlink l9 0 2024-06-03T01:20:51.019 INFO:tasks.workunit.client.1.smithi044.stdout:4/228: truncate d17/d23/d28/f3e 555561 0 2024-06-03T01:20:51.022 INFO:tasks.workunit.client.1.smithi044.stdout:4/229: dread d17/d23/d28/d2b/f3d [0,4194304] 0 2024-06-03T01:20:51.023 INFO:tasks.workunit.client.1.smithi044.stdout:4/230: rename d17/d23 to d17/d23/d28/d39/d50 22 2024-06-03T01:20:51.023 INFO:tasks.workunit.client.1.smithi044.stdout:4/231: dread - d17/d23/d28/d39/f4b zero size 2024-06-03T01:20:51.032 INFO:tasks.workunit.client.1.smithi044.stdout:7/335: dwrite d0/d16/f3c [0,4194304] 0 2024-06-03T01:20:51.032 INFO:tasks.workunit.client.1.smithi044.stdout:7/336: chown d0/d3/d8/d32/l5f 1807863 1 2024-06-03T01:20:51.032 INFO:tasks.workunit.client.1.smithi044.stdout:7/337: chown d0/d3/d8/lc 121 1 2024-06-03T01:20:51.032 INFO:tasks.workunit.client.1.smithi044.stdout:5/221: dwrite d2/d11/d1e/d21/d24/f38 [0,4194304] 0 2024-06-03T01:20:51.033 INFO:tasks.workunit.client.1.smithi044.stdout:5/222: fdatasync f1 0 2024-06-03T01:20:51.034 INFO:tasks.workunit.client.1.smithi044.stdout:7/338: rmdir d0/d16/d72 39 2024-06-03T01:20:51.035 INFO:tasks.workunit.client.1.smithi044.stdout:7/339: rmdir d0/d3/d8 39 2024-06-03T01:20:51.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/340: dread d0/d16/f4d [0,4194304] 0 2024-06-03T01:20:51.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/341: chown d0/d16/f3c 8607 1 2024-06-03T01:20:51.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/342: write d0/d16/f3c [4318675,24045] 0 2024-06-03T01:20:51.041 INFO:tasks.workunit.client.1.smithi044.stdout:7/343: dread d0/d3/f18 [0,4194304] 0 2024-06-03T01:20:51.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/223: write d2/dc/f22 [1548349,36090] 0 2024-06-03T01:20:51.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/224: write d2/d11/d3e/f40 [3975965,122013] 0 2024-06-03T01:20:51.043 INFO:tasks.workunit.client.1.smithi044.stdout:7/344: dread d0/d3/d8/d32/f48 [0,4194304] 0 2024-06-03T01:20:51.043 INFO:tasks.workunit.client.1.smithi044.stdout:7/345: write d0/d3/f18 [1893668,95277] 0 2024-06-03T01:20:51.043 INFO:tasks.workunit.client.1.smithi044.stdout:5/225: link d2/d11/d12/l23 d2/d11/l4d 0 2024-06-03T01:20:51.045 INFO:tasks.workunit.client.1.smithi044.stdout:5/226: creat d2/dc/d28/d4c/f4e x:0 0 0 2024-06-03T01:20:51.045 INFO:tasks.workunit.client.1.smithi044.stdout:7/346: getdents d0/d3/d8/d32/d33 0 2024-06-03T01:20:51.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/347: mkdir d0/d21/d27/d6e/d76 0 2024-06-03T01:20:51.049 INFO:tasks.workunit.client.1.smithi044.stdout:5/227: truncate d2/fa 2622364 0 2024-06-03T01:20:51.049 INFO:tasks.workunit.client.1.smithi044.stdout:5/228: chown d2/fa 137200766 1 2024-06-03T01:20:51.052 INFO:tasks.workunit.client.1.smithi044.stdout:5/229: rename d2/d11/d1e/d21/d24/l43 to d2/d11/d12/d2b/d4a/l4f 0 2024-06-03T01:20:51.052 INFO:tasks.workunit.client.1.smithi044.stdout:5/230: chown d2/dc/d28 23 1 2024-06-03T01:20:51.052 INFO:tasks.workunit.client.1.smithi044.stdout:5/231: readlink d2/d11/d12/d2b/d4a/l4f 0 2024-06-03T01:20:51.055 INFO:tasks.workunit.client.1.smithi044.stdout:5/232: unlink d2/dc/le 0 2024-06-03T01:20:51.056 INFO:tasks.workunit.client.1.smithi044.stdout:5/233: creat d2/d11/d1e/d45/f50 x:0 0 0 2024-06-03T01:20:51.057 INFO:tasks.workunit.client.1.smithi044.stdout:5/234: getdents d2/d11/d12/d2b 0 2024-06-03T01:20:51.058 INFO:tasks.workunit.client.1.smithi044.stdout:5/235: rename d2/dc/f32 to d2/d11/d1e/f51 0 2024-06-03T01:20:51.059 INFO:tasks.workunit.client.1.smithi044.stdout:5/236: truncate d2/d11/d12/f1b 2574143 0 2024-06-03T01:20:51.060 INFO:tasks.workunit.client.1.smithi044.stdout:7/348: dwrite d0/d3/d8/f1b [0,4194304] 0 2024-06-03T01:20:51.061 INFO:tasks.workunit.client.1.smithi044.stdout:5/237: symlink d2/d11/l52 0 2024-06-03T01:20:51.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/349: symlink d0/d16/l77 0 2024-06-03T01:20:51.071 INFO:tasks.workunit.client.1.smithi044.stdout:5/238: symlink d2/d11/d12/d2b/d4a/l53 0 2024-06-03T01:20:51.074 INFO:tasks.workunit.client.1.smithi044.stdout:5/239: mknod d2/d11/d12/d2b/d44/c54 0 2024-06-03T01:20:51.089 INFO:tasks.workunit.client.1.smithi044.stdout:7/350: dwrite d0/f74 [0,4194304] 0 2024-06-03T01:20:51.089 INFO:tasks.workunit.client.1.smithi044.stdout:7/351: write d0/d3/d8/f14 [2176021,115925] 0 2024-06-03T01:20:51.091 INFO:tasks.workunit.client.1.smithi044.stdout:5/240: dwrite d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:20:51.091 INFO:tasks.workunit.client.1.smithi044.stdout:5/241: stat d2/d11/d3e 0 2024-06-03T01:20:51.092 INFO:tasks.workunit.client.1.smithi044.stdout:7/352: creat d0/d16/d23/f78 x:0 0 0 2024-06-03T01:20:51.092 INFO:tasks.workunit.client.1.smithi044.stdout:7/353: creat d0/d3/d30/f79 x:0 0 0 2024-06-03T01:20:51.114 INFO:tasks.workunit.client.1.smithi044.stdout:5/242: dwrite d2/dc/d28/d4c/f4e [0,4194304] 0 2024-06-03T01:20:51.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/354: dwrite d0/d21/f3b [0,4194304] 0 2024-06-03T01:20:51.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/355: creat d0/d21/d63/f7a x:0 0 0 2024-06-03T01:20:51.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/356: stat d0/l45 0 2024-06-03T01:20:51.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/243: mknod d2/dc/d28/c55 0 2024-06-03T01:20:51.119 INFO:tasks.workunit.client.1.smithi044.stdout:7/357: rmdir d0/d3/d8 39 2024-06-03T01:20:51.127 INFO:tasks.workunit.client.1.smithi044.stdout:7/358: rmdir d0 39 2024-06-03T01:20:51.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/244: dwrite d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:20:51.141 INFO:tasks.workunit.client.1.smithi044.stdout:5/245: write d2/d11/d12/f37 [1272309,9058] 0 2024-06-03T01:20:51.141 INFO:tasks.workunit.client.1.smithi044.stdout:5/246: truncate d2/d11/d12/f39 755733 0 2024-06-03T01:20:51.141 INFO:tasks.workunit.client.1.smithi044.stdout:5/247: chown d2/d11/d12/f37 1709983314 1 2024-06-03T01:20:51.141 INFO:tasks.workunit.client.1.smithi044.stdout:5/248: write d2/f3c [1048868,119740] 0 2024-06-03T01:20:51.143 INFO:tasks.workunit.client.1.smithi044.stdout:5/249: symlink d2/d11/d1e/d2f/l56 0 2024-06-03T01:20:51.144 INFO:tasks.workunit.client.1.smithi044.stdout:5/250: dread d2/d11/d12/f25 [0,4194304] 0 2024-06-03T01:20:51.144 INFO:tasks.workunit.client.1.smithi044.stdout:7/359: dwrite d0/d3/d30/f73 [0,4194304] 0 2024-06-03T01:20:51.144 INFO:tasks.workunit.client.1.smithi044.stdout:5/251: dread d2/d11/d12/f29 [0,4194304] 0 2024-06-03T01:20:51.146 INFO:tasks.workunit.client.1.smithi044.stdout:7/360: mkdir d0/d3/d8/d32/d51/d7b 0 2024-06-03T01:20:51.147 INFO:tasks.workunit.client.1.smithi044.stdout:7/361: creat d0/f7c x:0 0 0 2024-06-03T01:20:51.149 INFO:tasks.workunit.client.1.smithi044.stdout:5/252: rename d2/d11/d1e/d45 to d2/d11/d12/d2b/d4a/d57 0 2024-06-03T01:20:51.156 INFO:tasks.workunit.client.1.smithi044.stdout:7/362: dread d0/d3/d8/d32/d51/f66 [0,4194304] 0 2024-06-03T01:20:51.158 INFO:tasks.workunit.client.1.smithi044.stdout:7/363: rename d0/d3/d8/lc to d0/d16/d72/d5c/l7d 0 2024-06-03T01:20:51.159 INFO:tasks.workunit.client.1.smithi044.stdout:7/364: rename d0/l45 to d0/d3/l7e 0 2024-06-03T01:20:51.159 INFO:tasks.workunit.client.1.smithi044.stdout:7/365: fsync d0/f4 0 2024-06-03T01:20:51.160 INFO:tasks.workunit.client.1.smithi044.stdout:7/366: mkdir d0/d21/d63/d7f 0 2024-06-03T01:20:51.161 INFO:tasks.workunit.client.1.smithi044.stdout:5/253: dread d2/f13 [0,4194304] 0 2024-06-03T01:20:51.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/254: creat d2/d11/d12/d2b/f58 x:0 0 0 2024-06-03T01:20:51.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/255: write d2/f13 [3150054,100000] 0 2024-06-03T01:20:51.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/256: write d2/f13 [4988093,41557] 0 2024-06-03T01:20:51.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/257: truncate d2/dc/f49 514214 0 2024-06-03T01:20:51.167 INFO:tasks.workunit.client.1.smithi044.stdout:7/367: dread d0/f74 [0,4194304] 0 2024-06-03T01:20:51.171 INFO:tasks.workunit.client.1.smithi044.stdout:5/258: dread d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:20:51.173 INFO:tasks.workunit.client.1.smithi044.stdout:5/259: creat d2/d11/d3e/f59 x:0 0 0 2024-06-03T01:20:51.173 INFO:tasks.workunit.client.1.smithi044.stdout:5/260: write d2/d11/d12/d2b/f58 [762873,45936] 0 2024-06-03T01:20:51.173 INFO:tasks.workunit.client.1.smithi044.stdout:9/202: sync 2024-06-03T01:20:51.174 INFO:tasks.workunit.client.1.smithi044.stdout:2/195: sync 2024-06-03T01:20:51.176 INFO:tasks.workunit.client.1.smithi044.stdout:9/203: rename d6/d8/db/c18 to d6/d38/c3e 0 2024-06-03T01:20:51.177 INFO:tasks.workunit.client.1.smithi044.stdout:2/196: getdents db/d15 0 2024-06-03T01:20:51.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/204: rename d6/dd/f2c to d6/d8/d16/f3f 0 2024-06-03T01:20:51.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/205: stat d6/dd/f21 0 2024-06-03T01:20:51.186 INFO:tasks.workunit.client.1.smithi044.stdout:9/206: dread f4 [0,4194304] 0 2024-06-03T01:20:51.187 INFO:tasks.workunit.client.1.smithi044.stdout:5/261: write d2/f8 [2847111,128959] 0 2024-06-03T01:20:51.187 INFO:tasks.workunit.client.1.smithi044.stdout:7/368: dwrite d0/d3/d8/d32/f38 [0,4194304] 0 2024-06-03T01:20:51.187 INFO:tasks.workunit.client.1.smithi044.stdout:7/369: write d0/fd [338487,113537] 0 2024-06-03T01:20:51.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/370: creat d0/d21/d27/d6e/d76/f80 x:0 0 0 2024-06-03T01:20:51.193 INFO:tasks.workunit.client.1.smithi044.stdout:2/197: dwrite f7 [4194304,4194304] 0 2024-06-03T01:20:51.193 INFO:tasks.workunit.client.1.smithi044.stdout:2/198: stat db/f12 0 2024-06-03T01:20:51.196 INFO:tasks.workunit.client.1.smithi044.stdout:5/262: rmdir d2 39 2024-06-03T01:20:51.199 INFO:tasks.workunit.client.1.smithi044.stdout:2/199: symlink db/d15/l44 0 2024-06-03T01:20:51.201 INFO:tasks.workunit.client.1.smithi044.stdout:2/200: getdents db/d1b/d3f 0 2024-06-03T01:20:51.204 INFO:tasks.workunit.client.1.smithi044.stdout:9/207: dwrite d6/d8/db/f32 [0,4194304] 0 2024-06-03T01:20:51.206 INFO:tasks.workunit.client.1.smithi044.stdout:9/208: mknod d6/d8/d3b/c40 0 2024-06-03T01:20:51.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/263: dwrite d2/d11/d1e/f51 [4194304,4194304] 0 2024-06-03T01:20:51.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/264: readlink d2/d11/d12/d2b/d44/l3a 0 2024-06-03T01:20:51.222 INFO:tasks.workunit.client.1.smithi044.stdout:7/371: dwrite d0/d16/d23/f78 [0,4194304] 0 2024-06-03T01:20:51.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/201: dwrite db/d1b/f22 [0,4194304] 0 2024-06-03T01:20:51.227 INFO:tasks.workunit.client.1.smithi044.stdout:5/265: dread d2/f13 [0,4194304] 0 2024-06-03T01:20:51.244 INFO:tasks.workunit.client.1.smithi044.stdout:5/266: dwrite d2/f8 [0,4194304] 0 2024-06-03T01:20:51.244 INFO:tasks.workunit.client.1.smithi044.stdout:7/372: dwrite d0/f4 [0,4194304] 0 2024-06-03T01:20:51.245 INFO:tasks.workunit.client.1.smithi044.stdout:7/373: dread - d0/d21/d63/f6b zero size 2024-06-03T01:20:51.246 INFO:tasks.workunit.client.1.smithi044.stdout:5/267: rename d2/d11/d12/d2b/d44/l3a to d2/d11/d1e/d3b/l5a 0 2024-06-03T01:20:51.248 INFO:tasks.workunit.client.1.smithi044.stdout:7/374: creat d0/d16/d72/d5c/f81 x:0 0 0 2024-06-03T01:20:51.249 INFO:tasks.workunit.client.1.smithi044.stdout:7/375: creat d0/d21/d63/d7f/f82 x:0 0 0 2024-06-03T01:20:51.251 INFO:tasks.workunit.client.1.smithi044.stdout:5/268: write d2/dc/f16 [1405723,129115] 0 2024-06-03T01:20:51.252 INFO:tasks.workunit.client.1.smithi044.stdout:5/269: write d2/dc/f16 [2220023,129402] 0 2024-06-03T01:20:51.252 INFO:tasks.workunit.client.1.smithi044.stdout:5/270: fsync d2/d11/d12/d2b/f58 0 2024-06-03T01:20:51.253 INFO:tasks.workunit.client.1.smithi044.stdout:7/376: link d0/d21/f65 d0/d21/d27/f83 0 2024-06-03T01:20:51.256 INFO:tasks.workunit.client.1.smithi044.stdout:5/271: mknod d2/d11/d12/d2b/d4a/c5b 0 2024-06-03T01:20:51.257 INFO:tasks.workunit.client.1.smithi044.stdout:7/377: mkdir d0/d21/d84 0 2024-06-03T01:20:51.259 INFO:tasks.workunit.client.1.smithi044.stdout:7/378: getdents d0/d16/d23 0 2024-06-03T01:20:51.260 INFO:tasks.workunit.client.1.smithi044.stdout:0/277: sync 2024-06-03T01:20:51.260 INFO:tasks.workunit.client.1.smithi044.stdout:1/243: sync 2024-06-03T01:20:51.260 INFO:tasks.workunit.client.1.smithi044.stdout:8/251: sync 2024-06-03T01:20:51.261 INFO:tasks.workunit.client.1.smithi044.stdout:1/244: creat d0/d3/d20/f54 x:0 0 0 2024-06-03T01:20:51.265 INFO:tasks.workunit.client.1.smithi044.stdout:1/245: creat d0/d3/d20/d40/f55 x:0 0 0 2024-06-03T01:20:51.269 INFO:tasks.workunit.client.1.smithi044.stdout:0/278: dread d3/d9/f13 [0,4194304] 0 2024-06-03T01:20:51.269 INFO:tasks.workunit.client.1.smithi044.stdout:0/279: creat d3/d49/d56/f60 x:0 0 0 2024-06-03T01:20:51.272 INFO:tasks.workunit.client.1.smithi044.stdout:0/280: mknod d3/d9/d55/c61 0 2024-06-03T01:20:51.273 INFO:tasks.workunit.client.1.smithi044.stdout:0/281: mkdir d3/d49/d56/d62 0 2024-06-03T01:20:51.274 INFO:tasks.workunit.client.1.smithi044.stdout:0/282: creat d3/d49/d20/f63 x:0 0 0 2024-06-03T01:20:51.275 INFO:tasks.workunit.client.1.smithi044.stdout:0/283: rmdir d3/d9/d36/d46 0 2024-06-03T01:20:51.277 INFO:tasks.workunit.client.1.smithi044.stdout:1/246: write d0/d3/f19 [3625572,121896] 0 2024-06-03T01:20:51.278 INFO:tasks.workunit.client.1.smithi044.stdout:0/284: link d3/f4 d3/d49/d20/d25/f64 0 2024-06-03T01:20:51.278 INFO:tasks.workunit.client.1.smithi044.stdout:5/272: dwrite d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:20:51.279 INFO:tasks.workunit.client.1.smithi044.stdout:7/379: dwrite d0/d3/d8/d32/f6c [0,4194304] 0 2024-06-03T01:20:51.279 INFO:tasks.workunit.client.1.smithi044.stdout:1/247: creat d0/d3/d20/d7/d42/f56 x:0 0 0 2024-06-03T01:20:51.279 INFO:tasks.workunit.client.1.smithi044.stdout:8/252: dwrite d5/d10/f34 [0,4194304] 0 2024-06-03T01:20:51.280 INFO:tasks.workunit.client.1.smithi044.stdout:8/253: readlink d5/d16/l41 0 2024-06-03T01:20:51.280 INFO:tasks.workunit.client.1.smithi044.stdout:5/273: creat d2/d11/d12/d2b/d44/f5c x:0 0 0 2024-06-03T01:20:51.283 INFO:tasks.workunit.client.1.smithi044.stdout:7/380: symlink d0/d21/d84/l85 0 2024-06-03T01:20:51.284 INFO:tasks.workunit.client.1.smithi044.stdout:1/248: creat d0/d3/d20/d7/d4e/f57 x:0 0 0 2024-06-03T01:20:51.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/285: dread d3/d49/d21/f4a [0,4194304] 0 2024-06-03T01:20:51.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/254: mkdir d5/d20/d47 0 2024-06-03T01:20:51.290 INFO:tasks.workunit.client.1.smithi044.stdout:5/274: creat d2/d11/d12/d2b/d4a/f5d x:0 0 0 2024-06-03T01:20:51.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/249: mknod d0/d3/d20/d40/d45/d4b/c58 0 2024-06-03T01:20:51.295 INFO:tasks.workunit.client.1.smithi044.stdout:0/286: creat d3/d49/d20/d25/f65 x:0 0 0 2024-06-03T01:20:51.295 INFO:tasks.workunit.client.1.smithi044.stdout:1/250: dread d0/d3/f19 [0,4194304] 0 2024-06-03T01:20:51.297 INFO:tasks.workunit.client.1.smithi044.stdout:8/255: rename d5/f11 to d5/d20/d2b/f48 0 2024-06-03T01:20:51.300 INFO:tasks.workunit.client.1.smithi044.stdout:5/275: creat d2/d11/d1e/d3b/f5e x:0 0 0 2024-06-03T01:20:51.301 INFO:tasks.workunit.client.1.smithi044.stdout:0/287: symlink d3/d5c/l66 0 2024-06-03T01:20:51.302 INFO:tasks.workunit.client.1.smithi044.stdout:8/256: write d5/f15 [2934742,118484] 0 2024-06-03T01:20:51.304 INFO:tasks.workunit.client.1.smithi044.stdout:0/288: truncate d3/d49/f3e 2138456 0 2024-06-03T01:20:51.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/257: creat d5/d20/d47/f49 x:0 0 0 2024-06-03T01:20:51.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/258: chown d5/d10/l29 353942 1 2024-06-03T01:20:51.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/259: fsync f3 0 2024-06-03T01:20:51.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/260: truncate d5/d16/f2e 658366 0 2024-06-03T01:20:51.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/261: write f3 [752564,52935] 0 2024-06-03T01:20:51.307 INFO:tasks.workunit.client.1.smithi044.stdout:0/289: link d3/f18 d3/d49/d20/d25/f67 0 2024-06-03T01:20:51.312 INFO:tasks.workunit.client.1.smithi044.stdout:1/251: dwrite d0/d3/d20/d7/d42/f56 [0,4194304] 0 2024-06-03T01:20:51.316 INFO:tasks.workunit.client.1.smithi044.stdout:1/252: creat d0/d3/d20/d7/d2f/f59 x:0 0 0 2024-06-03T01:20:51.316 INFO:tasks.workunit.client.1.smithi044.stdout:1/253: chown d0/d3/d20/d7/d2f 55 1 2024-06-03T01:20:51.316 INFO:tasks.workunit.client.1.smithi044.stdout:1/254: fdatasync d0/d3/d20/d7/d42/f56 0 2024-06-03T01:20:51.317 INFO:tasks.workunit.client.1.smithi044.stdout:1/255: mkdir d0/d3/d20/d40/d45/d4b/d5a 0 2024-06-03T01:20:51.318 INFO:tasks.workunit.client.1.smithi044.stdout:1/256: truncate d0/d3/f9 2995320 0 2024-06-03T01:20:51.321 INFO:tasks.workunit.client.1.smithi044.stdout:8/262: dwrite d5/d16/d1d/f23 [0,4194304] 0 2024-06-03T01:20:51.321 INFO:tasks.workunit.client.1.smithi044.stdout:0/290: write d3/d9/f22 [1510752,125071] 0 2024-06-03T01:20:51.324 INFO:tasks.workunit.client.1.smithi044.stdout:8/263: truncate d5/d16/f3c 768310 0 2024-06-03T01:20:51.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/291: mknod d3/d49/d21/c68 0 2024-06-03T01:20:51.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/292: creat d3/f69 x:0 0 0 2024-06-03T01:20:51.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/293: chown d3/d9/l2b 93177999 1 2024-06-03T01:20:51.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/294: chown d3 3 1 2024-06-03T01:20:51.330 INFO:tasks.workunit.client.1.smithi044.stdout:5/276: dwrite d2/d11/d1e/d21/d24/f38 [0,4194304] 0 2024-06-03T01:20:51.330 INFO:tasks.workunit.client.1.smithi044.stdout:5/277: readlink d2/d11/d1e/l36 0 2024-06-03T01:20:51.331 INFO:tasks.workunit.client.1.smithi044.stdout:0/295: unlink d3/d9/d55/d5a/d29/c41 0 2024-06-03T01:20:51.332 INFO:tasks.workunit.client.1.smithi044.stdout:8/264: getdents d5/d16/d1d 0 2024-06-03T01:20:51.332 INFO:tasks.workunit.client.1.smithi044.stdout:8/265: creat d5/d20/f4a x:0 0 0 2024-06-03T01:20:51.333 INFO:tasks.workunit.client.1.smithi044.stdout:8/266: fdatasync d5/d20/d47/f49 0 2024-06-03T01:20:51.333 INFO:tasks.workunit.client.1.smithi044.stdout:8/267: creat d5/d20/d33/f4b x:0 0 0 2024-06-03T01:20:51.337 INFO:tasks.workunit.client.1.smithi044.stdout:0/296: mknod d3/d49/d56/c6a 0 2024-06-03T01:20:51.337 INFO:tasks.workunit.client.1.smithi044.stdout:0/297: write d3/f14 [30212,21141] 0 2024-06-03T01:20:51.337 INFO:tasks.workunit.client.1.smithi044.stdout:1/257: dwrite d0/d3/d20/d7/d42/d47/f4c [0,4194304] 0 2024-06-03T01:20:51.341 INFO:tasks.workunit.client.1.smithi044.stdout:0/298: mkdir d3/d6b 0 2024-06-03T01:20:51.342 INFO:tasks.workunit.client.1.smithi044.stdout:8/268: getdents d5/d20/d47 0 2024-06-03T01:20:51.342 INFO:tasks.workunit.client.1.smithi044.stdout:8/269: fdatasync d5/d20/d2b/f48 0 2024-06-03T01:20:51.343 INFO:tasks.workunit.client.1.smithi044.stdout:0/299: symlink d3/d49/l6c 0 2024-06-03T01:20:51.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/300: link d3/d9/d36/f48 d3/f6d 0 2024-06-03T01:20:51.346 INFO:tasks.workunit.client.1.smithi044.stdout:1/258: dread d0/d3/f9 [0,4194304] 0 2024-06-03T01:20:51.346 INFO:tasks.workunit.client.1.smithi044.stdout:1/259: chown d0/d16 265251 1 2024-06-03T01:20:51.346 INFO:tasks.workunit.client.1.smithi044.stdout:1/260: dread - d0/d3/d20/d7/d4e/f57 zero size 2024-06-03T01:20:51.347 INFO:tasks.workunit.client.1.smithi044.stdout:8/270: rename d5/fe to d5/d20/d33/f4c 0 2024-06-03T01:20:51.348 INFO:tasks.workunit.client.1.smithi044.stdout:1/261: mkdir d0/d16/d5b 0 2024-06-03T01:20:51.349 INFO:tasks.workunit.client.1.smithi044.stdout:8/271: symlink d5/d20/l4d 0 2024-06-03T01:20:51.349 INFO:tasks.workunit.client.1.smithi044.stdout:8/272: write d5/d16/f3a [540318,120924] 0 2024-06-03T01:20:51.351 INFO:tasks.workunit.client.1.smithi044.stdout:0/301: rmdir d3/d49/d20 39 2024-06-03T01:20:51.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/302: dread - d3/f3a zero size 2024-06-03T01:20:51.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/303: truncate d3/d49/d56/f5d 991654 0 2024-06-03T01:20:51.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/304: write d3/f69 [1040414,82323] 0 2024-06-03T01:20:51.353 INFO:tasks.workunit.client.1.smithi044.stdout:0/305: truncate d3/f8 4016375 0 2024-06-03T01:20:51.353 INFO:tasks.workunit.client.1.smithi044.stdout:0/306: fsync d3/d9/f4d 0 2024-06-03T01:20:51.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/307: creat d3/f6e x:0 0 0 2024-06-03T01:20:51.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/308: fdatasync d3/d49/d20/d25/f65 0 2024-06-03T01:20:51.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/309: read - d3/f45 zero size 2024-06-03T01:20:51.355 INFO:tasks.workunit.client.1.smithi044.stdout:8/273: unlink d5/l14 0 2024-06-03T01:20:51.358 INFO:tasks.workunit.client.1.smithi044.stdout:8/274: rmdir d5/d20/d47 39 2024-06-03T01:20:51.358 INFO:tasks.workunit.client.1.smithi044.stdout:8/275: fsync d5/f15 0 2024-06-03T01:20:51.358 INFO:tasks.workunit.client.1.smithi044.stdout:8/276: write d5/d16/d1d/f42 [872338,67144] 0 2024-06-03T01:20:51.360 INFO:tasks.workunit.client.1.smithi044.stdout:0/310: write d3/d9/f13 [4897986,19821] 0 2024-06-03T01:20:51.362 INFO:tasks.workunit.client.1.smithi044.stdout:1/262: dwrite d0/d3/d20/d7/d2f/f59 [0,4194304] 0 2024-06-03T01:20:51.362 INFO:tasks.workunit.client.1.smithi044.stdout:1/263: stat d0/f32 0 2024-06-03T01:20:51.362 INFO:tasks.workunit.client.1.smithi044.stdout:8/277: getdents d5/d20/d2b 0 2024-06-03T01:20:51.364 INFO:tasks.workunit.client.1.smithi044.stdout:0/311: creat d3/d6b/f6f x:0 0 0 2024-06-03T01:20:51.368 INFO:tasks.workunit.client.1.smithi044.stdout:1/264: getdents d0/d3/d14/d30 0 2024-06-03T01:20:51.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/278: mknod d5/d20/d2b/c4e 0 2024-06-03T01:20:51.370 INFO:tasks.workunit.client.1.smithi044.stdout:0/312: mknod d3/d9/d55/d5a/d29/c70 0 2024-06-03T01:20:51.371 INFO:tasks.workunit.client.1.smithi044.stdout:1/265: unlink d0/f2c 0 2024-06-03T01:20:51.372 INFO:tasks.workunit.client.1.smithi044.stdout:1/266: chown d0/d3/d20/d7/d13/d4f 12 1 2024-06-03T01:20:51.372 INFO:tasks.workunit.client.1.smithi044.stdout:0/313: creat d3/d49/d20/f71 x:0 0 0 2024-06-03T01:20:51.373 INFO:tasks.workunit.client.1.smithi044.stdout:1/267: mkdir d0/d3/d20/d40/d45/d4b/d5c 0 2024-06-03T01:20:51.373 INFO:tasks.workunit.client.1.smithi044.stdout:1/268: write d0/d3/d14/f50 [326901,24133] 0 2024-06-03T01:20:51.374 INFO:tasks.workunit.client.1.smithi044.stdout:0/314: mknod d3/d49/d20/c72 0 2024-06-03T01:20:51.375 INFO:tasks.workunit.client.1.smithi044.stdout:1/269: symlink d0/d3/d20/d40/d45/l5d 0 2024-06-03T01:20:51.376 INFO:tasks.workunit.client.1.smithi044.stdout:1/270: creat d0/d3/d14/f5e x:0 0 0 2024-06-03T01:20:51.376 INFO:tasks.workunit.client.1.smithi044.stdout:8/279: dread f2 [0,4194304] 0 2024-06-03T01:20:51.377 INFO:tasks.workunit.client.1.smithi044.stdout:9/209: sync 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:6/291: sync 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:3/162: sync 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:9/210: rename d6/d8/d16/d27 to d6/d8/d16/d27/d41 22 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:4/232: sync 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:6/292: chown d2/d23/d30/d1e/c24 18417304 1 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:4/233: truncate d17/d23/d28/d39/f4b 267678 0 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:6/293: write d2/d23/d55/d5f/f59 [158153,64574] 0 2024-06-03T01:20:51.378 INFO:tasks.workunit.client.1.smithi044.stdout:1/271: rmdir d0/d16/d5b 0 2024-06-03T01:20:51.379 INFO:tasks.workunit.client.1.smithi044.stdout:8/280: symlink d5/d16/d1d/d32/l4f 0 2024-06-03T01:20:51.381 INFO:tasks.workunit.client.1.smithi044.stdout:4/234: mkdir d17/d23/d28/d2b/d51 0 2024-06-03T01:20:51.382 INFO:tasks.workunit.client.1.smithi044.stdout:6/294: creat d2/d23/d30/d1e/f66 x:0 0 0 2024-06-03T01:20:51.382 INFO:tasks.workunit.client.1.smithi044.stdout:6/295: read - d2/d23/d55/d5f/f62 zero size 2024-06-03T01:20:51.382 INFO:tasks.workunit.client.1.smithi044.stdout:6/296: write d2/d23/d30/f1f [198528,111200] 0 2024-06-03T01:20:51.383 INFO:tasks.workunit.client.1.smithi044.stdout:1/272: unlink d0/d3/d20/d40/d45/c53 0 2024-06-03T01:20:51.386 INFO:tasks.workunit.client.1.smithi044.stdout:3/163: link c0 df/d13/d1b/c2b 0 2024-06-03T01:20:51.386 INFO:tasks.workunit.client.1.smithi044.stdout:3/164: creat df/d13/f2c x:0 0 0 2024-06-03T01:20:51.387 INFO:tasks.workunit.client.1.smithi044.stdout:3/165: creat df/f2d x:0 0 0 2024-06-03T01:20:51.387 INFO:tasks.workunit.client.1.smithi044.stdout:3/166: fsync df/f11 0 2024-06-03T01:20:51.393 INFO:tasks.workunit.client.1.smithi044.stdout:9/211: rmdir d6/d8 39 2024-06-03T01:20:51.393 INFO:tasks.workunit.client.1.smithi044.stdout:9/212: dread - d6/d8/d16/f34 zero size 2024-06-03T01:20:51.400 INFO:tasks.workunit.client.1.smithi044.stdout:6/297: symlink d2/d23/d30/d1e/d34/d39/d64/l67 0 2024-06-03T01:20:51.400 INFO:tasks.workunit.client.1.smithi044.stdout:6/298: dread - d2/d23/f35 zero size 2024-06-03T01:20:51.400 INFO:tasks.workunit.client.1.smithi044.stdout:6/299: write d2/d23/d30/f1c [3696032,56408] 0 2024-06-03T01:20:51.400 INFO:tasks.workunit.client.1.smithi044.stdout:1/273: rename d0/d3/f28 to d0/d3/d20/d40/d45/d4b/d5a/f5f 0 2024-06-03T01:20:51.403 INFO:tasks.workunit.client.1.smithi044.stdout:8/281: mkdir d5/d50 0 2024-06-03T01:20:51.403 INFO:tasks.workunit.client.1.smithi044.stdout:8/282: dread - d5/d10/f46 zero size 2024-06-03T01:20:51.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/235: dwrite f5 [0,4194304] 0 2024-06-03T01:20:51.411 INFO:tasks.workunit.client.1.smithi044.stdout:3/167: dwrite df/f21 [0,4194304] 0 2024-06-03T01:20:51.411 INFO:tasks.workunit.client.1.smithi044.stdout:6/300: rename d2/d23/d55/c60 to d2/d4/d3f/c68 0 2024-06-03T01:20:51.411 INFO:tasks.workunit.client.1.smithi044.stdout:6/301: fsync d2/d23/d30/d1e/d34/d39/f56 0 2024-06-03T01:20:51.412 INFO:tasks.workunit.client.1.smithi044.stdout:9/213: dwrite d6/dd/f3d [0,4194304] 0 2024-06-03T01:20:51.412 INFO:tasks.workunit.client.1.smithi044.stdout:9/214: dread - d6/d8/d16/d27/f39 zero size 2024-06-03T01:20:51.414 INFO:tasks.workunit.client.1.smithi044.stdout:4/236: rmdir d17/d23/d28/d2b/d51 0 2024-06-03T01:20:51.414 INFO:tasks.workunit.client.1.smithi044.stdout:4/237: chown l9 51403 1 2024-06-03T01:20:51.420 INFO:tasks.workunit.client.1.smithi044.stdout:6/302: unlink d2/d23/d30/d1e/f48 0 2024-06-03T01:20:51.423 INFO:tasks.workunit.client.1.smithi044.stdout:1/274: link d0/d3/d20/d7/f36 d0/d3/d20/d7/d4e/f60 0 2024-06-03T01:20:51.430 INFO:tasks.workunit.client.1.smithi044.stdout:9/215: rename d6/d8 to d6/d38/d42 0 2024-06-03T01:20:51.432 INFO:tasks.workunit.client.1.smithi044.stdout:3/168: rmdir df/d13 39 2024-06-03T01:20:51.432 INFO:tasks.workunit.client.1.smithi044.stdout:3/169: fsync df/d13/f29 0 2024-06-03T01:20:51.432 INFO:tasks.workunit.client.1.smithi044.stdout:1/275: mkdir d0/d3/d20/d61 0 2024-06-03T01:20:51.432 INFO:tasks.workunit.client.1.smithi044.stdout:8/283: dwrite d5/f2d [0,4194304] 0 2024-06-03T01:20:51.433 INFO:tasks.workunit.client.1.smithi044.stdout:9/216: truncate d6/d38/d42/d16/f3f 3268557 0 2024-06-03T01:20:51.434 INFO:tasks.workunit.client.1.smithi044.stdout:1/276: truncate d0/d3/d20/d7/d4e/f60 3281981 0 2024-06-03T01:20:51.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/217: read d6/f12 [153109,78198] 0 2024-06-03T01:20:51.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/218: chown d6/d38/d42/d16/f37 229063 1 2024-06-03T01:20:51.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/284: dread - d5/d20/d2b/f35 zero size 2024-06-03T01:20:51.435 INFO:tasks.workunit.client.1.smithi044.stdout:2/202: sync 2024-06-03T01:20:51.435 INFO:tasks.workunit.client.1.smithi044.stdout:7/381: sync 2024-06-03T01:20:51.435 INFO:tasks.workunit.client.1.smithi044.stdout:7/382: dread - d0/d3/d8/d32/d51/f5e zero size 2024-06-03T01:20:51.435 INFO:tasks.workunit.client.1.smithi044.stdout:7/383: readlink d0/d16/l59 0 2024-06-03T01:20:51.435 INFO:tasks.workunit.client.1.smithi044.stdout:8/285: symlink d5/d50/l51 0 2024-06-03T01:20:51.436 INFO:tasks.workunit.client.1.smithi044.stdout:2/203: rename db/dd/d28 to db/d45 0 2024-06-03T01:20:51.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/384: symlink d0/d21/d27/d6e/d76/l86 0 2024-06-03T01:20:51.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/385: fdatasync d0/d21/d27/f83 0 2024-06-03T01:20:51.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/386: fdatasync d0/d21/d27/d6e/d76/f80 0 2024-06-03T01:20:51.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/387: write d0/d3/f7 [2899351,7739] 0 2024-06-03T01:20:51.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/204: mknod db/d1b/d2a/c46 0 2024-06-03T01:20:51.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/205: write db/d1b/d2a/f2b [944664,129231] 0 2024-06-03T01:20:51.438 INFO:tasks.workunit.client.1.smithi044.stdout:9/219: dread d6/d38/d42/f9 [8388608,4194304] 0 2024-06-03T01:20:51.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/206: chown db/dd/l34 0 1 2024-06-03T01:20:51.440 INFO:tasks.workunit.client.1.smithi044.stdout:7/388: link d0/d3/d8/d32/l5d d0/d21/d63/l87 0 2024-06-03T01:20:51.440 INFO:tasks.workunit.client.1.smithi044.stdout:7/389: write d0/d3/d8/d32/d33/f57 [544928,76338] 0 2024-06-03T01:20:51.441 INFO:tasks.workunit.client.1.smithi044.stdout:8/286: rmdir d5/d20/d33 39 2024-06-03T01:20:51.443 INFO:tasks.workunit.client.1.smithi044.stdout:1/277: dread d0/d3/d20/d7/d13/d2d/f4d [0,4194304] 0 2024-06-03T01:20:51.443 INFO:tasks.workunit.client.1.smithi044.stdout:9/220: dread d6/d38/d42/db/f22 [0,4194304] 0 2024-06-03T01:20:51.443 INFO:tasks.workunit.client.1.smithi044.stdout:9/221: stat d6/d38/d42/db/f15 0 2024-06-03T01:20:51.443 INFO:tasks.workunit.client.1.smithi044.stdout:7/390: symlink d0/d3/l88 0 2024-06-03T01:20:51.444 INFO:tasks.workunit.client.1.smithi044.stdout:1/278: creat d0/d3/f62 x:0 0 0 2024-06-03T01:20:51.445 INFO:tasks.workunit.client.1.smithi044.stdout:6/303: dwrite d2/f31 [0,4194304] 0 2024-06-03T01:20:51.446 INFO:tasks.workunit.client.1.smithi044.stdout:9/222: symlink d6/d38/d42/d3b/l43 0 2024-06-03T01:20:51.447 INFO:tasks.workunit.client.1.smithi044.stdout:7/391: mkdir d0/d21/d63/d89 0 2024-06-03T01:20:51.447 INFO:tasks.workunit.client.1.smithi044.stdout:1/279: dread d0/d3/f9 [0,4194304] 0 2024-06-03T01:20:51.449 INFO:tasks.workunit.client.1.smithi044.stdout:8/287: truncate d5/f25 1865573 0 2024-06-03T01:20:51.449 INFO:tasks.workunit.client.1.smithi044.stdout:8/288: chown d5/d50 369816523 1 2024-06-03T01:20:51.452 INFO:tasks.workunit.client.1.smithi044.stdout:6/304: mknod d2/d6/db/d5c/c69 0 2024-06-03T01:20:51.454 INFO:tasks.workunit.client.1.smithi044.stdout:9/223: creat d6/f44 x:0 0 0 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/392: rename d0/d3/d8/d32/d51/d7b to d0/d3/d8/d8a 0 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/393: chown d0/d16/d23/l4b 75717 1 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/394: stat d0/d3/d8/d32/d33/f57 0 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/395: write d0/f4 [4577713,94792] 0 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/396: dread - d0/d21/d63/d7f/f82 zero size 2024-06-03T01:20:51.459 INFO:tasks.workunit.client.1.smithi044.stdout:7/397: dread - d0/d21/d63/d7f/f82 zero size 2024-06-03T01:20:51.460 INFO:tasks.workunit.client.1.smithi044.stdout:1/280: fsync d0/f5 0 2024-06-03T01:20:51.461 INFO:tasks.workunit.client.1.smithi044.stdout:9/224: mkdir d6/dd/d45 0 2024-06-03T01:20:51.464 INFO:tasks.workunit.client.1.smithi044.stdout:7/398: mkdir d0/d21/d63/d89/d8b 0 2024-06-03T01:20:51.466 INFO:tasks.workunit.client.1.smithi044.stdout:1/281: rmdir d0/d3/d20/d7/d13/d4f 0 2024-06-03T01:20:51.467 INFO:tasks.workunit.client.1.smithi044.stdout:6/305: dwrite d2/d23/d30/d1e/d34/d39/f3c [0,4194304] 0 2024-06-03T01:20:51.467 INFO:tasks.workunit.client.1.smithi044.stdout:8/289: dwrite d5/d20/f3d [0,4194304] 0 2024-06-03T01:20:51.467 INFO:tasks.workunit.client.1.smithi044.stdout:9/225: mkdir d6/d38/d42/d46 0 2024-06-03T01:20:51.469 INFO:tasks.workunit.client.1.smithi044.stdout:1/282: creat d0/d3/d20/d40/d45/d4b/d5c/f63 x:0 0 0 2024-06-03T01:20:51.470 INFO:tasks.workunit.client.1.smithi044.stdout:1/283: read - d0/d3/f62 zero size 2024-06-03T01:20:51.470 INFO:tasks.workunit.client.1.smithi044.stdout:6/306: creat d2/d23/d30/d1e/d34/d39/d64/f6a x:0 0 0 2024-06-03T01:20:51.470 INFO:tasks.workunit.client.1.smithi044.stdout:6/307: write d2/d23/d30/f1a [486599,36779] 0 2024-06-03T01:20:51.483 INFO:tasks.workunit.client.1.smithi044.stdout:9/226: dwrite d6/d38/d42/d16/f3f [0,4194304] 0 2024-06-03T01:20:51.485 INFO:tasks.workunit.client.1.smithi044.stdout:8/290: rmdir d5/d20/d2b 39 2024-06-03T01:20:51.487 INFO:tasks.workunit.client.1.smithi044.stdout:6/308: rename d2/c11 to d2/d4/d3f/c6b 0 2024-06-03T01:20:51.492 INFO:tasks.workunit.client.1.smithi044.stdout:9/227: write d6/d38/d42/d16/f3f [1327725,10547] 0 2024-06-03T01:20:51.492 INFO:tasks.workunit.client.1.smithi044.stdout:9/228: chown d6/dd/l2a 39218170 1 2024-06-03T01:20:51.500 INFO:tasks.workunit.client.1.smithi044.stdout:1/284: getdents d0/d3/d20/d7/d2f 0 2024-06-03T01:20:51.503 INFO:tasks.workunit.client.1.smithi044.stdout:6/309: rmdir d2/d23/d55/d5f 39 2024-06-03T01:20:51.504 INFO:tasks.workunit.client.1.smithi044.stdout:9/229: creat d6/d38/d42/d3b/f47 x:0 0 0 2024-06-03T01:20:51.507 INFO:tasks.workunit.client.1.smithi044.stdout:8/291: rename d5/c17 to d5/d16/c52 0 2024-06-03T01:20:51.521 INFO:tasks.workunit.client.1.smithi044.stdout:9/230: truncate d6/d38/d42/d16/f3f 727148 0 2024-06-03T01:20:51.522 INFO:tasks.workunit.client.1.smithi044.stdout:9/231: creat d6/d38/d42/d46/f48 x:0 0 0 2024-06-03T01:20:51.522 INFO:tasks.workunit.client.1.smithi044.stdout:9/232: stat d6/d38/d42/l1d 0 2024-06-03T01:20:51.524 INFO:tasks.workunit.client.1.smithi044.stdout:8/292: truncate d5/d16/d1d/f27 3094874 0 2024-06-03T01:20:51.525 INFO:tasks.workunit.client.1.smithi044.stdout:8/293: link d5/d20/c2f d5/d10/c53 0 2024-06-03T01:20:51.525 INFO:tasks.workunit.client.1.smithi044.stdout:8/294: truncate d5/d10/f34 4700415 0 2024-06-03T01:20:51.526 INFO:tasks.workunit.client.1.smithi044.stdout:8/295: symlink d5/d50/l54 0 2024-06-03T01:20:51.526 INFO:tasks.workunit.client.1.smithi044.stdout:8/296: write d5/f15 [6289686,9348] 0 2024-06-03T01:20:51.527 INFO:tasks.workunit.client.1.smithi044.stdout:5/278: sync 2024-06-03T01:20:51.528 INFO:tasks.workunit.client.1.smithi044.stdout:0/315: sync 2024-06-03T01:20:51.528 INFO:tasks.workunit.client.1.smithi044.stdout:6/310: rename d2/d6/db to d2/d23/d30/d1e/d38/d6c 0 2024-06-03T01:20:51.530 INFO:tasks.workunit.client.1.smithi044.stdout:0/316: mkdir d3/d9/d55/d73 0 2024-06-03T01:20:51.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/279: dread d2/dc/f15 [0,4194304] 0 2024-06-03T01:20:51.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/280: write d2/d11/d12/d2b/d4a/d57/f50 [974218,121118] 0 2024-06-03T01:20:51.533 INFO:tasks.workunit.client.1.smithi044.stdout:1/285: rename d0/d3/d20/d61 to d0/d16/d64 0 2024-06-03T01:20:51.535 INFO:tasks.workunit.client.1.smithi044.stdout:6/311: creat d2/d23/d55/d5f/d53/d5a/f6d x:0 0 0 2024-06-03T01:20:51.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/233: dwrite d6/d38/d42/db/f32 [4194304,4194304] 0 2024-06-03T01:20:51.536 INFO:tasks.workunit.client.1.smithi044.stdout:8/297: mkdir d5/d55 0 2024-06-03T01:20:51.537 INFO:tasks.workunit.client.1.smithi044.stdout:9/234: dread d6/d38/d42/d16/f3f [0,4194304] 0 2024-06-03T01:20:51.537 INFO:tasks.workunit.client.1.smithi044.stdout:9/235: fsync d6/d38/d42/db/f32 0 2024-06-03T01:20:51.538 INFO:tasks.workunit.client.1.smithi044.stdout:5/281: truncate d2/f3c 438393 0 2024-06-03T01:20:51.538 INFO:tasks.workunit.client.1.smithi044.stdout:5/282: write d2/d11/d12/d2b/d44/f5c [951357,71613] 0 2024-06-03T01:20:51.539 INFO:tasks.workunit.client.1.smithi044.stdout:8/298: dread d5/d16/f3c [0,4194304] 0 2024-06-03T01:20:51.540 INFO:tasks.workunit.client.1.smithi044.stdout:0/317: rename d3/d49/d20/f63 to d3/d6b/f74 0 2024-06-03T01:20:51.540 INFO:tasks.workunit.client.1.smithi044.stdout:0/318: write d3/d9/d55/d5a/f34 [442093,125764] 0 2024-06-03T01:20:51.540 INFO:tasks.workunit.client.1.smithi044.stdout:1/286: creat d0/d3/d20/d7/d42/d52/f65 x:0 0 0 2024-06-03T01:20:51.543 INFO:tasks.workunit.client.1.smithi044.stdout:6/312: creat d2/d23/d55/d5f/f6e x:0 0 0 2024-06-03T01:20:51.543 INFO:tasks.workunit.client.1.smithi044.stdout:6/313: read - d2/d23/d30/f45 zero size 2024-06-03T01:20:51.543 INFO:tasks.workunit.client.1.smithi044.stdout:6/314: write d2/d23/d55/d5f/d53/f61 [70830,18768] 0 2024-06-03T01:20:51.543 INFO:tasks.workunit.client.1.smithi044.stdout:6/315: write d2/d23/d55/d5f/d53/f61 [867018,115459] 0 2024-06-03T01:20:51.547 INFO:tasks.workunit.client.1.smithi044.stdout:9/236: truncate d6/d38/d42/d16/f1b 3842239 0 2024-06-03T01:20:51.548 INFO:tasks.workunit.client.1.smithi044.stdout:8/299: stat d5/d20/d2b/f30 0 2024-06-03T01:20:51.550 INFO:tasks.workunit.client.1.smithi044.stdout:6/316: rmdir d2/d23/d55 39 2024-06-03T01:20:51.557 INFO:tasks.workunit.client.1.smithi044.stdout:6/317: rename d2/d23/d30/d1e/d34/d39/f42 to d2/d23/d55/f6f 0 2024-06-03T01:20:51.560 INFO:tasks.workunit.client.1.smithi044.stdout:8/300: rmdir d5/d20 39 2024-06-03T01:20:51.561 INFO:tasks.workunit.client.1.smithi044.stdout:6/318: mkdir d2/d23/d30/d1e/d34/d39/d64/d70 0 2024-06-03T01:20:51.561 INFO:tasks.workunit.client.1.smithi044.stdout:5/283: dwrite d2/f8 [4194304,4194304] 0 2024-06-03T01:20:51.562 INFO:tasks.workunit.client.1.smithi044.stdout:5/284: read - d2/dc/d28/f2c zero size 2024-06-03T01:20:51.562 INFO:tasks.workunit.client.1.smithi044.stdout:5/285: creat d2/d11/d12/d2b/d44/f5f x:0 0 0 2024-06-03T01:20:51.563 INFO:tasks.workunit.client.1.smithi044.stdout:8/301: truncate d5/d16/d1d/f27 1231422 0 2024-06-03T01:20:51.563 INFO:tasks.workunit.client.1.smithi044.stdout:8/302: chown d5/d55 353854670 1 2024-06-03T01:20:51.564 INFO:tasks.workunit.client.1.smithi044.stdout:1/287: dwrite d0/d3/d20/d7/d2f/f59 [0,4194304] 0 2024-06-03T01:20:51.564 INFO:tasks.workunit.client.1.smithi044.stdout:1/288: fdatasync d0/d3/f62 0 2024-06-03T01:20:51.567 INFO:tasks.workunit.client.1.smithi044.stdout:0/319: dwrite d3/d9/f53 [0,4194304] 0 2024-06-03T01:20:51.568 INFO:tasks.workunit.client.1.smithi044.stdout:8/303: mkdir d5/d55/d56 0 2024-06-03T01:20:51.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/237: dwrite d6/d38/d42/d16/f3f [0,4194304] 0 2024-06-03T01:20:51.573 INFO:tasks.workunit.client.1.smithi044.stdout:6/319: dread d2/d23/d55/d5f/f63 [0,4194304] 0 2024-06-03T01:20:51.573 INFO:tasks.workunit.client.1.smithi044.stdout:6/320: fdatasync d2/d23/d30/d1e/d38/d6c/f3b 0 2024-06-03T01:20:51.573 INFO:tasks.workunit.client.1.smithi044.stdout:6/321: chown d2/d23/f33 188 1 2024-06-03T01:20:51.574 INFO:tasks.workunit.client.1.smithi044.stdout:8/304: creat d5/d50/f57 x:0 0 0 2024-06-03T01:20:51.574 INFO:tasks.workunit.client.1.smithi044.stdout:8/305: stat d5/d10/l36 0 2024-06-03T01:20:51.576 INFO:tasks.workunit.client.1.smithi044.stdout:0/320: write d3/d49/d21/f26 [1060009,4690] 0 2024-06-03T01:20:51.576 INFO:tasks.workunit.client.1.smithi044.stdout:0/321: fdatasync d3/d6b/f74 0 2024-06-03T01:20:51.578 INFO:tasks.workunit.client.1.smithi044.stdout:9/238: symlink d6/l49 0 2024-06-03T01:20:51.581 INFO:tasks.workunit.client.1.smithi044.stdout:6/322: rmdir d2/d23/d30/d1e/d38/d6c 39 2024-06-03T01:20:51.581 INFO:tasks.workunit.client.1.smithi044.stdout:6/323: readlink d2/d4/l4a 0 2024-06-03T01:20:51.581 INFO:tasks.workunit.client.1.smithi044.stdout:6/324: truncate d2/d23/d30/f3a 1235539 0 2024-06-03T01:20:51.583 INFO:tasks.workunit.client.1.smithi044.stdout:0/322: creat d3/d9/d55/f75 x:0 0 0 2024-06-03T01:20:51.584 INFO:tasks.workunit.client.1.smithi044.stdout:9/239: creat d6/d26/f4a x:0 0 0 2024-06-03T01:20:51.586 INFO:tasks.workunit.client.1.smithi044.stdout:8/306: mkdir d5/d20/d2b/d58 0 2024-06-03T01:20:51.588 INFO:tasks.workunit.client.1.smithi044.stdout:0/323: creat d3/d49/d21/f76 x:0 0 0 2024-06-03T01:20:51.588 INFO:tasks.workunit.client.1.smithi044.stdout:0/324: truncate d3/d49/d21/f76 945646 0 2024-06-03T01:20:51.588 INFO:tasks.workunit.client.1.smithi044.stdout:0/325: readlink d3/d9/d55/d5a/d29/l31 0 2024-06-03T01:20:51.589 INFO:tasks.workunit.client.1.smithi044.stdout:0/326: truncate d3/d9/f13 5574789 0 2024-06-03T01:20:51.589 INFO:tasks.workunit.client.1.smithi044.stdout:0/327: write d3/d9/d36/f48 [780466,106801] 0 2024-06-03T01:20:51.590 INFO:tasks.workunit.client.1.smithi044.stdout:1/289: rename d0/d3/d20/d7/d42/d52 to d0/d66 0 2024-06-03T01:20:51.592 INFO:tasks.workunit.client.1.smithi044.stdout:8/307: mkdir d5/d16/d1d/d59 0 2024-06-03T01:20:51.592 INFO:tasks.workunit.client.1.smithi044.stdout:8/308: chown d5/d16/f2e 12 1 2024-06-03T01:20:51.594 INFO:tasks.workunit.client.1.smithi044.stdout:0/328: creat d3/d9/d55/d73/f77 x:0 0 0 2024-06-03T01:20:51.594 INFO:tasks.workunit.client.1.smithi044.stdout:0/329: stat d3/d9/d55/d5a/f34 0 2024-06-03T01:20:51.595 INFO:tasks.workunit.client.1.smithi044.stdout:8/309: mknod d5/d20/d2b/d58/c5a 0 2024-06-03T01:20:51.595 INFO:tasks.workunit.client.1.smithi044.stdout:8/310: chown d5/d10/f13 15906727 1 2024-06-03T01:20:51.595 INFO:tasks.workunit.client.1.smithi044.stdout:0/330: truncate d3/d49/d20/d25/f64 1735386 0 2024-06-03T01:20:51.596 INFO:tasks.workunit.client.1.smithi044.stdout:8/311: mknod d5/d20/d47/c5b 0 2024-06-03T01:20:51.597 INFO:tasks.workunit.client.1.smithi044.stdout:8/312: fsync d5/d20/d2b/f48 0 2024-06-03T01:20:51.599 INFO:tasks.workunit.client.1.smithi044.stdout:1/290: dread d0/d3/d20/d7/d42/d47/f4c [0,4194304] 0 2024-06-03T01:20:51.599 INFO:tasks.workunit.client.1.smithi044.stdout:1/291: creat d0/d16/f67 x:0 0 0 2024-06-03T01:20:51.605 INFO:tasks.workunit.client.1.smithi044.stdout:9/240: dwrite d6/d38/d42/d16/f3f [4194304,4194304] 0 2024-06-03T01:20:51.606 INFO:tasks.workunit.client.1.smithi044.stdout:9/241: chown d6/d38/d42/l35 50 1 2024-06-03T01:20:51.607 INFO:tasks.workunit.client.1.smithi044.stdout:9/242: read f4 [5991986,49440] 0 2024-06-03T01:20:51.609 INFO:tasks.workunit.client.1.smithi044.stdout:8/313: dread d5/ff [0,4194304] 0 2024-06-03T01:20:51.613 INFO:tasks.workunit.client.1.smithi044.stdout:9/243: mkdir d6/d38/d42/d4b 0 2024-06-03T01:20:51.615 INFO:tasks.workunit.client.1.smithi044.stdout:8/314: unlink d5/lc 0 2024-06-03T01:20:51.616 INFO:tasks.workunit.client.1.smithi044.stdout:8/315: chown d5/d20/f3d 44 1 2024-06-03T01:20:51.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/244: dread d6/f11 [0,4194304] 0 2024-06-03T01:20:51.621 INFO:tasks.workunit.client.1.smithi044.stdout:9/245: mknod d6/d38/d42/c4c 0 2024-06-03T01:20:51.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/331: dwrite d3/d9/d55/d5a/f3f [4194304,4194304] 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:1/292: dwrite d0/d3/d14/f5e [0,4194304] 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/246: rename d6/d38/d42/d4b to d6/d38/d42/d16/d4d 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/247: creat d6/d38/d42/d46/f4e x:0 0 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/248: truncate d6/d38/d42/d16/f37 704078 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/249: creat d6/d26/f4f x:0 0 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/250: write d6/d38/d42/d16/d27/f39 [76240,24373] 0 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/251: chown d6/l1e 13332 1 2024-06-03T01:20:51.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/252: dread - d6/d38/d42/d16/f34 zero size 2024-06-03T01:20:51.626 INFO:tasks.workunit.client.1.smithi044.stdout:0/332: truncate d3/d9/f1d 3731341 0 2024-06-03T01:20:51.626 INFO:tasks.workunit.client.1.smithi044.stdout:9/253: read d6/d38/d42/db/f22 [163798,118835] 0 2024-06-03T01:20:51.630 INFO:tasks.workunit.client.1.smithi044.stdout:0/333: unlink d3/d49/d20/d25/f67 0 2024-06-03T01:20:51.630 INFO:tasks.workunit.client.1.smithi044.stdout:0/334: chown d3/d5c 355 1 2024-06-03T01:20:51.631 INFO:tasks.workunit.client.1.smithi044.stdout:9/254: truncate d6/f12 1014583 0 2024-06-03T01:20:51.633 INFO:tasks.workunit.client.1.smithi044.stdout:8/316: dwrite d5/d20/f3d [0,4194304] 0 2024-06-03T01:20:51.633 INFO:tasks.workunit.client.1.smithi044.stdout:9/255: truncate d6/dd/f3d 2277386 0 2024-06-03T01:20:51.633 INFO:tasks.workunit.client.1.smithi044.stdout:9/256: write d6/d38/d42/d16/d27/f39 [572916,48391] 0 2024-06-03T01:20:51.635 INFO:tasks.workunit.client.1.smithi044.stdout:8/317: mkdir d5/d55/d56/d5c 0 2024-06-03T01:20:51.635 INFO:tasks.workunit.client.1.smithi044.stdout:0/335: dread d3/d9/d55/d5a/f34 [0,4194304] 0 2024-06-03T01:20:51.643 INFO:tasks.workunit.client.1.smithi044.stdout:0/336: write d3/f8 [7596776,1420] 0 2024-06-03T01:20:51.643 INFO:tasks.workunit.client.1.smithi044.stdout:0/337: creat d3/d49/d20/f78 x:0 0 0 2024-06-03T01:20:51.647 INFO:tasks.workunit.client.1.smithi044.stdout:0/338: rename d3/d6b/f74 to d3/d9/d55/f79 0 2024-06-03T01:20:51.648 INFO:tasks.workunit.client.1.smithi044.stdout:0/339: symlink d3/d9/d55/d73/l7a 0 2024-06-03T01:20:51.648 INFO:tasks.workunit.client.1.smithi044.stdout:0/340: readlink d3/d9/l1e 0 2024-06-03T01:20:51.648 INFO:tasks.workunit.client.1.smithi044.stdout:0/341: dread - d3/f6e zero size 2024-06-03T01:20:51.648 INFO:tasks.workunit.client.1.smithi044.stdout:0/342: dread - d3/d9/d36/f57 zero size 2024-06-03T01:20:51.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/343: unlink d3/d9/d55/d5a/d29/l52 0 2024-06-03T01:20:51.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/344: chown d3/d49/d56/f5d 63 1 2024-06-03T01:20:51.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/345: write d3/d9/d55/d5a/f38 [389664,61126] 0 2024-06-03T01:20:51.650 INFO:tasks.workunit.client.1.smithi044.stdout:0/346: mkdir d3/d9/d55/d7b 0 2024-06-03T01:20:51.651 INFO:tasks.workunit.client.1.smithi044.stdout:0/347: mknod d3/d9/d55/d5a/d29/c7c 0 2024-06-03T01:20:51.651 INFO:tasks.workunit.client.1.smithi044.stdout:0/348: dread - d3/d9/f1f zero size 2024-06-03T01:20:51.652 INFO:tasks.workunit.client.1.smithi044.stdout:0/349: creat d3/d49/d20/d25/f7d x:0 0 0 2024-06-03T01:20:51.652 INFO:tasks.workunit.client.1.smithi044.stdout:0/350: symlink d3/d49/l7e 0 2024-06-03T01:20:51.659 INFO:tasks.workunit.client.1.smithi044.stdout:8/318: dwrite d5/d16/f3c [0,4194304] 0 2024-06-03T01:20:51.663 INFO:tasks.workunit.client.1.smithi044.stdout:0/351: dread d3/d9/d55/d5a/f3f [4194304,4194304] 0 2024-06-03T01:20:51.664 INFO:tasks.workunit.client.1.smithi044.stdout:0/352: mknod d3/d9/d55/d5a/d29/d58/c7f 0 2024-06-03T01:20:51.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/319: dread d5/d16/d1d/f27 [0,4194304] 0 2024-06-03T01:20:51.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/320: write d5/d16/f3a [747394,57103] 0 2024-06-03T01:20:51.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/321: fsync d5/d20/d33/f4c 0 2024-06-03T01:20:51.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/322: write f3 [3277054,6146] 0 2024-06-03T01:20:51.665 INFO:tasks.workunit.client.1.smithi044.stdout:0/353: symlink d3/d49/d56/d62/l80 0 2024-06-03T01:20:51.667 INFO:tasks.workunit.client.1.smithi044.stdout:8/323: creat d5/d55/f5d x:0 0 0 2024-06-03T01:20:51.669 INFO:tasks.workunit.client.1.smithi044.stdout:8/324: unlink d5/d16/d1d/f27 0 2024-06-03T01:20:51.670 INFO:tasks.workunit.client.1.smithi044.stdout:8/325: chown d5/la 7 1 2024-06-03T01:20:51.670 INFO:tasks.workunit.client.1.smithi044.stdout:8/326: stat d5/d55/d56/d5c 0 2024-06-03T01:20:51.670 INFO:tasks.workunit.client.1.smithi044.stdout:8/327: creat d5/d16/f5e x:0 0 0 2024-06-03T01:20:51.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/328: chown d5/d16/d1d/f42 167 1 2024-06-03T01:20:51.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/329: fdatasync d5/d20/d33/f3f 0 2024-06-03T01:20:51.673 INFO:tasks.workunit.client.1.smithi044.stdout:8/330: write d5/f15 [7109747,73340] 0 2024-06-03T01:20:51.673 INFO:tasks.workunit.client.1.smithi044.stdout:8/331: dread - d5/d20/f4a zero size 2024-06-03T01:20:51.674 INFO:tasks.workunit.client.1.smithi044.stdout:7/399: sync 2024-06-03T01:20:51.674 INFO:tasks.workunit.client.1.smithi044.stdout:3/170: sync 2024-06-03T01:20:51.674 INFO:tasks.workunit.client.1.smithi044.stdout:2/207: sync 2024-06-03T01:20:51.674 INFO:tasks.workunit.client.1.smithi044.stdout:4/238: sync 2024-06-03T01:20:51.674 INFO:tasks.workunit.client.1.smithi044.stdout:3/171: write df/f2d [110204,76860] 0 2024-06-03T01:20:51.675 INFO:tasks.workunit.client.1.smithi044.stdout:8/332: mkdir d5/d10/d5f 0 2024-06-03T01:20:51.675 INFO:tasks.workunit.client.1.smithi044.stdout:7/400: mknod d0/d3/d8/d32/c8c 0 2024-06-03T01:20:51.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/208: mkdir db/d1b/d3f/d47 0 2024-06-03T01:20:51.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/209: read db/f12 [1193814,121978] 0 2024-06-03T01:20:51.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/210: write f2 [4145047,2573] 0 2024-06-03T01:20:51.676 INFO:tasks.workunit.client.1.smithi044.stdout:3/172: getdents df/d1e 0 2024-06-03T01:20:51.676 INFO:tasks.workunit.client.1.smithi044.stdout:8/333: mknod d5/d10/c60 0 2024-06-03T01:20:51.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/401: truncate d0/d3/fa 9322952 0 2024-06-03T01:20:51.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/173: link df/f23 df/d13/f2e 0 2024-06-03T01:20:51.681 INFO:tasks.workunit.client.1.smithi044.stdout:8/334: unlink d5/d10/f46 0 2024-06-03T01:20:51.682 INFO:tasks.workunit.client.1.smithi044.stdout:7/402: mknod d0/d3/d8/d32/c8d 0 2024-06-03T01:20:51.682 INFO:tasks.workunit.client.1.smithi044.stdout:7/403: read - d0/d3/d30/f79 zero size 2024-06-03T01:20:51.683 INFO:tasks.workunit.client.1.smithi044.stdout:3/174: link df/d13/f24 df/f2f 0 2024-06-03T01:20:51.685 INFO:tasks.workunit.client.1.smithi044.stdout:8/335: unlink d5/d20/d2b/c4e 0 2024-06-03T01:20:51.685 INFO:tasks.workunit.client.1.smithi044.stdout:8/336: chown d5/d16/d1d/d32/l4f 10097580 1 2024-06-03T01:20:51.688 INFO:tasks.workunit.client.1.smithi044.stdout:0/354: dwrite d3/f17 [4194304,4194304] 0 2024-06-03T01:20:51.688 INFO:tasks.workunit.client.1.smithi044.stdout:3/175: creat df/d1e/d2a/f30 x:0 0 0 2024-06-03T01:20:51.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/176: creat df/f31 x:0 0 0 2024-06-03T01:20:51.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/177: creat df/f32 x:0 0 0 2024-06-03T01:20:51.689 INFO:tasks.workunit.client.1.smithi044.stdout:8/337: symlink d5/d10/d5f/l61 0 2024-06-03T01:20:51.689 INFO:tasks.workunit.client.1.smithi044.stdout:8/338: creat d5/d16/d1d/d32/f62 x:0 0 0 2024-06-03T01:20:51.690 INFO:tasks.workunit.client.1.smithi044.stdout:7/404: rmdir d0/d21/d63/d7f 39 2024-06-03T01:20:51.690 INFO:tasks.workunit.client.1.smithi044.stdout:7/405: chown d0/d21/l44 9986215 1 2024-06-03T01:20:51.692 INFO:tasks.workunit.client.1.smithi044.stdout:0/355: dread d3/d9/f13 [0,4194304] 0 2024-06-03T01:20:51.692 INFO:tasks.workunit.client.1.smithi044.stdout:2/211: dread db/d1b/d27/f32 [4194304,4194304] 0 2024-06-03T01:20:51.693 INFO:tasks.workunit.client.1.smithi044.stdout:3/178: creat df/d13/d1b/f33 x:0 0 0 2024-06-03T01:20:51.693 INFO:tasks.workunit.client.1.smithi044.stdout:3/179: write df/d13/f2c [975369,83073] 0 2024-06-03T01:20:51.693 INFO:tasks.workunit.client.1.smithi044.stdout:3/180: creat df/d13/d1b/f34 x:0 0 0 2024-06-03T01:20:51.696 INFO:tasks.workunit.client.1.smithi044.stdout:7/406: rmdir d0/d21/d27 39 2024-06-03T01:20:51.701 INFO:tasks.workunit.client.1.smithi044.stdout:3/181: creat df/d1e/f35 x:0 0 0 2024-06-03T01:20:51.701 INFO:tasks.workunit.client.1.smithi044.stdout:3/182: fdatasync df/f2d 0 2024-06-03T01:20:51.701 INFO:tasks.workunit.client.1.smithi044.stdout:4/239: dwrite d17/d23/d28/d2b/f1b [0,4194304] 0 2024-06-03T01:20:51.701 INFO:tasks.workunit.client.1.smithi044.stdout:4/240: creat d17/d1c/f52 x:0 0 0 2024-06-03T01:20:51.702 INFO:tasks.workunit.client.1.smithi044.stdout:0/356: dread d3/d49/f3b [0,4194304] 0 2024-06-03T01:20:51.702 INFO:tasks.workunit.client.1.smithi044.stdout:0/357: dread - d3/d9/d55/d73/f77 zero size 2024-06-03T01:20:51.704 INFO:tasks.workunit.client.1.smithi044.stdout:7/407: creat d0/d16/f8e x:0 0 0 2024-06-03T01:20:51.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/241: symlink d17/d23/l53 0 2024-06-03T01:20:51.722 INFO:tasks.workunit.client.1.smithi044.stdout:8/339: rename d5/f7 to d5/d20/d33/f63 0 2024-06-03T01:20:51.726 INFO:tasks.workunit.client.1.smithi044.stdout:8/340: mkdir d5/d20/d47/d64 0 2024-06-03T01:20:51.727 INFO:tasks.workunit.client.1.smithi044.stdout:3/183: dwrite df/f19 [0,4194304] 0 2024-06-03T01:20:51.728 INFO:tasks.workunit.client.1.smithi044.stdout:8/341: creat d5/d20/d33/f65 x:0 0 0 2024-06-03T01:20:51.731 INFO:tasks.workunit.client.1.smithi044.stdout:3/184: rmdir df/d1e 39 2024-06-03T01:20:51.735 INFO:tasks.workunit.client.1.smithi044.stdout:4/242: dwrite ff [0,4194304] 0 2024-06-03T01:20:51.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/408: dwrite d0/d21/d63/f6b [0,4194304] 0 2024-06-03T01:20:51.739 INFO:tasks.workunit.client.1.smithi044.stdout:4/243: rename d17/d23/f30 to d17/d23/d28/f54 0 2024-06-03T01:20:51.739 INFO:tasks.workunit.client.1.smithi044.stdout:4/244: chown d17/d23/d28/l36 11211 1 2024-06-03T01:20:51.743 INFO:tasks.workunit.client.1.smithi044.stdout:7/409: creat d0/d21/d63/f8f x:0 0 0 2024-06-03T01:20:51.743 INFO:tasks.workunit.client.1.smithi044.stdout:7/410: chown d0/d3/d8/f1b 16510810 1 2024-06-03T01:20:51.746 INFO:tasks.workunit.client.1.smithi044.stdout:0/358: dwrite d3/d9/f22 [0,4194304] 0 2024-06-03T01:20:51.751 INFO:tasks.workunit.client.1.smithi044.stdout:0/359: unlink d3/d49/d56/c6a 0 2024-06-03T01:20:51.751 INFO:tasks.workunit.client.1.smithi044.stdout:0/360: truncate d3/d9/f44 5000478 0 2024-06-03T01:20:51.752 INFO:tasks.workunit.client.1.smithi044.stdout:8/342: dwrite d5/f25 [0,4194304] 0 2024-06-03T01:20:51.752 INFO:tasks.workunit.client.1.smithi044.stdout:3/185: dwrite df/f10 [4194304,4194304] 0 2024-06-03T01:20:51.753 INFO:tasks.workunit.client.1.smithi044.stdout:0/361: creat d3/f81 x:0 0 0 2024-06-03T01:20:51.753 INFO:tasks.workunit.client.1.smithi044.stdout:0/362: dread - d3/d49/d20/d25/f7d zero size 2024-06-03T01:20:51.753 INFO:tasks.workunit.client.1.smithi044.stdout:0/363: fsync d3/d49/d56/f60 0 2024-06-03T01:20:51.754 INFO:tasks.workunit.client.1.smithi044.stdout:8/343: symlink d5/d20/d47/d64/l66 0 2024-06-03T01:20:51.754 INFO:tasks.workunit.client.1.smithi044.stdout:8/344: write d5/d20/d33/f3f [36029,88633] 0 2024-06-03T01:20:51.756 INFO:tasks.workunit.client.1.smithi044.stdout:0/364: mkdir d3/d9/d36/d82 0 2024-06-03T01:20:51.756 INFO:tasks.workunit.client.1.smithi044.stdout:6/325: sync 2024-06-03T01:20:51.756 INFO:tasks.workunit.client.1.smithi044.stdout:5/286: sync 2024-06-03T01:20:51.756 INFO:tasks.workunit.client.1.smithi044.stdout:5/287: chown d2/l5 7729 1 2024-06-03T01:20:51.756 INFO:tasks.workunit.client.1.smithi044.stdout:6/326: chown d2/d4/l40 128530 1 2024-06-03T01:20:51.757 INFO:tasks.workunit.client.1.smithi044.stdout:6/327: dread - d2/d23/d30/d1e/d34/d39/d64/f6a zero size 2024-06-03T01:20:51.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/345: write d5/d50/f57 [659958,7114] 0 2024-06-03T01:20:51.758 INFO:tasks.workunit.client.1.smithi044.stdout:0/365: creat d3/d9/d36/d82/f83 x:0 0 0 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:0/366: creat d3/d6b/f84 x:0 0 0 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:0/367: creat d3/f85 x:0 0 0 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:0/368: dread - d3/d9/d36/d82/f83 zero size 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:0/369: chown d3/d49/d21/f76 0 1 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:5/288: getdents d2/d11 0 2024-06-03T01:20:51.759 INFO:tasks.workunit.client.1.smithi044.stdout:6/328: creat d2/d23/d44/f71 x:0 0 0 2024-06-03T01:20:51.760 INFO:tasks.workunit.client.1.smithi044.stdout:6/329: dread - d2/d4/f4b zero size 2024-06-03T01:20:51.760 INFO:tasks.workunit.client.1.smithi044.stdout:7/411: dwrite d0/d21/d63/f6b [0,4194304] 0 2024-06-03T01:20:51.760 INFO:tasks.workunit.client.1.smithi044.stdout:3/186: dread df/f1c [0,4194304] 0 2024-06-03T01:20:51.763 INFO:tasks.workunit.client.1.smithi044.stdout:5/289: symlink d2/l60 0 2024-06-03T01:20:51.763 INFO:tasks.workunit.client.1.smithi044.stdout:5/290: fsync d2/f13 0 2024-06-03T01:20:51.763 INFO:tasks.workunit.client.1.smithi044.stdout:5/291: fsync d2/d11/d12/d2b/d44/f5f 0 2024-06-03T01:20:51.763 INFO:tasks.workunit.client.1.smithi044.stdout:5/292: write d2/d11/d12/f29 [1395810,19686] 0 2024-06-03T01:20:51.763 INFO:tasks.workunit.client.1.smithi044.stdout:8/346: symlink d5/d16/d1d/d59/l67 0 2024-06-03T01:20:51.769 INFO:tasks.workunit.client.1.smithi044.stdout:7/412: creat d0/d3/d8/d32/f90 x:0 0 0 2024-06-03T01:20:51.771 INFO:tasks.workunit.client.1.smithi044.stdout:3/187: link f9 df/d1e/d2a/f36 0 2024-06-03T01:20:51.772 INFO:tasks.workunit.client.1.smithi044.stdout:3/188: fdatasync df/d13/f1d 0 2024-06-03T01:20:51.772 INFO:tasks.workunit.client.1.smithi044.stdout:3/189: write fd [2025247,33537] 0 2024-06-03T01:20:51.774 INFO:tasks.workunit.client.1.smithi044.stdout:7/413: dread d0/d21/d27/f50 [0,4194304] 0 2024-06-03T01:20:51.775 INFO:tasks.workunit.client.1.smithi044.stdout:3/190: creat df/d1e/f37 x:0 0 0 2024-06-03T01:20:51.776 INFO:tasks.workunit.client.1.smithi044.stdout:3/191: creat df/d13/d1b/f38 x:0 0 0 2024-06-03T01:20:51.776 INFO:tasks.workunit.client.1.smithi044.stdout:8/347: mkdir d5/d55/d56/d5c/d68 0 2024-06-03T01:20:51.777 INFO:tasks.workunit.client.1.smithi044.stdout:0/370: dwrite d3/d9/d55/d5a/f3f [0,4194304] 0 2024-06-03T01:20:51.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/192: mkdir df/d13/d39 0 2024-06-03T01:20:51.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/193: readlink l4 0 2024-06-03T01:20:51.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/194: chown df/d13/f26 402274 1 2024-06-03T01:20:51.779 INFO:tasks.workunit.client.1.smithi044.stdout:5/293: dread d2/f8 [4194304,4194304] 0 2024-06-03T01:20:51.779 INFO:tasks.workunit.client.1.smithi044.stdout:5/294: write d2/d11/d12/d2b/d4a/d57/f50 [1951422,1070] 0 2024-06-03T01:20:51.779 INFO:tasks.workunit.client.1.smithi044.stdout:5/295: chown d2/d11/d12/l23 0 1 2024-06-03T01:20:51.780 INFO:tasks.workunit.client.1.smithi044.stdout:8/348: rename d5/d16/d1d/f23 to d5/d20/f69 0 2024-06-03T01:20:51.780 INFO:tasks.workunit.client.1.smithi044.stdout:8/349: fsync d5/d20/d2b/f30 0 2024-06-03T01:20:51.782 INFO:tasks.workunit.client.1.smithi044.stdout:0/371: rmdir d3/d49 39 2024-06-03T01:20:51.783 INFO:tasks.workunit.client.1.smithi044.stdout:3/195: mkdir df/d1e/d3a 0 2024-06-03T01:20:51.787 INFO:tasks.workunit.client.1.smithi044.stdout:8/350: creat d5/f6a x:0 0 0 2024-06-03T01:20:51.789 INFO:tasks.workunit.client.1.smithi044.stdout:6/330: dwrite d2/d23/d30/d1e/d34/d39/d64/f6a [0,4194304] 0 2024-06-03T01:20:51.789 INFO:tasks.workunit.client.1.smithi044.stdout:0/372: rename d3/d9/d55/f75 to d3/d9/f86 0 2024-06-03T01:20:51.792 INFO:tasks.workunit.client.1.smithi044.stdout:8/351: mkdir d5/d16/d1d/d6b 0 2024-06-03T01:20:51.792 INFO:tasks.workunit.client.1.smithi044.stdout:8/352: fsync d5/d20/d47/f49 0 2024-06-03T01:20:51.792 INFO:tasks.workunit.client.1.smithi044.stdout:7/414: dwrite d0/d3/f18 [0,4194304] 0 2024-06-03T01:20:51.795 INFO:tasks.workunit.client.1.smithi044.stdout:1/293: sync 2024-06-03T01:20:51.795 INFO:tasks.workunit.client.1.smithi044.stdout:9/257: sync 2024-06-03T01:20:51.796 INFO:tasks.workunit.client.1.smithi044.stdout:9/258: chown d6/l49 22246 1 2024-06-03T01:20:51.796 INFO:tasks.workunit.client.1.smithi044.stdout:6/331: getdents d2/d23/d30/d1e/d38/d6c/d5c 0 2024-06-03T01:20:51.797 INFO:tasks.workunit.client.1.smithi044.stdout:8/353: link d5/d16/l1e d5/d16/l6c 0 2024-06-03T01:20:51.798 INFO:tasks.workunit.client.1.smithi044.stdout:1/294: dread - d0/f3f zero size 2024-06-03T01:20:51.799 INFO:tasks.workunit.client.1.smithi044.stdout:6/332: truncate d2/d23/d30/f3a 97873 0 2024-06-03T01:20:51.799 INFO:tasks.workunit.client.1.smithi044.stdout:1/295: chown d0/l10 6362661 1 2024-06-03T01:20:51.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/373: dread d3/d9/d36/f42 [0,4194304] 0 2024-06-03T01:20:51.800 INFO:tasks.workunit.client.1.smithi044.stdout:6/333: creat d2/d23/d30/d1e/d38/f72 x:0 0 0 2024-06-03T01:20:51.801 INFO:tasks.workunit.client.1.smithi044.stdout:1/296: creat d0/d3/d20/d7/d2f/f68 x:0 0 0 2024-06-03T01:20:51.801 INFO:tasks.workunit.client.1.smithi044.stdout:1/297: creat d0/d3/d20/d7/d42/f69 x:0 0 0 2024-06-03T01:20:51.801 INFO:tasks.workunit.client.1.smithi044.stdout:3/196: dwrite df/d1e/f37 [0,4194304] 0 2024-06-03T01:20:51.802 INFO:tasks.workunit.client.1.smithi044.stdout:0/374: symlink d3/d9/d36/l87 0 2024-06-03T01:20:51.806 INFO:tasks.workunit.client.1.smithi044.stdout:0/375: symlink d3/d9/l88 0 2024-06-03T01:20:51.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/354: dread d5/d20/f69 [0,4194304] 0 2024-06-03T01:20:51.809 INFO:tasks.workunit.client.1.smithi044.stdout:0/376: unlink d3/c7 0 2024-06-03T01:20:51.810 INFO:tasks.workunit.client.1.smithi044.stdout:1/298: dread d0/d3/d20/d7/d42/d47/f4c [0,4194304] 0 2024-06-03T01:20:51.811 INFO:tasks.workunit.client.1.smithi044.stdout:3/197: fsync df/d13/d1b/f38 0 2024-06-03T01:20:51.811 INFO:tasks.workunit.client.1.smithi044.stdout:3/198: readlink lb 0 2024-06-03T01:20:51.813 INFO:tasks.workunit.client.1.smithi044.stdout:1/299: mkdir d0/d16/d64/d6a 0 2024-06-03T01:20:51.813 INFO:tasks.workunit.client.1.smithi044.stdout:1/300: read - d0/d3/d20/d40/d45/d4b/d5c/f63 zero size 2024-06-03T01:20:51.815 INFO:tasks.workunit.client.1.smithi044.stdout:1/301: mkdir d0/d3/d20/d40/d45/d6b 0 2024-06-03T01:20:51.816 INFO:tasks.workunit.client.1.smithi044.stdout:6/334: dwrite d2/d23/f35 [0,4194304] 0 2024-06-03T01:20:51.824 INFO:tasks.workunit.client.1.smithi044.stdout:8/355: read d5/d10/f34 [38963,67639] 0 2024-06-03T01:20:51.826 INFO:tasks.workunit.client.1.smithi044.stdout:8/356: mkdir d5/d16/d1d/d6b/d6d 0 2024-06-03T01:20:51.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/335: dread d2/d23/d30/f2a [0,4194304] 0 2024-06-03T01:20:51.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/336: fdatasync d2/d23/d30/f19 0 2024-06-03T01:20:51.828 INFO:tasks.workunit.client.1.smithi044.stdout:6/337: creat d2/d23/d55/d5f/d53/f73 x:0 0 0 2024-06-03T01:20:51.828 INFO:tasks.workunit.client.1.smithi044.stdout:6/338: read d2/d23/f33 [241482,112241] 0 2024-06-03T01:20:51.828 INFO:tasks.workunit.client.1.smithi044.stdout:0/377: getdents d3/d9/d36 0 2024-06-03T01:20:51.829 INFO:tasks.workunit.client.1.smithi044.stdout:8/357: getdents d5/d16/d1d 0 2024-06-03T01:20:51.829 INFO:tasks.workunit.client.1.smithi044.stdout:8/358: write f3 [3969239,38935] 0 2024-06-03T01:20:51.829 INFO:tasks.workunit.client.1.smithi044.stdout:8/359: stat d5/d16/l41 0 2024-06-03T01:20:51.830 INFO:tasks.workunit.client.1.smithi044.stdout:1/302: dwrite d0/d3/d20/d7/d13/f3d [0,4194304] 0 2024-06-03T01:20:51.830 INFO:tasks.workunit.client.1.smithi044.stdout:6/339: creat d2/d23/d30/f74 x:0 0 0 2024-06-03T01:20:51.830 INFO:tasks.workunit.client.1.smithi044.stdout:6/340: chown d2/c3 106 1 2024-06-03T01:20:51.831 INFO:tasks.workunit.client.1.smithi044.stdout:0/378: rmdir d3/d9/d36 39 2024-06-03T01:20:51.835 INFO:tasks.workunit.client.1.smithi044.stdout:1/303: creat d0/f6c x:0 0 0 2024-06-03T01:20:51.835 INFO:tasks.workunit.client.1.smithi044.stdout:6/341: truncate d2/d23/d30/f2a 2006558 0 2024-06-03T01:20:51.836 INFO:tasks.workunit.client.1.smithi044.stdout:1/304: creat d0/d3/d20/f6d x:0 0 0 2024-06-03T01:20:51.836 INFO:tasks.workunit.client.1.smithi044.stdout:6/342: mknod d2/d23/d44/c75 0 2024-06-03T01:20:51.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/360: dread d5/d20/d2b/f48 [0,4194304] 0 2024-06-03T01:20:51.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/361: getdents d5/d16/d1d 0 2024-06-03T01:20:51.838 INFO:tasks.workunit.client.1.smithi044.stdout:6/343: symlink d2/d23/d30/l76 0 2024-06-03T01:20:51.845 INFO:tasks.workunit.client.1.smithi044.stdout:6/344: write d2/d23/d30/d1e/d34/d39/f56 [3845491,129096] 0 2024-06-03T01:20:51.861 INFO:tasks.workunit.client.1.smithi044.stdout:6/345: dwrite d2/f2c [0,4194304] 0 2024-06-03T01:20:51.861 INFO:tasks.workunit.client.1.smithi044.stdout:6/346: chown d2/d23/d55/d5f/f62 5400 1 2024-06-03T01:20:51.862 INFO:tasks.workunit.client.1.smithi044.stdout:6/347: mkdir d2/d23/d30/d1e/d38/d77 0 2024-06-03T01:20:51.862 INFO:tasks.workunit.client.1.smithi044.stdout:6/348: creat d2/d23/d30/d1e/d38/f78 x:0 0 0 2024-06-03T01:20:51.863 INFO:tasks.workunit.client.1.smithi044.stdout:6/349: mkdir d2/d23/d30/d1e/d38/d6c/d5c/d79 0 2024-06-03T01:20:51.863 INFO:tasks.workunit.client.1.smithi044.stdout:6/350: read d2/d23/d55/f6f [2705406,10688] 0 2024-06-03T01:20:51.864 INFO:tasks.workunit.client.1.smithi044.stdout:6/351: mknod d2/d23/d55/d5f/c7a 0 2024-06-03T01:20:51.864 INFO:tasks.workunit.client.1.smithi044.stdout:6/352: fdatasync d2/d23/d55/d5f/d53/f73 0 2024-06-03T01:20:51.865 INFO:tasks.workunit.client.1.smithi044.stdout:6/353: rename d2/d23/d30/d1e/d34/c52 to d2/d23/d44/c7b 0 2024-06-03T01:20:51.865 INFO:tasks.workunit.client.1.smithi044.stdout:6/354: fsync d2/d23/d30/d1e/d38/d6c/f3b 0 2024-06-03T01:20:51.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/355: chown d2/d23/d44/c7b 240 1 2024-06-03T01:20:51.867 INFO:tasks.workunit.client.1.smithi044.stdout:6/356: link d2/d23/d55/d5f/d53/f73 d2/d23/d30/d1e/d38/d77/f7c 0 2024-06-03T01:20:51.868 INFO:tasks.workunit.client.1.smithi044.stdout:6/357: mkdir d2/d23/d30/d1e/d34/d39/d7d 0 2024-06-03T01:20:51.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/358: dread d2/d23/d30/f1f [0,4194304] 0 2024-06-03T01:20:51.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/359: getdents d2/d23/d30/d1e 0 2024-06-03T01:20:51.878 INFO:tasks.workunit.client.1.smithi044.stdout:0/379: write d3/f17 [4799442,53653] 0 2024-06-03T01:20:51.878 INFO:tasks.workunit.client.1.smithi044.stdout:0/380: stat d3/d49/d21/f76 0 2024-06-03T01:20:51.878 INFO:tasks.workunit.client.1.smithi044.stdout:3/199: dwrite df/d13/f2c [0,4194304] 0 2024-06-03T01:20:51.878 INFO:tasks.workunit.client.1.smithi044.stdout:3/200: chown df/d1e/f35 4 1 2024-06-03T01:20:51.878 INFO:tasks.workunit.client.1.smithi044.stdout:8/362: dwrite d5/d20/d2b/f35 [0,4194304] 0 2024-06-03T01:20:51.879 INFO:tasks.workunit.client.1.smithi044.stdout:8/363: creat d5/d16/f6e x:0 0 0 2024-06-03T01:20:51.879 INFO:tasks.workunit.client.1.smithi044.stdout:8/364: dread - d5/d16/f5e zero size 2024-06-03T01:20:51.883 INFO:tasks.workunit.client.1.smithi044.stdout:3/201: unlink df/f2d 0 2024-06-03T01:20:51.883 INFO:tasks.workunit.client.1.smithi044.stdout:0/381: dread d3/d49/f3e [0,4194304] 0 2024-06-03T01:20:51.884 INFO:tasks.workunit.client.1.smithi044.stdout:6/360: write d2/f2c [2210004,45053] 0 2024-06-03T01:20:51.884 INFO:tasks.workunit.client.1.smithi044.stdout:6/361: write d2/d23/d30/d1e/d34/d39/f3c [4351843,70001] 0 2024-06-03T01:20:51.884 INFO:tasks.workunit.client.1.smithi044.stdout:6/362: write d2/d23/d44/f54 [350483,63745] 0 2024-06-03T01:20:51.884 INFO:tasks.workunit.client.1.smithi044.stdout:6/363: creat d2/d23/d55/d5f/d53/f7e x:0 0 0 2024-06-03T01:20:51.890 INFO:tasks.workunit.client.1.smithi044.stdout:3/202: creat df/d13/f3b x:0 0 0 2024-06-03T01:20:51.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/364: rename d2/d4/f4b to d2/d23/d30/f7f 0 2024-06-03T01:20:51.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/365: truncate d2/d23/d55/d5f/d53/f61 1453031 0 2024-06-03T01:20:51.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/366: readlink d2/d23/d30/d1e/d38/d6c/l1b 0 2024-06-03T01:20:51.895 INFO:tasks.workunit.client.1.smithi044.stdout:3/203: mknod df/c3c 0 2024-06-03T01:20:51.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/382: dwrite d3/d49/d20/f71 [0,4194304] 0 2024-06-03T01:20:51.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/383: write d3/f4 [1854610,63580] 0 2024-06-03T01:20:51.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/367: mknod d2/d4/c80 0 2024-06-03T01:20:51.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/368: readlink d2/d23/d30/l76 0 2024-06-03T01:20:51.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/369: chown d2/d23/f18 0 1 2024-06-03T01:20:51.901 INFO:tasks.workunit.client.1.smithi044.stdout:0/384: dread d3/d49/d20/d25/f4b [0,4194304] 0 2024-06-03T01:20:51.906 INFO:tasks.workunit.client.1.smithi044.stdout:2/212: sync 2024-06-03T01:20:51.910 INFO:tasks.workunit.client.1.smithi044.stdout:2/213: rename db/d1b/d2a/f2d to db/f48 0 2024-06-03T01:20:51.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/204: dwrite df/d13/f29 [0,4194304] 0 2024-06-03T01:20:51.918 INFO:tasks.workunit.client.1.smithi044.stdout:6/370: dwrite d2/d23/d30/d1e/d38/d77/f7c [0,4194304] 0 2024-06-03T01:20:51.918 INFO:tasks.workunit.client.1.smithi044.stdout:6/371: readlink d2/d23/d30/l76 0 2024-06-03T01:20:51.918 INFO:tasks.workunit.client.1.smithi044.stdout:6/372: fsync d2/d23/d30/d1e/f66 0 2024-06-03T01:20:51.920 INFO:tasks.workunit.client.1.smithi044.stdout:6/373: getdents d2/d23/d55 0 2024-06-03T01:20:51.920 INFO:tasks.workunit.client.1.smithi044.stdout:3/205: rmdir df/d13/d1b 39 2024-06-03T01:20:51.921 INFO:tasks.workunit.client.1.smithi044.stdout:2/214: write db/d1b/d3f/f41 [2897218,73480] 0 2024-06-03T01:20:51.922 INFO:tasks.workunit.client.1.smithi044.stdout:6/374: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/f81 x:0 0 0 2024-06-03T01:20:51.922 INFO:tasks.workunit.client.1.smithi044.stdout:6/375: chown d2/f27 1 1 2024-06-03T01:20:51.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/376: read d2/d23/d30/f1a [9832,122616] 0 2024-06-03T01:20:51.924 INFO:tasks.workunit.client.1.smithi044.stdout:0/385: dwrite d3/d9/f22 [0,4194304] 0 2024-06-03T01:20:51.925 INFO:tasks.workunit.client.1.smithi044.stdout:0/386: symlink d3/d9/d55/d5a/d29/d58/l89 0 2024-06-03T01:20:51.925 INFO:tasks.workunit.client.1.smithi044.stdout:0/387: fsync d3/d49/d20/f5b 0 2024-06-03T01:20:51.927 INFO:tasks.workunit.client.1.smithi044.stdout:0/388: link d3/d49/l6c d3/d9/l8a 0 2024-06-03T01:20:51.930 INFO:tasks.workunit.client.1.smithi044.stdout:3/206: write df/d13/d1b/f20 [3652645,118879] 0 2024-06-03T01:20:51.931 INFO:tasks.workunit.client.1.smithi044.stdout:0/389: rmdir d3/d9/d55/d5a/d29 39 2024-06-03T01:20:51.931 INFO:tasks.workunit.client.1.smithi044.stdout:0/390: write d3/d9/f86 [998779,91320] 0 2024-06-03T01:20:51.932 INFO:tasks.workunit.client.1.smithi044.stdout:3/207: truncate df/d13/f29 1249019 0 2024-06-03T01:20:51.933 INFO:tasks.workunit.client.1.smithi044.stdout:0/391: unlink d3/d6b/f84 0 2024-06-03T01:20:51.933 INFO:tasks.workunit.client.1.smithi044.stdout:6/377: dread d2/f14 [0,4194304] 0 2024-06-03T01:20:51.934 INFO:tasks.workunit.client.1.smithi044.stdout:3/208: mkdir df/d13/d39/d3d 0 2024-06-03T01:20:51.934 INFO:tasks.workunit.client.1.smithi044.stdout:3/209: write fe [2800318,25696] 0 2024-06-03T01:20:51.935 INFO:tasks.workunit.client.1.smithi044.stdout:0/392: rename d3/d49/d21/f4a to d3/d49/f8b 0 2024-06-03T01:20:51.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/378: rename d2/d23/d30/l76 to d2/d23/d30/d1e/d38/d6c/l82 0 2024-06-03T01:20:51.939 INFO:tasks.workunit.client.1.smithi044.stdout:0/393: rename d3/d49/d20/c72 to d3/d49/d56/d62/c8c 0 2024-06-03T01:20:51.941 INFO:tasks.workunit.client.1.smithi044.stdout:0/394: dread d3/d49/f3e [0,4194304] 0 2024-06-03T01:20:51.943 INFO:tasks.workunit.client.1.smithi044.stdout:0/395: truncate d3/d49/f3b 857393 0 2024-06-03T01:20:51.943 INFO:tasks.workunit.client.1.smithi044.stdout:0/396: truncate d3/f19 8722016 0 2024-06-03T01:20:51.945 INFO:tasks.workunit.client.1.smithi044.stdout:0/397: symlink d3/d9/d55/d5a/d29/d58/l8d 0 2024-06-03T01:20:51.945 INFO:tasks.workunit.client.1.smithi044.stdout:0/398: dread - d3/f85 zero size 2024-06-03T01:20:51.946 INFO:tasks.workunit.client.1.smithi044.stdout:0/399: write d3/d49/f3e [2361784,30802] 0 2024-06-03T01:20:51.946 INFO:tasks.workunit.client.1.smithi044.stdout:6/379: dread d2/d23/f35 [0,4194304] 0 2024-06-03T01:20:51.947 INFO:tasks.workunit.client.1.smithi044.stdout:6/380: rmdir d2/d4 39 2024-06-03T01:20:51.947 INFO:tasks.workunit.client.1.smithi044.stdout:6/381: write d2/f2c [3649858,104552] 0 2024-06-03T01:20:51.947 INFO:tasks.workunit.client.1.smithi044.stdout:6/382: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/f83 x:0 0 0 2024-06-03T01:20:51.948 INFO:tasks.workunit.client.1.smithi044.stdout:6/383: read d2/f27 [70439,109453] 0 2024-06-03T01:20:51.948 INFO:tasks.workunit.client.1.smithi044.stdout:6/384: dread - d2/d23/d30/f7f zero size 2024-06-03T01:20:51.950 INFO:tasks.workunit.client.1.smithi044.stdout:6/385: creat d2/d23/d30/d1e/d38/d6c/f84 x:0 0 0 2024-06-03T01:20:51.950 INFO:tasks.workunit.client.1.smithi044.stdout:3/210: dwrite df/f31 [0,4194304] 0 2024-06-03T01:20:51.961 INFO:tasks.workunit.client.1.smithi044.stdout:3/211: unlink lb 0 2024-06-03T01:20:51.963 INFO:tasks.workunit.client.1.smithi044.stdout:3/212: link df/d13/f1a df/d1e/d2a/f3e 0 2024-06-03T01:20:51.964 INFO:tasks.workunit.client.1.smithi044.stdout:3/213: rename df/d1e/d2a/f3e to df/d13/f3f 0 2024-06-03T01:20:51.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/400: dwrite d3/d9/d55/d73/f77 [0,4194304] 0 2024-06-03T01:20:51.967 INFO:tasks.workunit.client.1.smithi044.stdout:0/401: read d3/f6d [73793,91024] 0 2024-06-03T01:20:51.970 INFO:tasks.workunit.client.1.smithi044.stdout:0/402: dread d3/d9/f1d [0,4194304] 0 2024-06-03T01:20:51.973 INFO:tasks.workunit.client.1.smithi044.stdout:0/403: mkdir d3/d49/d56/d8e 0 2024-06-03T01:20:51.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/245: sync 2024-06-03T01:20:51.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/246: chown d17/d23/l1f 16302408 1 2024-06-03T01:20:51.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/247: readlink d17/d23/d28/d2b/l29 0 2024-06-03T01:20:51.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/248: stat d17/d23/d35 0 2024-06-03T01:20:51.980 INFO:tasks.workunit.client.1.smithi044.stdout:0/404: creat d3/d5c/f8f x:0 0 0 2024-06-03T01:20:51.980 INFO:tasks.workunit.client.1.smithi044.stdout:0/405: truncate d3/f85 590592 0 2024-06-03T01:20:51.980 INFO:tasks.workunit.client.1.smithi044.stdout:0/406: stat d3/d9/d55/d5a/d29 0 2024-06-03T01:20:51.981 INFO:tasks.workunit.client.1.smithi044.stdout:0/407: symlink d3/d49/d56/d62/l90 0 2024-06-03T01:20:51.981 INFO:tasks.workunit.client.1.smithi044.stdout:0/408: fsync d3/d9/f1f 0 2024-06-03T01:20:51.983 INFO:tasks.workunit.client.1.smithi044.stdout:3/214: dwrite df/f21 [0,4194304] 0 2024-06-03T01:20:51.983 INFO:tasks.workunit.client.1.smithi044.stdout:6/386: dwrite d2/d4/d3f/f46 [0,4194304] 0 2024-06-03T01:20:51.984 INFO:tasks.workunit.client.1.smithi044.stdout:0/409: rename d3/d9/d55/d5a/f3f to d3/f91 0 2024-06-03T01:20:51.988 INFO:tasks.workunit.client.1.smithi044.stdout:3/215: truncate df/d13/f25 891311 0 2024-06-03T01:20:51.990 INFO:tasks.workunit.client.1.smithi044.stdout:3/216: symlink df/d1e/d3a/l40 0 2024-06-03T01:20:51.991 INFO:tasks.workunit.client.1.smithi044.stdout:3/217: symlink df/d13/d1b/l41 0 2024-06-03T01:20:51.993 INFO:tasks.workunit.client.1.smithi044.stdout:3/218: mknod df/c42 0 2024-06-03T01:20:52.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/387: dwrite d2/d23/d30/d1e/f49 [0,4194304] 0 2024-06-03T01:20:52.005 INFO:tasks.workunit.client.1.smithi044.stdout:4/249: dwrite d17/d23/d28/f3e [0,4194304] 0 2024-06-03T01:20:52.015 INFO:tasks.workunit.client.1.smithi044.stdout:0/410: dwrite d3/d49/d20/d25/f65 [0,4194304] 0 2024-06-03T01:20:52.016 INFO:tasks.workunit.client.1.smithi044.stdout:0/411: chown d3/d9/f86 123209304 1 2024-06-03T01:20:52.016 INFO:tasks.workunit.client.1.smithi044.stdout:3/219: dwrite df/d13/f3f [0,4194304] 0 2024-06-03T01:20:52.018 INFO:tasks.workunit.client.1.smithi044.stdout:0/412: creat d3/d49/d56/d8e/f92 x:0 0 0 2024-06-03T01:20:52.018 INFO:tasks.workunit.client.1.smithi044.stdout:0/413: write d3/d49/d20/f5b [196457,72495] 0 2024-06-03T01:20:52.019 INFO:tasks.workunit.client.1.smithi044.stdout:3/220: mkdir df/d13/d39/d3d/d43 0 2024-06-03T01:20:52.020 INFO:tasks.workunit.client.1.smithi044.stdout:0/414: creat d3/d9/d55/d5a/f93 x:0 0 0 2024-06-03T01:20:52.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/221: creat df/d13/d39/d3d/f44 x:0 0 0 2024-06-03T01:20:52.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/222: readlink df/d13/d1b/l41 0 2024-06-03T01:20:52.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/223: read df/d13/f25 [180170,37094] 0 2024-06-03T01:20:52.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/224: truncate df/f32 461684 0 2024-06-03T01:20:52.029 INFO:tasks.workunit.client.1.smithi044.stdout:6/388: dwrite d2/d4/f5e [0,4194304] 0 2024-06-03T01:20:52.029 INFO:tasks.workunit.client.1.smithi044.stdout:6/389: chown d2/f14 64453 1 2024-06-03T01:20:52.029 INFO:tasks.workunit.client.1.smithi044.stdout:6/390: fdatasync d2/d23/d30/f1d 0 2024-06-03T01:20:52.029 INFO:tasks.workunit.client.1.smithi044.stdout:4/250: dwrite d17/f31 [0,4194304] 0 2024-06-03T01:20:52.038 INFO:tasks.workunit.client.1.smithi044.stdout:4/251: mkdir d17/d23/d55 0 2024-06-03T01:20:52.040 INFO:tasks.workunit.client.1.smithi044.stdout:4/252: mknod d17/d23/c56 0 2024-06-03T01:20:52.043 INFO:tasks.workunit.client.1.smithi044.stdout:4/253: link fc d17/d23/f57 0 2024-06-03T01:20:52.044 INFO:tasks.workunit.client.1.smithi044.stdout:0/415: dwrite d3/d9/f28 [0,4194304] 0 2024-06-03T01:20:52.044 INFO:tasks.workunit.client.1.smithi044.stdout:0/416: dread - d3/f6e zero size 2024-06-03T01:20:52.044 INFO:tasks.workunit.client.1.smithi044.stdout:3/225: dwrite df/d13/f29 [0,4194304] 0 2024-06-03T01:20:52.044 INFO:tasks.workunit.client.1.smithi044.stdout:3/226: creat df/d1e/f45 x:0 0 0 2024-06-03T01:20:52.044 INFO:tasks.workunit.client.1.smithi044.stdout:4/254: mkdir d17/d23/d58 0 2024-06-03T01:20:52.052 INFO:tasks.workunit.client.1.smithi044.stdout:3/227: creat df/d13/d1b/f46 x:0 0 0 2024-06-03T01:20:52.052 INFO:tasks.workunit.client.1.smithi044.stdout:3/228: creat df/f47 x:0 0 0 2024-06-03T01:20:52.052 INFO:tasks.workunit.client.1.smithi044.stdout:0/417: getdents d3/d49/d20/d25 0 2024-06-03T01:20:52.054 INFO:tasks.workunit.client.1.smithi044.stdout:3/229: mknod df/d13/d39/c48 0 2024-06-03T01:20:52.054 INFO:tasks.workunit.client.1.smithi044.stdout:3/230: creat df/f49 x:0 0 0 2024-06-03T01:20:52.055 INFO:tasks.workunit.client.1.smithi044.stdout:3/231: readlink l4 0 2024-06-03T01:20:52.057 INFO:tasks.workunit.client.1.smithi044.stdout:3/232: rename df/d13/f2e to df/d1e/d3a/f4a 0 2024-06-03T01:20:52.061 INFO:tasks.workunit.client.1.smithi044.stdout:0/418: dread d3/d49/d20/d25/f40 [0,4194304] 0 2024-06-03T01:20:52.063 INFO:tasks.workunit.client.1.smithi044.stdout:0/419: unlink d3/d49/d21/c68 0 2024-06-03T01:20:52.063 INFO:tasks.workunit.client.1.smithi044.stdout:4/255: dwrite d17/f31 [0,4194304] 0 2024-06-03T01:20:52.063 INFO:tasks.workunit.client.1.smithi044.stdout:4/256: dread - d17/d1c/f52 zero size 2024-06-03T01:20:52.063 INFO:tasks.workunit.client.1.smithi044.stdout:4/257: read d17/d23/d28/d2b/f21 [105748,110007] 0 2024-06-03T01:20:52.065 INFO:tasks.workunit.client.1.smithi044.stdout:6/391: dwrite d2/d4/d3f/f46 [0,4194304] 0 2024-06-03T01:20:52.065 INFO:tasks.workunit.client.1.smithi044.stdout:0/420: mkdir d3/d49/d21/d94 0 2024-06-03T01:20:52.068 INFO:tasks.workunit.client.1.smithi044.stdout:0/421: mkdir d3/d6b/d95 0 2024-06-03T01:20:52.068 INFO:tasks.workunit.client.1.smithi044.stdout:0/422: chown d3/d6b/d95 551476217 1 2024-06-03T01:20:52.068 INFO:tasks.workunit.client.1.smithi044.stdout:0/423: fsync d3/f69 0 2024-06-03T01:20:52.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/415: sync 2024-06-03T01:20:52.071 INFO:tasks.workunit.client.1.smithi044.stdout:5/296: sync 2024-06-03T01:20:52.071 INFO:tasks.workunit.client.1.smithi044.stdout:3/233: dwrite df/d1e/f35 [0,4194304] 0 2024-06-03T01:20:52.072 INFO:tasks.workunit.client.1.smithi044.stdout:7/416: truncate d0/d16/f3c 2343191 0 2024-06-03T01:20:52.075 INFO:tasks.workunit.client.1.smithi044.stdout:5/297: creat d2/dc/d28/d4c/f61 x:0 0 0 2024-06-03T01:20:52.086 INFO:tasks.workunit.client.1.smithi044.stdout:6/392: dwrite d2/d23/d55/d5f/f6e [0,4194304] 0 2024-06-03T01:20:52.092 INFO:tasks.workunit.client.1.smithi044.stdout:2/215: sync 2024-06-03T01:20:52.092 INFO:tasks.workunit.client.1.smithi044.stdout:1/305: sync 2024-06-03T01:20:52.092 INFO:tasks.workunit.client.1.smithi044.stdout:8/365: sync 2024-06-03T01:20:52.092 INFO:tasks.workunit.client.1.smithi044.stdout:9/259: sync 2024-06-03T01:20:52.092 INFO:tasks.workunit.client.1.smithi044.stdout:4/258: dwrite d17/d23/d28/d2b/f4a [0,4194304] 0 2024-06-03T01:20:52.093 INFO:tasks.workunit.client.1.smithi044.stdout:2/216: mkdir db/d15/d49 0 2024-06-03T01:20:52.093 INFO:tasks.workunit.client.1.smithi044.stdout:3/234: dwrite df/f47 [0,4194304] 0 2024-06-03T01:20:52.093 INFO:tasks.workunit.client.1.smithi044.stdout:1/306: mknod d0/d3/d20/d7/d2f/c6e 0 2024-06-03T01:20:52.096 INFO:tasks.workunit.client.1.smithi044.stdout:8/366: link d5/d16/d1d/f1f d5/d16/d1d/d59/f6f 0 2024-06-03T01:20:52.096 INFO:tasks.workunit.client.1.smithi044.stdout:6/393: dread d2/d23/f33 [0,4194304] 0 2024-06-03T01:20:52.097 INFO:tasks.workunit.client.1.smithi044.stdout:9/260: mknod d6/d38/c50 0 2024-06-03T01:20:52.097 INFO:tasks.workunit.client.1.smithi044.stdout:9/261: write d6/d38/d42/d16/f3f [8465347,124484] 0 2024-06-03T01:20:52.100 INFO:tasks.workunit.client.1.smithi044.stdout:4/259: mkdir d17/d23/d59 0 2024-06-03T01:20:52.102 INFO:tasks.workunit.client.1.smithi044.stdout:6/394: mkdir d2/d23/d55/d5f/d85 0 2024-06-03T01:20:52.102 INFO:tasks.workunit.client.1.smithi044.stdout:6/395: dread - d2/d23/d30/f74 zero size 2024-06-03T01:20:52.104 INFO:tasks.workunit.client.1.smithi044.stdout:6/396: mknod d2/d4/d3f/c86 0 2024-06-03T01:20:52.104 INFO:tasks.workunit.client.1.smithi044.stdout:6/397: creat d2/d23/d30/d1e/d34/d39/d64/f87 x:0 0 0 2024-06-03T01:20:52.104 INFO:tasks.workunit.client.1.smithi044.stdout:6/398: write d2/d4/f7 [92874,126268] 0 2024-06-03T01:20:52.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/235: read df/f19 [6231100,44419] 0 2024-06-03T01:20:52.105 INFO:tasks.workunit.client.1.smithi044.stdout:1/307: dread d0/d3/d20/d7/d13/f3d [0,4194304] 0 2024-06-03T01:20:52.105 INFO:tasks.workunit.client.1.smithi044.stdout:9/262: rename d6/d38/d42 to d6/dd/d51 0 2024-06-03T01:20:52.106 INFO:tasks.workunit.client.1.smithi044.stdout:6/399: symlink d2/d23/d55/d5f/d53/d5a/l88 0 2024-06-03T01:20:52.107 INFO:tasks.workunit.client.1.smithi044.stdout:6/400: creat d2/d23/d30/d1e/d38/d6c/f89 x:0 0 0 2024-06-03T01:20:52.107 INFO:tasks.workunit.client.1.smithi044.stdout:3/236: mkdir df/d13/d1b/d4b 0 2024-06-03T01:20:52.107 INFO:tasks.workunit.client.1.smithi044.stdout:1/308: mkdir d0/d3/d20/d40/d45/d6f 0 2024-06-03T01:20:52.108 INFO:tasks.workunit.client.1.smithi044.stdout:1/309: chown d0/d3/d20/d7/d4e/f57 723693523 1 2024-06-03T01:20:52.112 INFO:tasks.workunit.client.1.smithi044.stdout:1/310: dread d0/d3/d20/d7/f17 [0,4194304] 0 2024-06-03T01:20:52.113 INFO:tasks.workunit.client.1.smithi044.stdout:6/401: link d2/d23/f35 d2/d23/d30/d1e/d34/d39/d64/d70/f8a 0 2024-06-03T01:20:52.115 INFO:tasks.workunit.client.1.smithi044.stdout:1/311: chown d0/f15 109001 1 2024-06-03T01:20:52.115 INFO:tasks.workunit.client.1.smithi044.stdout:1/312: write d0/d3/f62 [428130,115993] 0 2024-06-03T01:20:52.117 INFO:tasks.workunit.client.1.smithi044.stdout:6/402: symlink d2/d23/l8b 0 2024-06-03T01:20:52.117 INFO:tasks.workunit.client.1.smithi044.stdout:6/403: read d2/d4/f7 [645008,57033] 0 2024-06-03T01:20:52.118 INFO:tasks.workunit.client.1.smithi044.stdout:9/263: dread d6/f11 [0,4194304] 0 2024-06-03T01:20:52.118 INFO:tasks.workunit.client.1.smithi044.stdout:9/264: write d6/dd/d51/db/f15 [10758014,9797] 0 2024-06-03T01:20:52.119 INFO:tasks.workunit.client.1.smithi044.stdout:1/313: mkdir d0/d16/d70 0 2024-06-03T01:20:52.119 INFO:tasks.workunit.client.1.smithi044.stdout:8/367: dwrite d5/d20/d33/f3f [0,4194304] 0 2024-06-03T01:20:52.120 INFO:tasks.workunit.client.1.smithi044.stdout:6/404: rename d2/d23/c65 to d2/d4/d3f/c8c 0 2024-06-03T01:20:52.120 INFO:tasks.workunit.client.1.smithi044.stdout:6/405: dread - d2/d23/d30/d1e/d38/d6c/d5c/d79/f81 zero size 2024-06-03T01:20:52.124 INFO:tasks.workunit.client.1.smithi044.stdout:8/368: creat d5/d16/d1d/d6b/d6d/f70 x:0 0 0 2024-06-03T01:20:52.124 INFO:tasks.workunit.client.1.smithi044.stdout:1/314: dread d0/f15 [0,4194304] 0 2024-06-03T01:20:52.125 INFO:tasks.workunit.client.1.smithi044.stdout:9/265: rename d6/d38/c3e to d6/d26/c52 0 2024-06-03T01:20:52.126 INFO:tasks.workunit.client.1.smithi044.stdout:8/369: creat d5/d20/d47/f71 x:0 0 0 2024-06-03T01:20:52.127 INFO:tasks.workunit.client.1.smithi044.stdout:1/315: mknod d0/d3/d20/d40/d45/d4b/d5a/c71 0 2024-06-03T01:20:52.128 INFO:tasks.workunit.client.1.smithi044.stdout:9/266: link d6/dd/f36 d6/dd/d51/d16/d27/f53 0 2024-06-03T01:20:52.129 INFO:tasks.workunit.client.1.smithi044.stdout:8/370: creat d5/d55/d56/d5c/f72 x:0 0 0 2024-06-03T01:20:52.129 INFO:tasks.workunit.client.1.smithi044.stdout:8/371: creat d5/d10/f73 x:0 0 0 2024-06-03T01:20:52.130 INFO:tasks.workunit.client.1.smithi044.stdout:1/316: rename d0/d3/d20/d7/d2f/f68 to d0/d3/d20/d40/d45/d4b/d5a/f72 0 2024-06-03T01:20:52.131 INFO:tasks.workunit.client.1.smithi044.stdout:8/372: read d5/d16/d1d/f42 [90483,87133] 0 2024-06-03T01:20:52.131 INFO:tasks.workunit.client.1.smithi044.stdout:4/260: dwrite d17/d23/f4d [0,4194304] 0 2024-06-03T01:20:52.131 INFO:tasks.workunit.client.1.smithi044.stdout:9/267: symlink d6/dd/d51/d16/d4d/l54 0 2024-06-03T01:20:52.133 INFO:tasks.workunit.client.1.smithi044.stdout:1/317: symlink d0/d3/d20/d7/l73 0 2024-06-03T01:20:52.135 INFO:tasks.workunit.client.1.smithi044.stdout:4/261: symlink d17/l5a 0 2024-06-03T01:20:52.135 INFO:tasks.workunit.client.1.smithi044.stdout:1/318: rename d0/d3/d20/f1d to d0/d16/d64/d6a/f74 0 2024-06-03T01:20:52.138 INFO:tasks.workunit.client.1.smithi044.stdout:3/237: dwrite df/d13/f29 [0,4194304] 0 2024-06-03T01:20:52.138 INFO:tasks.workunit.client.1.smithi044.stdout:3/238: write df/d13/d1b/f38 [267241,33524] 0 2024-06-03T01:20:52.138 INFO:tasks.workunit.client.1.smithi044.stdout:3/239: readlink df/d13/d1b/l41 0 2024-06-03T01:20:52.154 INFO:tasks.workunit.client.1.smithi044.stdout:8/373: dwrite d5/d16/d1d/f31 [0,4194304] 0 2024-06-03T01:20:52.158 INFO:tasks.workunit.client.1.smithi044.stdout:9/268: dwrite f2 [0,4194304] 0 2024-06-03T01:20:52.161 INFO:tasks.workunit.client.1.smithi044.stdout:8/374: readlink d5/l8 0 2024-06-03T01:20:52.165 INFO:tasks.workunit.client.1.smithi044.stdout:9/269: write d6/dd/f17 [523351,118481] 0 2024-06-03T01:20:52.167 INFO:tasks.workunit.client.1.smithi044.stdout:9/270: mkdir d6/dd/d51/d16/d4d/d55 0 2024-06-03T01:20:52.167 INFO:tasks.workunit.client.1.smithi044.stdout:1/319: dwrite d0/d3/d20/f6d [0,4194304] 0 2024-06-03T01:20:52.168 INFO:tasks.workunit.client.1.smithi044.stdout:9/271: creat d6/dd/d45/f56 x:0 0 0 2024-06-03T01:20:52.169 INFO:tasks.workunit.client.1.smithi044.stdout:6/406: dwrite d2/d23/d55/d5f/d53/f57 [0,4194304] 0 2024-06-03T01:20:52.171 INFO:tasks.workunit.client.1.smithi044.stdout:1/320: creat d0/d16/d64/d6a/f75 x:0 0 0 2024-06-03T01:20:52.175 INFO:tasks.workunit.client.1.smithi044.stdout:1/321: symlink d0/d3/d14/l76 0 2024-06-03T01:20:52.177 INFO:tasks.workunit.client.1.smithi044.stdout:1/322: unlink d0/d3/d14/l76 0 2024-06-03T01:20:52.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/272: dread d6/dd/d51/d16/f3f [0,4194304] 0 2024-06-03T01:20:52.178 INFO:tasks.workunit.client.1.smithi044.stdout:1/323: creat d0/d3/d20/d40/d45/d6b/f77 x:0 0 0 2024-06-03T01:20:52.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/273: rmdir d6/dd/d51/d33 0 2024-06-03T01:20:52.179 INFO:tasks.workunit.client.1.smithi044.stdout:1/324: getdents d0/d16/d64 0 2024-06-03T01:20:52.179 INFO:tasks.workunit.client.1.smithi044.stdout:1/325: stat d0/d3/d20/d7/d2f/f59 0 2024-06-03T01:20:52.180 INFO:tasks.workunit.client.1.smithi044.stdout:9/274: symlink d6/dd/d51/d3b/l57 0 2024-06-03T01:20:52.180 INFO:tasks.workunit.client.1.smithi044.stdout:9/275: write d6/dd/d51/f13 [310339,114815] 0 2024-06-03T01:20:52.181 INFO:tasks.workunit.client.1.smithi044.stdout:6/407: read d2/fe [8223833,107016] 0 2024-06-03T01:20:52.181 INFO:tasks.workunit.client.1.smithi044.stdout:3/240: dwrite df/d13/f24 [0,4194304] 0 2024-06-03T01:20:52.181 INFO:tasks.workunit.client.1.smithi044.stdout:1/326: write d0/f32 [262833,61477] 0 2024-06-03T01:20:52.182 INFO:tasks.workunit.client.1.smithi044.stdout:4/262: dwrite ff [0,4194304] 0 2024-06-03T01:20:52.182 INFO:tasks.workunit.client.1.smithi044.stdout:3/241: write df/f49 [641120,80202] 0 2024-06-03T01:20:52.182 INFO:tasks.workunit.client.1.smithi044.stdout:9/276: symlink d6/dd/d51/d46/l58 0 2024-06-03T01:20:52.186 INFO:tasks.workunit.client.1.smithi044.stdout:6/408: write d2/f14 [2321066,116453] 0 2024-06-03T01:20:52.186 INFO:tasks.workunit.client.1.smithi044.stdout:6/409: fdatasync d2/d23/d44/f54 0 2024-06-03T01:20:52.187 INFO:tasks.workunit.client.1.smithi044.stdout:9/277: dread d6/dd/d51/f13 [0,4194304] 0 2024-06-03T01:20:52.188 INFO:tasks.workunit.client.1.smithi044.stdout:4/263: rename d17/d23/l4c to d17/d23/d59/l5b 0 2024-06-03T01:20:52.189 INFO:tasks.workunit.client.1.smithi044.stdout:6/410: creat d2/d4/f8d x:0 0 0 2024-06-03T01:20:52.190 INFO:tasks.workunit.client.1.smithi044.stdout:6/411: fsync d2/d23/d55/f6f 0 2024-06-03T01:20:52.190 INFO:tasks.workunit.client.1.smithi044.stdout:3/242: write df/f2f [2468339,105268] 0 2024-06-03T01:20:52.190 INFO:tasks.workunit.client.1.smithi044.stdout:3/243: creat df/d13/d1b/f4c x:0 0 0 2024-06-03T01:20:52.190 INFO:tasks.workunit.client.1.smithi044.stdout:3/244: fdatasync df/d13/d1b/f4c 0 2024-06-03T01:20:52.192 INFO:tasks.workunit.client.1.smithi044.stdout:6/412: link d2/d23/d30/d1e/d38/d6c/f12 d2/d23/d30/d1e/d38/d77/f8e 0 2024-06-03T01:20:52.192 INFO:tasks.workunit.client.1.smithi044.stdout:6/413: write d2/d23/d55/f6f [3927418,119577] 0 2024-06-03T01:20:52.195 INFO:tasks.workunit.client.1.smithi044.stdout:3/245: mkdir df/d13/d39/d3d/d4d 0 2024-06-03T01:20:52.198 INFO:tasks.workunit.client.1.smithi044.stdout:6/414: link d2/d23/d30/d1e/c32 d2/d23/d55/d5f/d53/d5a/c8f 0 2024-06-03T01:20:52.199 INFO:tasks.workunit.client.1.smithi044.stdout:6/415: dread - d2/d23/d30/d1e/f47 zero size 2024-06-03T01:20:52.199 INFO:tasks.workunit.client.1.smithi044.stdout:3/246: readlink lc 0 2024-06-03T01:20:52.199 INFO:tasks.workunit.client.1.smithi044.stdout:3/247: readlink df/l27 0 2024-06-03T01:20:52.199 INFO:tasks.workunit.client.1.smithi044.stdout:3/248: read - df/d1e/d2a/f30 zero size 2024-06-03T01:20:52.200 INFO:tasks.workunit.client.1.smithi044.stdout:6/416: creat d2/d4/d3f/f90 x:0 0 0 2024-06-03T01:20:52.201 INFO:tasks.workunit.client.1.smithi044.stdout:3/249: mknod df/d1e/d2a/c4e 0 2024-06-03T01:20:52.201 INFO:tasks.workunit.client.1.smithi044.stdout:6/417: dread d2/d23/d55/d5f/f59 [0,4194304] 0 2024-06-03T01:20:52.203 INFO:tasks.workunit.client.1.smithi044.stdout:6/418: mknod d2/d23/d55/d5f/d85/c91 0 2024-06-03T01:20:52.203 INFO:tasks.workunit.client.1.smithi044.stdout:6/419: creat d2/d23/d30/d1e/d38/d6c/f92 x:0 0 0 2024-06-03T01:20:52.203 INFO:tasks.workunit.client.1.smithi044.stdout:6/420: readlink d2/d6/l5b 0 2024-06-03T01:20:52.207 INFO:tasks.workunit.client.1.smithi044.stdout:6/421: rename d2/d23/d55/d5f/d53/f57 to d2/d23/d55/d5f/d53/d5a/f93 0 2024-06-03T01:20:52.207 INFO:tasks.workunit.client.1.smithi044.stdout:6/422: creat d2/d23/d44/f94 x:0 0 0 2024-06-03T01:20:52.207 INFO:tasks.workunit.client.1.smithi044.stdout:1/327: write d0/d16/d64/d6a/f74 [1903343,73856] 0 2024-06-03T01:20:52.209 INFO:tasks.workunit.client.1.smithi044.stdout:1/328: truncate d0/d3/d20/f6d 1171267 0 2024-06-03T01:20:52.209 INFO:tasks.workunit.client.1.smithi044.stdout:3/250: write df/d13/f29 [3592817,122735] 0 2024-06-03T01:20:52.210 INFO:tasks.workunit.client.1.smithi044.stdout:1/329: stat d0/f32 0 2024-06-03T01:20:52.212 INFO:tasks.workunit.client.1.smithi044.stdout:1/330: rename d0/d3/d20/d7/f44 to d0/d3/d20/d7/d4e/f78 0 2024-06-03T01:20:52.212 INFO:tasks.workunit.client.1.smithi044.stdout:1/331: chown d0/d3/d20/d40/d45/c46 100 1 2024-06-03T01:20:52.213 INFO:tasks.workunit.client.1.smithi044.stdout:3/251: creat df/d13/d1b/d4b/f4f x:0 0 0 2024-06-03T01:20:52.214 INFO:tasks.workunit.client.1.smithi044.stdout:1/332: dread d0/f32 [0,4194304] 0 2024-06-03T01:20:52.214 INFO:tasks.workunit.client.1.smithi044.stdout:1/333: chown d0/d3/d20/l48 406825638 1 2024-06-03T01:20:52.214 INFO:tasks.workunit.client.1.smithi044.stdout:3/252: mkdir df/d13/d39/d3d/d50 0 2024-06-03T01:20:52.216 INFO:tasks.workunit.client.1.smithi044.stdout:1/334: creat d0/d3/d20/d7/d13/d2d/f79 x:0 0 0 2024-06-03T01:20:52.217 INFO:tasks.workunit.client.1.smithi044.stdout:3/253: mknod df/d13/d1b/d4b/c51 0 2024-06-03T01:20:52.217 INFO:tasks.workunit.client.1.smithi044.stdout:3/254: read - df/d1e/d2a/f30 zero size 2024-06-03T01:20:52.218 INFO:tasks.workunit.client.1.smithi044.stdout:1/335: mkdir d0/d3/d20/d7/d13/d7a 0 2024-06-03T01:20:52.220 INFO:tasks.workunit.client.1.smithi044.stdout:6/423: dwrite d2/d23/f18 [4194304,4194304] 0 2024-06-03T01:20:52.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/336: write d0/f15 [2046092,87623] 0 2024-06-03T01:20:52.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/337: read d0/d3/d20/d7/d13/f24 [1520989,109513] 0 2024-06-03T01:20:52.223 INFO:tasks.workunit.client.1.smithi044.stdout:6/424: creat d2/d23/d30/d1e/d34/d39/d7d/f95 x:0 0 0 2024-06-03T01:20:52.225 INFO:tasks.workunit.client.1.smithi044.stdout:6/425: unlink d2/d6/f13 0 2024-06-03T01:20:52.227 INFO:tasks.workunit.client.1.smithi044.stdout:6/426: mknod d2/d23/d44/d4d/c96 0 2024-06-03T01:20:52.227 INFO:tasks.workunit.client.1.smithi044.stdout:1/338: write d0/d16/f26 [2377,19150] 0 2024-06-03T01:20:52.228 INFO:tasks.workunit.client.1.smithi044.stdout:6/427: mknod d2/c97 0 2024-06-03T01:20:52.230 INFO:tasks.workunit.client.1.smithi044.stdout:1/339: rename d0/d3/l11 to d0/d3/d20/d40/d45/l7b 0 2024-06-03T01:20:52.230 INFO:tasks.workunit.client.1.smithi044.stdout:1/340: write d0/d16/f26 [3888164,41271] 0 2024-06-03T01:20:52.231 INFO:tasks.workunit.client.1.smithi044.stdout:1/341: chown d0/d3/f19 29 1 2024-06-03T01:20:52.231 INFO:tasks.workunit.client.1.smithi044.stdout:3/255: dwrite df/d13/f1d [0,4194304] 0 2024-06-03T01:20:52.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/428: mknod d2/d23/d30/d1e/d34/d39/c98 0 2024-06-03T01:20:52.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/429: fsync d2/d23/d30/d1e/d38/f72 0 2024-06-03T01:20:52.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/430: fsync d2/d23/d55/d5f/d53/f73 0 2024-06-03T01:20:52.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/431: chown d2/d23/d55/d5f/f59 383 1 2024-06-03T01:20:52.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/432: chown d2/d23/d30/d1e/d38/d6c/d5c/c51 7901 1 2024-06-03T01:20:52.238 INFO:tasks.workunit.client.1.smithi044.stdout:0/424: sync 2024-06-03T01:20:52.238 INFO:tasks.workunit.client.1.smithi044.stdout:1/342: write d0/d3/d20/d7/d2f/f59 [878189,83578] 0 2024-06-03T01:20:52.240 INFO:tasks.workunit.client.1.smithi044.stdout:0/425: rename d3/d9/d55/d73/l7a to d3/d6b/d95/l96 0 2024-06-03T01:20:52.241 INFO:tasks.workunit.client.1.smithi044.stdout:6/433: getdents d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:52.241 INFO:tasks.workunit.client.1.smithi044.stdout:6/434: getdents d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:52.241 INFO:tasks.workunit.client.1.smithi044.stdout:6/435: write d2/d23/d30/f1d [1110418,117076] 0 2024-06-03T01:20:52.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/256: dread fd [0,4194304] 0 2024-06-03T01:20:52.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/257: chown df/d13/d39/c48 0 1 2024-06-03T01:20:52.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/258: fsync df/d13/f3f 0 2024-06-03T01:20:52.242 INFO:tasks.workunit.client.1.smithi044.stdout:1/343: fsync d0/f15 0 2024-06-03T01:20:52.243 INFO:tasks.workunit.client.1.smithi044.stdout:6/436: mkdir d2/d23/d55/d5f/d99 0 2024-06-03T01:20:52.244 INFO:tasks.workunit.client.1.smithi044.stdout:3/259: mknod df/d13/d1b/d4b/c52 0 2024-06-03T01:20:52.245 INFO:tasks.workunit.client.1.smithi044.stdout:1/344: truncate d0/d3/d20/d7/d2f/f33 420806 0 2024-06-03T01:20:52.245 INFO:tasks.workunit.client.1.smithi044.stdout:1/345: write d0/f32 [1420529,56679] 0 2024-06-03T01:20:52.246 INFO:tasks.workunit.client.1.smithi044.stdout:0/426: write d3/d9/d36/f42 [2032059,8299] 0 2024-06-03T01:20:52.246 INFO:tasks.workunit.client.1.smithi044.stdout:6/437: unlink d2/d4/f5e 0 2024-06-03T01:20:52.248 INFO:tasks.workunit.client.1.smithi044.stdout:3/260: symlink df/d13/d39/l53 0 2024-06-03T01:20:52.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/346: mknod d0/d3/d20/d40/d45/c7c 0 2024-06-03T01:20:52.250 INFO:tasks.workunit.client.1.smithi044.stdout:0/427: unlink d3/f17 0 2024-06-03T01:20:52.251 INFO:tasks.workunit.client.1.smithi044.stdout:0/428: readlink d3/d9/d55/d5a/l51 0 2024-06-03T01:20:52.252 INFO:tasks.workunit.client.1.smithi044.stdout:1/347: unlink d0/d3/d20/f8 0 2024-06-03T01:20:52.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/429: mkdir d3/d49/d56/d8e/d97 0 2024-06-03T01:20:52.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/430: chown d3/d6b/d95/l96 6 1 2024-06-03T01:20:52.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/431: creat d3/f98 x:0 0 0 2024-06-03T01:20:52.260 INFO:tasks.workunit.client.1.smithi044.stdout:0/432: symlink d3/d9/d36/d82/l99 0 2024-06-03T01:20:52.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/261: dwrite df/d13/f26 [4194304,4194304] 0 2024-06-03T01:20:52.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/262: rmdir df/d13/d39/d3d/d4d 0 2024-06-03T01:20:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/348: dwrite d0/d3/d20/d7/f17 [0,4194304] 0 2024-06-03T01:20:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/349: truncate d0/f32 1586500 0 2024-06-03T01:20:52.274 INFO:tasks.workunit.client.1.smithi044.stdout:1/350: creat d0/f7d x:0 0 0 2024-06-03T01:20:52.274 INFO:tasks.workunit.client.1.smithi044.stdout:1/351: readlink d0/d16/l3e 0 2024-06-03T01:20:52.275 INFO:tasks.workunit.client.1.smithi044.stdout:3/263: write df/f10 [2851243,82814] 0 2024-06-03T01:20:52.276 INFO:tasks.workunit.client.1.smithi044.stdout:3/264: getdents df/d1e/d3a 0 2024-06-03T01:20:52.276 INFO:tasks.workunit.client.1.smithi044.stdout:3/265: creat df/d13/d1b/d4b/f54 x:0 0 0 2024-06-03T01:20:52.283 INFO:tasks.workunit.client.1.smithi044.stdout:1/352: write d0/d3/d20/d7/f17 [1131959,106575] 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:3/266: write df/d13/f26 [6011330,55173] 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:3/267: unlink df/d13/d1b/d4b/c52 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:1/353: mknod d0/c7e 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:1/354: stat d0/c7e 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:1/355: write d0/d3/d20/d40/f55 [173690,113614] 0 2024-06-03T01:20:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:1/356: mknod d0/d3/d20/d7/d42/d47/c7f 0 2024-06-03T01:20:52.287 INFO:tasks.workunit.client.1.smithi044.stdout:1/357: truncate d0/d3/d20/d40/d45/d4b/d5c/f63 627401 0 2024-06-03T01:20:52.287 INFO:tasks.workunit.client.1.smithi044.stdout:1/358: chown d0/d3/d20/d40 44 1 2024-06-03T01:20:52.287 INFO:tasks.workunit.client.1.smithi044.stdout:1/359: truncate d0/d3/d20/d7/d13/f24 2381393 0 2024-06-03T01:20:52.288 INFO:tasks.workunit.client.1.smithi044.stdout:1/360: mkdir d0/d3/d20/d40/d45/d4b/d80 0 2024-06-03T01:20:52.296 INFO:tasks.workunit.client.1.smithi044.stdout:1/361: dread d0/d3/d20/d7/d4e/f60 [0,4194304] 0 2024-06-03T01:20:52.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/362: dread d0/d3/d20/d7/d4e/f60 [0,4194304] 0 2024-06-03T01:20:52.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/363: fsync d0/f6c 0 2024-06-03T01:20:52.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/364: truncate d0/f3f 402519 0 2024-06-03T01:20:52.300 INFO:tasks.workunit.client.1.smithi044.stdout:3/268: dwrite df/d1e/f45 [0,4194304] 0 2024-06-03T01:20:52.311 INFO:tasks.workunit.client.1.smithi044.stdout:1/365: dread d0/d3/d20/d7/d42/f56 [0,4194304] 0 2024-06-03T01:20:52.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/366: rename d0/d3/d20/d7/d42 to d0/d3/d14/d81 0 2024-06-03T01:20:52.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/367: read - d0/d3/d20/d40/d45/d6b/f77 zero size 2024-06-03T01:20:52.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/368: fdatasync d0/f7d 0 2024-06-03T01:20:52.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/369: chown d0/d3/c49 186428169 1 2024-06-03T01:20:52.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/370: truncate d0/d66/f65 296695 0 2024-06-03T01:20:52.315 INFO:tasks.workunit.client.1.smithi044.stdout:1/371: creat d0/d16/d64/f82 x:0 0 0 2024-06-03T01:20:52.315 INFO:tasks.workunit.client.1.smithi044.stdout:1/372: write d0/d3/d20/d40/d45/d4b/d5a/f5f [652645,6112] 0 2024-06-03T01:20:52.318 INFO:tasks.workunit.client.1.smithi044.stdout:1/373: dread d0/d3/d20/d7/d13/f3d [0,4194304] 0 2024-06-03T01:20:52.318 INFO:tasks.workunit.client.1.smithi044.stdout:3/269: dwrite df/d13/f26 [8388608,4194304] 0 2024-06-03T01:20:52.319 INFO:tasks.workunit.client.1.smithi044.stdout:1/374: truncate d0/d16/f26 2906970 0 2024-06-03T01:20:52.319 INFO:tasks.workunit.client.1.smithi044.stdout:3/270: fsync f9 0 2024-06-03T01:20:52.328 INFO:tasks.workunit.client.1.smithi044.stdout:3/271: dread df/d13/f1a [0,4194304] 0 2024-06-03T01:20:52.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/375: dwrite d0/d66/f65 [0,4194304] 0 2024-06-03T01:20:52.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:51 smithi032 ceph-mon[29360]: pgmap v255: 65 pgs: 65 active+clean; 413 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 806 op/s 2024-06-03T01:20:52.344 INFO:tasks.workunit.client.1.smithi044.stdout:1/376: dread d0/d3/d20/d7/d4e/f78 [0,4194304] 0 2024-06-03T01:20:52.346 INFO:tasks.workunit.client.1.smithi044.stdout:3/272: dwrite df/f1c [4194304,4194304] 0 2024-06-03T01:20:52.346 INFO:tasks.workunit.client.1.smithi044.stdout:3/273: readlink df/l27 0 2024-06-03T01:20:52.348 INFO:tasks.workunit.client.1.smithi044.stdout:3/274: creat df/d13/d39/d3d/d43/f55 x:0 0 0 2024-06-03T01:20:52.350 INFO:tasks.workunit.client.1.smithi044.stdout:3/275: unlink df/d13/d1b/f4c 0 2024-06-03T01:20:52.350 INFO:tasks.workunit.client.1.smithi044.stdout:3/276: truncate df/d1e/f35 2354756 0 2024-06-03T01:20:52.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/277: mkdir df/d13/d1b/d56 0 2024-06-03T01:20:52.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/278: write df/d13/d1b/f20 [4377662,103015] 0 2024-06-03T01:20:52.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/279: write df/d13/d1b/f38 [226282,5343] 0 2024-06-03T01:20:52.357 INFO:tasks.workunit.client.1.smithi044.stdout:1/377: dwrite d0/d3/d20/d7/d4e/f78 [0,4194304] 0 2024-06-03T01:20:52.358 INFO:tasks.workunit.client.1.smithi044.stdout:1/378: read d0/d3/d20/d40/d45/d4b/d5a/f5f [1022298,97738] 0 2024-06-03T01:20:52.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/379: getdents d0/d16/d64 0 2024-06-03T01:20:52.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/380: write d0/d3/d20/d7/d4e/f60 [3622344,84813] 0 2024-06-03T01:20:52.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/381: creat d0/d3/d20/d40/d45/d6b/f83 x:0 0 0 2024-06-03T01:20:52.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/382: fdatasync d0/d3/d20/d7/d2f/f59 0 2024-06-03T01:20:52.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/383: write d0/d3/d20/d40/d45/d4b/d5a/f5f [1759334,6365] 0 2024-06-03T01:20:52.367 INFO:tasks.workunit.client.1.smithi044.stdout:1/384: dread d0/f5 [0,4194304] 0 2024-06-03T01:20:52.369 INFO:tasks.workunit.client.1.smithi044.stdout:1/385: getdents d0/d3/d20/d7/d13 0 2024-06-03T01:20:52.369 INFO:tasks.workunit.client.1.smithi044.stdout:1/386: write d0/d3/d20/d40/d45/d6b/f77 [147313,24914] 0 2024-06-03T01:20:52.375 INFO:tasks.workunit.client.1.smithi044.stdout:3/280: dwrite df/d13/d1b/f20 [0,4194304] 0 2024-06-03T01:20:52.375 INFO:tasks.workunit.client.1.smithi044.stdout:3/281: dread - df/d13/f3b zero size 2024-06-03T01:20:52.376 INFO:tasks.workunit.client.1.smithi044.stdout:3/282: truncate df/d13/d1b/d4b/f4f 235464 0 2024-06-03T01:20:52.377 INFO:tasks.workunit.client.1.smithi044.stdout:3/283: symlink df/d13/d39/d3d/d50/l57 0 2024-06-03T01:20:52.377 INFO:tasks.workunit.client.1.smithi044.stdout:3/284: creat df/f58 x:0 0 0 2024-06-03T01:20:52.377 INFO:tasks.workunit.client.1.smithi044.stdout:3/285: stat df/f17 0 2024-06-03T01:20:52.386 INFO:tasks.workunit.client.1.smithi044.stdout:3/286: write df/d13/f26 [2447177,51905] 0 2024-06-03T01:20:52.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/387: dwrite d0/d3/d20/d7/d13/d2d/f79 [0,4194304] 0 2024-06-03T01:20:52.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/388: rename d0/d3/d20/d7/f2b to d0/d3/d20/d7/d13/f84 0 2024-06-03T01:20:52.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/389: readlink d0/d16/l3e 0 2024-06-03T01:20:52.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/390: readlink d0/d3/d20/l48 0 2024-06-03T01:20:52.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/391: write d0/d16/f67 [469402,91908] 0 2024-06-03T01:20:52.400 INFO:tasks.workunit.client.1.smithi044.stdout:1/392: rmdir d0 39 2024-06-03T01:20:52.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/298: sync 2024-06-03T01:20:52.403 INFO:tasks.workunit.client.1.smithi044.stdout:2/217: sync 2024-06-03T01:20:52.403 INFO:tasks.workunit.client.1.smithi044.stdout:7/417: sync 2024-06-03T01:20:52.404 INFO:tasks.workunit.client.1.smithi044.stdout:5/299: creat d2/dc/f62 x:0 0 0 2024-06-03T01:20:52.404 INFO:tasks.workunit.client.1.smithi044.stdout:5/300: chown d2/d11 252721407 1 2024-06-03T01:20:52.405 INFO:tasks.workunit.client.1.smithi044.stdout:7/418: rename d0/d3/f18 to d0/d21/d63/d7f/f91 0 2024-06-03T01:20:52.405 INFO:tasks.workunit.client.1.smithi044.stdout:7/419: fdatasync d0/d3/f4f 0 2024-06-03T01:20:52.405 INFO:tasks.workunit.client.1.smithi044.stdout:7/420: dread - d0/d3/d30/f79 zero size 2024-06-03T01:20:52.407 INFO:tasks.workunit.client.1.smithi044.stdout:5/301: rename d2/l60 to d2/d11/d1e/d2f/l63 0 2024-06-03T01:20:52.409 INFO:tasks.workunit.client.1.smithi044.stdout:2/218: dread f2 [0,4194304] 0 2024-06-03T01:20:52.409 INFO:tasks.workunit.client.1.smithi044.stdout:3/287: dwrite df/d13/f24 [0,4194304] 0 2024-06-03T01:20:52.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/302: creat d2/d11/d1e/d2f/f64 x:0 0 0 2024-06-03T01:20:52.410 INFO:tasks.workunit.client.1.smithi044.stdout:2/219: truncate db/d1b/d27/f32 1414016 0 2024-06-03T01:20:52.411 INFO:tasks.workunit.client.1.smithi044.stdout:5/303: rmdir d2/dc/d28 39 2024-06-03T01:20:52.412 INFO:tasks.workunit.client.1.smithi044.stdout:5/304: stat d2/d11/d1e/d2f/l56 0 2024-06-03T01:20:52.412 INFO:tasks.workunit.client.1.smithi044.stdout:5/305: creat d2/dc/f65 x:0 0 0 2024-06-03T01:20:52.413 INFO:tasks.workunit.client.1.smithi044.stdout:5/306: mkdir d2/dc/d28/d66 0 2024-06-03T01:20:52.415 INFO:tasks.workunit.client.1.smithi044.stdout:7/421: write d0/d3/d8/d32/f6c [1303172,71364] 0 2024-06-03T01:20:52.418 INFO:tasks.workunit.client.1.smithi044.stdout:7/422: creat d0/d21/d27/f92 x:0 0 0 2024-06-03T01:20:52.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/393: dwrite d0/d3/d14/d81/d47/f4c [4194304,4194304] 0 2024-06-03T01:20:52.420 INFO:tasks.workunit.client.1.smithi044.stdout:2/220: dread db/f19 [0,4194304] 0 2024-06-03T01:20:52.420 INFO:tasks.workunit.client.1.smithi044.stdout:2/221: readlink db/d1b/d27/l3b 0 2024-06-03T01:20:52.421 INFO:tasks.workunit.client.1.smithi044.stdout:7/423: truncate d0/d3/d8/d32/d51/f66 601846 0 2024-06-03T01:20:52.421 INFO:tasks.workunit.client.1.smithi044.stdout:7/424: stat d0/d3/d8/f6d 0 2024-06-03T01:20:52.422 INFO:tasks.workunit.client.1.smithi044.stdout:3/288: truncate df/d13/d1b/f20 2010391 0 2024-06-03T01:20:52.422 INFO:tasks.workunit.client.1.smithi044.stdout:3/289: write df/d13/d1b/d4b/f4f [247442,116613] 0 2024-06-03T01:20:52.422 INFO:tasks.workunit.client.1.smithi044.stdout:3/290: fsync df/f23 0 2024-06-03T01:20:52.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/394: dread d0/d3/d20/d7/f34 [0,4194304] 0 2024-06-03T01:20:52.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/395: truncate d0/d16/f67 564036 0 2024-06-03T01:20:52.424 INFO:tasks.workunit.client.1.smithi044.stdout:2/222: creat db/d1b/d3f/f4a x:0 0 0 2024-06-03T01:20:52.425 INFO:tasks.workunit.client.1.smithi044.stdout:7/425: truncate d0/d21/f3b 2603876 0 2024-06-03T01:20:52.427 INFO:tasks.workunit.client.1.smithi044.stdout:3/291: mkdir df/d13/d1b/d59 0 2024-06-03T01:20:52.427 INFO:tasks.workunit.client.1.smithi044.stdout:1/396: chown d0/c2 693846089 1 2024-06-03T01:20:52.428 INFO:tasks.workunit.client.1.smithi044.stdout:2/223: symlink db/d45/l4b 0 2024-06-03T01:20:52.429 INFO:tasks.workunit.client.1.smithi044.stdout:2/224: chown db/d15/l44 363560381 1 2024-06-03T01:20:52.429 INFO:tasks.workunit.client.1.smithi044.stdout:7/426: rmdir d0/d16/d23 39 2024-06-03T01:20:52.430 INFO:tasks.workunit.client.1.smithi044.stdout:7/427: write d0/f7c [526717,5099] 0 2024-06-03T01:20:52.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/292: mknod df/d13/d1b/d56/c5a 0 2024-06-03T01:20:52.432 INFO:tasks.workunit.client.1.smithi044.stdout:5/307: dwrite d2/d11/d3e/f40 [4194304,4194304] 0 2024-06-03T01:20:52.435 INFO:tasks.workunit.client.1.smithi044.stdout:1/397: mknod d0/d3/d14/c85 0 2024-06-03T01:20:52.436 INFO:tasks.workunit.client.1.smithi044.stdout:2/225: mkdir db/d1b/d2a/d2f/d4c 0 2024-06-03T01:20:52.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/428: creat d0/d21/d84/f93 x:0 0 0 2024-06-03T01:20:52.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/429: readlink d0/d3/d8/l4e 0 2024-06-03T01:20:52.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/430: write d0/d21/d27/f83 [789802,41192] 0 2024-06-03T01:20:52.438 INFO:tasks.workunit.client.1.smithi044.stdout:5/308: creat d2/d11/d12/f67 x:0 0 0 2024-06-03T01:20:52.438 INFO:tasks.workunit.client.1.smithi044.stdout:5/309: truncate d2/d11/d12/d2b/d4a/f5d 261385 0 2024-06-03T01:20:52.440 INFO:tasks.workunit.client.1.smithi044.stdout:1/398: creat d0/d3/d20/f86 x:0 0 0 2024-06-03T01:20:52.444 INFO:tasks.workunit.client.1.smithi044.stdout:2/226: mknod db/d1b/d2a/d2f/d4c/c4d 0 2024-06-03T01:20:52.445 INFO:tasks.workunit.client.1.smithi044.stdout:7/431: mknod d0/d3/d30/c94 0 2024-06-03T01:20:52.447 INFO:tasks.workunit.client.1.smithi044.stdout:5/310: rename l0 to d2/d11/d1e/d21/d24/l68 0 2024-06-03T01:20:52.448 INFO:tasks.workunit.client.1.smithi044.stdout:2/227: read db/d1b/d27/f32 [1305432,29120] 0 2024-06-03T01:20:52.448 INFO:tasks.workunit.client.1.smithi044.stdout:1/399: link d0/d3/c27 d0/d16/d64/c87 0 2024-06-03T01:20:52.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/311: rmdir d2/d11/d1e/d21/d24 39 2024-06-03T01:20:52.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/312: dread - d2/dc/f65 zero size 2024-06-03T01:20:52.452 INFO:tasks.workunit.client.1.smithi044.stdout:1/400: read d0/d3/d20/d7/f34 [911950,64290] 0 2024-06-03T01:20:52.453 INFO:tasks.workunit.client.1.smithi044.stdout:5/313: write d2/d11/d12/f29 [1386692,103962] 0 2024-06-03T01:20:52.458 INFO:tasks.workunit.client.1.smithi044.stdout:2/228: dread f7 [0,4194304] 0 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/229: mkdir db/d45/d4e 0 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/230: chown db/d1b/d27/f32 73 1 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/231: chown db/dd/f11 1735291 1 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/232: creat db/f4f x:0 0 0 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/233: read - db/d1b/d3f/f4a zero size 2024-06-03T01:20:52.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/234: chown db/d15/c35 46339 1 2024-06-03T01:20:52.465 INFO:tasks.workunit.client.1.smithi044.stdout:7/432: dwrite d0/d21/f3b [0,4194304] 0 2024-06-03T01:20:52.465 INFO:tasks.workunit.client.1.smithi044.stdout:7/433: chown d0/d21/d27/l60 14 1 2024-06-03T01:20:52.467 INFO:tasks.workunit.client.1.smithi044.stdout:7/434: unlink d0/d3/d8/l2b 0 2024-06-03T01:20:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:52 smithi044 ceph-mon[36956]: pgmap v255: 65 pgs: 65 active+clean; 413 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 806 op/s 2024-06-03T01:20:52.472 INFO:tasks.workunit.client.1.smithi044.stdout:7/435: rename d0/d21 to d0/d3/d8/d95 0 2024-06-03T01:20:52.475 INFO:tasks.workunit.client.1.smithi044.stdout:7/436: rename d0/d16/d72 to d0/d3/d8/d95/d27/d6e/d76/d96 0 2024-06-03T01:20:52.476 INFO:tasks.workunit.client.1.smithi044.stdout:7/437: creat d0/d3/d8/d95/d63/d89/d8b/f97 x:0 0 0 2024-06-03T01:20:52.477 INFO:tasks.workunit.client.1.smithi044.stdout:2/235: dwrite db/f12 [0,4194304] 0 2024-06-03T01:20:52.477 INFO:tasks.workunit.client.1.smithi044.stdout:2/236: fsync f7 0 2024-06-03T01:20:52.489 INFO:tasks.workunit.client.1.smithi044.stdout:2/237: dread db/f12 [0,4194304] 0 2024-06-03T01:20:52.489 INFO:tasks.workunit.client.1.smithi044.stdout:2/238: fsync db/d1b/d27/f39 0 2024-06-03T01:20:52.489 INFO:tasks.workunit.client.1.smithi044.stdout:2/239: creat db/d1b/d27/f50 x:0 0 0 2024-06-03T01:20:52.489 INFO:tasks.workunit.client.1.smithi044.stdout:7/438: dread d0/d3/d8/d95/d63/f6b [0,4194304] 0 2024-06-03T01:20:52.490 INFO:tasks.workunit.client.1.smithi044.stdout:7/439: stat d0/c13 0 2024-06-03T01:20:52.490 INFO:tasks.workunit.client.1.smithi044.stdout:2/240: symlink db/d1b/d2a/l51 0 2024-06-03T01:20:52.490 INFO:tasks.workunit.client.1.smithi044.stdout:7/440: symlink d0/d3/d8/d32/d51/l98 0 2024-06-03T01:20:52.491 INFO:tasks.workunit.client.1.smithi044.stdout:7/441: chown d0/d16/f4d 59 1 2024-06-03T01:20:52.491 INFO:tasks.workunit.client.1.smithi044.stdout:2/241: mkdir db/d1b/d2a/d52 0 2024-06-03T01:20:52.492 INFO:tasks.workunit.client.1.smithi044.stdout:2/242: mkdir db/d1b/d3f/d47/d53 0 2024-06-03T01:20:52.494 INFO:tasks.workunit.client.1.smithi044.stdout:2/243: unlink db/d1b/d2a/d2f/c31 0 2024-06-03T01:20:52.495 INFO:tasks.workunit.client.1.smithi044.stdout:2/244: mkdir db/d1b/d2a/d52/d54 0 2024-06-03T01:20:52.497 INFO:tasks.workunit.client.1.smithi044.stdout:2/245: truncate f2 2707794 0 2024-06-03T01:20:52.511 INFO:tasks.workunit.client.1.smithi044.stdout:7/442: dwrite d0/d16/f4d [0,4194304] 0 2024-06-03T01:20:52.511 INFO:tasks.workunit.client.1.smithi044.stdout:7/443: readlink d0/d3/d8/d95/l61 0 2024-06-03T01:20:52.528 INFO:tasks.workunit.client.1.smithi044.stdout:2/246: dwrite db/f26 [0,4194304] 0 2024-06-03T01:20:52.529 INFO:tasks.workunit.client.1.smithi044.stdout:2/247: mknod db/d1b/d3f/d47/d53/c55 0 2024-06-03T01:20:52.530 INFO:tasks.workunit.client.1.smithi044.stdout:2/248: creat db/d1b/d2a/d52/d54/f56 x:0 0 0 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:6/438: sync 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:4/264: sync 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:6/439: fdatasync d2/d23/d55/d5f/d53/f73 0 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:0/433: sync 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:8/375: sync 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:9/278: sync 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:8/376: chown d5/d20/d47/f71 1773928 1 2024-06-03T01:20:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:0/434: chown d3/d9/f53 0 1 2024-06-03T01:20:52.533 INFO:tasks.workunit.client.1.smithi044.stdout:4/265: creat d17/d23/d28/d39/f5c x:0 0 0 2024-06-03T01:20:52.533 INFO:tasks.workunit.client.1.smithi044.stdout:4/266: dread d17/d23/d28/f2c [0,4194304] 0 2024-06-03T01:20:52.535 INFO:tasks.workunit.client.1.smithi044.stdout:6/440: rename d2/d4/f7 to d2/d4/d3f/f9a 0 2024-06-03T01:20:52.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/279: truncate d6/dd/d51/d16/f1b 508622 0 2024-06-03T01:20:52.536 INFO:tasks.workunit.client.1.smithi044.stdout:0/435: symlink d3/d9/d55/d73/l9a 0 2024-06-03T01:20:52.537 INFO:tasks.workunit.client.1.smithi044.stdout:6/441: mkdir d2/d23/d30/d1e/d34/d39/d9b 0 2024-06-03T01:20:52.538 INFO:tasks.workunit.client.1.smithi044.stdout:0/436: rmdir d3/d9/d55/d5a/d29 39 2024-06-03T01:20:52.538 INFO:tasks.workunit.client.1.smithi044.stdout:2/249: dread db/f26 [0,4194304] 0 2024-06-03T01:20:52.538 INFO:tasks.workunit.client.1.smithi044.stdout:2/250: dread - db/d1b/f36 zero size 2024-06-03T01:20:52.538 INFO:tasks.workunit.client.1.smithi044.stdout:2/251: dread - db/d1b/d3f/f4a zero size 2024-06-03T01:20:52.540 INFO:tasks.workunit.client.1.smithi044.stdout:6/442: creat d2/d23/d30/d1e/d34/d39/d9b/f9c x:0 0 0 2024-06-03T01:20:52.548 INFO:tasks.workunit.client.1.smithi044.stdout:6/443: rename d2/d23/d30/d1e/d34/d39/d64/f6a to d2/d23/d30/d1e/d34/d39/d7d/f9d 0 2024-06-03T01:20:52.549 INFO:tasks.workunit.client.1.smithi044.stdout:0/437: rmdir d3/d5c 39 2024-06-03T01:20:52.550 INFO:tasks.workunit.client.1.smithi044.stdout:2/252: getdents db/d1b/d2a/d52/d54 0 2024-06-03T01:20:52.550 INFO:tasks.workunit.client.1.smithi044.stdout:2/253: dread - db/d1b/d27/f39 zero size 2024-06-03T01:20:52.553 INFO:tasks.workunit.client.1.smithi044.stdout:8/377: dwrite d5/d20/d2b/f35 [0,4194304] 0 2024-06-03T01:20:52.556 INFO:tasks.workunit.client.1.smithi044.stdout:8/378: getdents d5/d20/d2b 0 2024-06-03T01:20:52.557 INFO:tasks.workunit.client.1.smithi044.stdout:8/379: stat d5/d16/f3a 0 2024-06-03T01:20:52.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/380: mkdir d5/d55/d74 0 2024-06-03T01:20:52.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/381: write d5/d55/f5d [254561,51682] 0 2024-06-03T01:20:52.565 INFO:tasks.workunit.client.1.smithi044.stdout:0/438: dread d3/d9/f28 [0,4194304] 0 2024-06-03T01:20:52.565 INFO:tasks.workunit.client.1.smithi044.stdout:0/439: creat d3/d49/d21/f9b x:0 0 0 2024-06-03T01:20:52.566 INFO:tasks.workunit.client.1.smithi044.stdout:0/440: readlink d3/d49/d56/d62/l80 0 2024-06-03T01:20:52.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/254: read db/f48 [1466931,31643] 0 2024-06-03T01:20:52.586 INFO:tasks.workunit.client.1.smithi044.stdout:0/441: read d3/d9/f86 [869752,69900] 0 2024-06-03T01:20:52.589 INFO:tasks.workunit.client.1.smithi044.stdout:4/267: dwrite d17/d23/d28/d39/f4b [0,4194304] 0 2024-06-03T01:20:52.589 INFO:tasks.workunit.client.1.smithi044.stdout:4/268: creat d17/d23/d28/d39/f5d x:0 0 0 2024-06-03T01:20:52.594 INFO:tasks.workunit.client.1.smithi044.stdout:6/444: dwrite d2/d23/d30/d1e/d34/d39/d7d/f9d [0,4194304] 0 2024-06-03T01:20:52.598 INFO:tasks.workunit.client.1.smithi044.stdout:8/382: dwrite d5/d20/d2b/f48 [0,4194304] 0 2024-06-03T01:20:52.606 INFO:tasks.workunit.client.1.smithi044.stdout:4/269: write d17/d23/d28/d2b/f34 [531182,32957] 0 2024-06-03T01:20:52.621 INFO:tasks.workunit.client.1.smithi044.stdout:4/270: dread d17/f31 [0,4194304] 0 2024-06-03T01:20:52.622 INFO:tasks.workunit.client.1.smithi044.stdout:4/271: mknod d17/d23/d28/d39/c5e 0 2024-06-03T01:20:52.622 INFO:tasks.workunit.client.1.smithi044.stdout:4/272: fdatasync d17/f31 0 2024-06-03T01:20:52.623 INFO:tasks.workunit.client.1.smithi044.stdout:4/273: link d17/d23/d28/f2c d17/d23/d59/f5f 0 2024-06-03T01:20:52.628 INFO:tasks.workunit.client.1.smithi044.stdout:6/445: dwrite d2/d23/d55/d5f/f63 [0,4194304] 0 2024-06-03T01:20:52.631 INFO:tasks.workunit.client.1.smithi044.stdout:8/383: dwrite d5/d20/f69 [4194304,4194304] 0 2024-06-03T01:20:52.631 INFO:tasks.workunit.client.1.smithi044.stdout:2/255: dwrite db/f1f [0,4194304] 0 2024-06-03T01:20:52.632 INFO:tasks.workunit.client.1.smithi044.stdout:2/256: stat db/f3e 0 2024-06-03T01:20:52.632 INFO:tasks.workunit.client.1.smithi044.stdout:0/442: dwrite d3/f45 [0,4194304] 0 2024-06-03T01:20:52.633 INFO:tasks.workunit.client.1.smithi044.stdout:6/446: unlink d2/d23/d30/d1e/d34/d39/d7d/f95 0 2024-06-03T01:20:52.633 INFO:tasks.workunit.client.1.smithi044.stdout:6/447: creat d2/d23/d30/d1e/d34/d39/d64/f9e x:0 0 0 2024-06-03T01:20:52.633 INFO:tasks.workunit.client.1.smithi044.stdout:6/448: creat d2/d23/d30/d1e/d34/d39/d64/f9f x:0 0 0 2024-06-03T01:20:52.635 INFO:tasks.workunit.client.1.smithi044.stdout:8/384: creat d5/d55/d56/f75 x:0 0 0 2024-06-03T01:20:52.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/274: dwrite d17/d23/d28/d39/f5d [0,4194304] 0 2024-06-03T01:20:52.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/275: dread - d17/d1c/f3a zero size 2024-06-03T01:20:52.642 INFO:tasks.workunit.client.1.smithi044.stdout:2/257: write db/f12 [1739300,26393] 0 2024-06-03T01:20:52.642 INFO:tasks.workunit.client.1.smithi044.stdout:0/443: rename d3/d9/d36/f42 to d3/d9/d36/d82/f9c 0 2024-06-03T01:20:52.642 INFO:tasks.workunit.client.1.smithi044.stdout:0/444: dread - d3/d9/f4d zero size 2024-06-03T01:20:52.642 INFO:tasks.workunit.client.1.smithi044.stdout:2/258: write db/d1b/f36 [645185,56508] 0 2024-06-03T01:20:52.643 INFO:tasks.workunit.client.1.smithi044.stdout:0/445: dread d3/d9/d55/d5a/f34 [0,4194304] 0 2024-06-03T01:20:52.643 INFO:tasks.workunit.client.1.smithi044.stdout:0/446: creat d3/d9/d55/d73/f9d x:0 0 0 2024-06-03T01:20:52.644 INFO:tasks.workunit.client.1.smithi044.stdout:6/449: mkdir d2/d23/d44/d4d/da0 0 2024-06-03T01:20:52.644 INFO:tasks.workunit.client.1.smithi044.stdout:6/450: creat d2/d4/d3f/fa1 x:0 0 0 2024-06-03T01:20:52.646 INFO:tasks.workunit.client.1.smithi044.stdout:2/259: link db/d1b/d2a/d2f/d4c/c4d db/d1b/d3f/d47/c57 0 2024-06-03T01:20:52.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/447: unlink d3/d9/f1f 0 2024-06-03T01:20:52.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/448: write d3/d49/d21/f26 [4540817,31330] 0 2024-06-03T01:20:52.650 INFO:tasks.workunit.client.1.smithi044.stdout:8/385: dwrite d5/d16/f6e [0,4194304] 0 2024-06-03T01:20:52.651 INFO:tasks.workunit.client.1.smithi044.stdout:6/451: rename d2/d23/d44/c75 to d2/d23/d30/ca2 0 2024-06-03T01:20:52.651 INFO:tasks.workunit.client.1.smithi044.stdout:6/452: chown d2/d23/d44/d4d/c96 24666 1 2024-06-03T01:20:52.651 INFO:tasks.workunit.client.1.smithi044.stdout:6/453: stat d2/d4/d3f/f90 0 2024-06-03T01:20:52.654 INFO:tasks.workunit.client.1.smithi044.stdout:0/449: symlink d3/d9/d55/d5a/d29/d58/l9e 0 2024-06-03T01:20:52.656 INFO:tasks.workunit.client.1.smithi044.stdout:3/293: sync 2024-06-03T01:20:52.656 INFO:tasks.workunit.client.1.smithi044.stdout:3/294: creat df/d13/d39/d3d/d43/f5b x:0 0 0 2024-06-03T01:20:52.658 INFO:tasks.workunit.client.1.smithi044.stdout:8/386: write d5/f15 [7571896,16024] 0 2024-06-03T01:20:52.660 INFO:tasks.workunit.client.1.smithi044.stdout:6/454: dread d2/d23/f18 [0,4194304] 0 2024-06-03T01:20:52.660 INFO:tasks.workunit.client.1.smithi044.stdout:6/455: chown d2/d4/d3f/f46 788242 1 2024-06-03T01:20:52.660 INFO:tasks.workunit.client.1.smithi044.stdout:2/260: rename db/d1b/l1e to db/d1b/d2a/d2f/l58 0 2024-06-03T01:20:52.660 INFO:tasks.workunit.client.1.smithi044.stdout:2/261: creat db/d1b/d3f/f59 x:0 0 0 2024-06-03T01:20:52.660 INFO:tasks.workunit.client.1.smithi044.stdout:0/450: symlink d3/d49/d21/d94/l9f 0 2024-06-03T01:20:52.662 INFO:tasks.workunit.client.1.smithi044.stdout:2/262: dread db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:20:52.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/295: mkdir df/d13/d1b/d56/d5c 0 2024-06-03T01:20:52.666 INFO:tasks.workunit.client.1.smithi044.stdout:0/451: rename d3/d49/d56/d8e/d97 to d3/d9/d55/d73/da0 0 2024-06-03T01:20:52.666 INFO:tasks.workunit.client.1.smithi044.stdout:8/387: dread d5/f15 [0,4194304] 0 2024-06-03T01:20:52.669 INFO:tasks.workunit.client.1.smithi044.stdout:0/452: creat d3/d49/d56/d62/fa1 x:0 0 0 2024-06-03T01:20:52.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/388: rmdir d5/d16/d1d 39 2024-06-03T01:20:52.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/389: chown d5/d20/d2b/f35 61 1 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:1/401: sync 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:9/280: sync 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:5/314: sync 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:1/402: fdatasync d0/d3/d14/f5e 0 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:9/281: dread - d6/d26/f4a zero size 2024-06-03T01:20:52.673 INFO:tasks.workunit.client.1.smithi044.stdout:7/444: sync 2024-06-03T01:20:52.675 INFO:tasks.workunit.client.1.smithi044.stdout:8/390: link d5/d50/l51 d5/d20/d2b/l76 0 2024-06-03T01:20:52.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/445: creat d0/d3/d8/d95/d63/d7f/f99 x:0 0 0 2024-06-03T01:20:52.678 INFO:tasks.workunit.client.1.smithi044.stdout:2/263: rmdir db/d1b/d2a 39 2024-06-03T01:20:52.678 INFO:tasks.workunit.client.1.smithi044.stdout:1/403: dread d0/d3/d20/d7/d13/f3d [0,4194304] 0 2024-06-03T01:20:52.678 INFO:tasks.workunit.client.1.smithi044.stdout:6/456: dwrite d2/d23/d30/d1e/d38/d6c/f3b [0,4194304] 0 2024-06-03T01:20:52.678 INFO:tasks.workunit.client.1.smithi044.stdout:9/282: getdents d6/dd 0 2024-06-03T01:20:52.683 INFO:tasks.workunit.client.1.smithi044.stdout:7/446: write d0/d16/f3c [2116007,50752] 0 2024-06-03T01:20:52.684 INFO:tasks.workunit.client.1.smithi044.stdout:7/447: stat d0/d3/d8/d95/d27/f83 0 2024-06-03T01:20:52.684 INFO:tasks.workunit.client.1.smithi044.stdout:7/448: stat d0/d3/d8/d95/d27/l60 0 2024-06-03T01:20:52.685 INFO:tasks.workunit.client.1.smithi044.stdout:1/404: unlink d0/c12 0 2024-06-03T01:20:52.685 INFO:tasks.workunit.client.1.smithi044.stdout:1/405: stat d0/d3/d14/d81 0 2024-06-03T01:20:52.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/449: dread d0/d3/d8/d32/f48 [0,4194304] 0 2024-06-03T01:20:52.686 INFO:tasks.workunit.client.1.smithi044.stdout:1/406: write d0/d3/d20/d40/f55 [70280,47413] 0 2024-06-03T01:20:52.688 INFO:tasks.workunit.client.1.smithi044.stdout:7/450: mkdir d0/d3/d8/d32/d51/d9a 0 2024-06-03T01:20:52.690 INFO:tasks.workunit.client.1.smithi044.stdout:6/457: write d2/d23/d55/d5f/f6e [3444915,32729] 0 2024-06-03T01:20:52.690 INFO:tasks.workunit.client.1.smithi044.stdout:6/458: chown d2/d23/d55/d5f/d53/d5a/c8f 5 1 2024-06-03T01:20:52.690 INFO:tasks.workunit.client.1.smithi044.stdout:1/407: creat d0/d3/d20/d40/f88 x:0 0 0 2024-06-03T01:20:52.696 INFO:tasks.workunit.client.1.smithi044.stdout:6/459: rename d2/d4/d3f/f46 to d2/d23/d30/d1e/d38/d6c/fa3 0 2024-06-03T01:20:52.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/453: dwrite d3/d9/f1d [0,4194304] 0 2024-06-03T01:20:52.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/454: write d3/d49/d20/d25/f7d [837420,23877] 0 2024-06-03T01:20:52.697 INFO:tasks.workunit.client.1.smithi044.stdout:5/315: dwrite d2/dc/d28/d4c/f4e [0,4194304] 0 2024-06-03T01:20:52.697 INFO:tasks.workunit.client.1.smithi044.stdout:5/316: dread - d2/d11/d12/d2b/d44/f5f zero size 2024-06-03T01:20:52.700 INFO:tasks.workunit.client.1.smithi044.stdout:0/455: rename d3/d9/f2f to d3/d5c/fa2 0 2024-06-03T01:20:52.700 INFO:tasks.workunit.client.1.smithi044.stdout:9/283: dwrite d6/dd/d51/db/f32 [0,4194304] 0 2024-06-03T01:20:52.701 INFO:tasks.workunit.client.1.smithi044.stdout:9/284: readlink d6/dd/d51/l35 0 2024-06-03T01:20:52.703 INFO:tasks.workunit.client.1.smithi044.stdout:0/456: mkdir d3/d49/d21/d94/da3 0 2024-06-03T01:20:52.703 INFO:tasks.workunit.client.1.smithi044.stdout:0/457: write d3/f69 [1580816,68937] 0 2024-06-03T01:20:52.705 INFO:tasks.workunit.client.1.smithi044.stdout:9/285: symlink d6/dd/d51/d16/d27/l59 0 2024-06-03T01:20:52.705 INFO:tasks.workunit.client.1.smithi044.stdout:9/286: truncate d6/dd/d51/d46/f48 354406 0 2024-06-03T01:20:52.705 INFO:tasks.workunit.client.1.smithi044.stdout:9/287: write d6/dd/f36 [1745159,14355] 0 2024-06-03T01:20:52.707 INFO:tasks.workunit.client.1.smithi044.stdout:7/451: dwrite d0/fd [0,4194304] 0 2024-06-03T01:20:52.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/458: getdents d3/d9/d36/d82 0 2024-06-03T01:20:52.708 INFO:tasks.workunit.client.1.smithi044.stdout:5/317: write d2/f8 [1602073,112099] 0 2024-06-03T01:20:52.708 INFO:tasks.workunit.client.1.smithi044.stdout:9/288: dread d6/dd/d51/d16/f1b [0,4194304] 0 2024-06-03T01:20:52.709 INFO:tasks.workunit.client.1.smithi044.stdout:0/459: symlink d3/d9/d36/la4 0 2024-06-03T01:20:52.710 INFO:tasks.workunit.client.1.smithi044.stdout:9/289: creat d6/dd/d51/d16/f5a x:0 0 0 2024-06-03T01:20:52.711 INFO:tasks.workunit.client.1.smithi044.stdout:9/290: dread - d6/dd/d51/d3b/f47 zero size 2024-06-03T01:20:52.711 INFO:tasks.workunit.client.1.smithi044.stdout:9/291: chown d6/f44 1889410 1 2024-06-03T01:20:52.711 INFO:tasks.workunit.client.1.smithi044.stdout:0/460: mknod d3/d6b/d95/ca5 0 2024-06-03T01:20:52.712 INFO:tasks.workunit.client.1.smithi044.stdout:0/461: getdents d3/d9/d55/d5a/d29/d58 0 2024-06-03T01:20:52.712 INFO:tasks.workunit.client.1.smithi044.stdout:0/462: stat d3/d49/d21/f76 0 2024-06-03T01:20:52.715 INFO:tasks.workunit.client.1.smithi044.stdout:6/460: dwrite d2/d23/d30/f1c [4194304,4194304] 0 2024-06-03T01:20:52.716 INFO:tasks.workunit.client.1.smithi044.stdout:7/452: dread d0/d3/d8/d95/f53 [0,4194304] 0 2024-06-03T01:20:52.720 INFO:tasks.workunit.client.1.smithi044.stdout:6/461: mkdir d2/d23/d44/d4d/da0/da4 0 2024-06-03T01:20:52.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/462: truncate d2/d23/d44/f54 983773 0 2024-06-03T01:20:52.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/463: creat d2/d23/d30/d1e/d38/fa5 x:0 0 0 2024-06-03T01:20:52.721 INFO:tasks.workunit.client.1.smithi044.stdout:7/453: creat d0/d3/d8/d95/d27/d6e/f9b x:0 0 0 2024-06-03T01:20:52.724 INFO:tasks.workunit.client.1.smithi044.stdout:6/464: link d2/d23/d30/f2a d2/d4/d3f/fa6 0 2024-06-03T01:20:52.725 INFO:tasks.workunit.client.1.smithi044.stdout:0/463: rename d3/d49 to d3/da6 0 2024-06-03T01:20:52.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/318: truncate d2/f8 5026842 0 2024-06-03T01:20:52.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/319: dread - d2/dc/d28/d4c/f61 zero size 2024-06-03T01:20:52.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/320: write d2/fa [1330474,129457] 0 2024-06-03T01:20:52.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/321: chown d2/dc/f65 430086623 1 2024-06-03T01:20:52.741 INFO:tasks.workunit.client.1.smithi044.stdout:0/464: dwrite d3/da6/d20/d25/f7d [0,4194304] 0 2024-06-03T01:20:52.742 INFO:tasks.workunit.client.1.smithi044.stdout:0/465: write d3/da6/d56/f60 [230506,85347] 0 2024-06-03T01:20:52.744 INFO:tasks.workunit.client.1.smithi044.stdout:0/466: unlink d3/f45 0 2024-06-03T01:20:52.745 INFO:tasks.workunit.client.1.smithi044.stdout:0/467: getdents d3/d9 0 2024-06-03T01:20:52.747 INFO:tasks.workunit.client.1.smithi044.stdout:0/468: getdents d3/d9/d55/d5a 0 2024-06-03T01:20:52.748 INFO:tasks.workunit.client.1.smithi044.stdout:0/469: symlink d3/d9/d36/d82/la7 0 2024-06-03T01:20:52.749 INFO:tasks.workunit.client.1.smithi044.stdout:0/470: mknod d3/da6/d21/d94/ca8 0 2024-06-03T01:20:52.853 INFO:tasks.workunit.client.1.smithi044.stdout:9/292: dwrite d6/dd/f17 [0,4194304] 0 2024-06-03T01:20:52.858 INFO:tasks.workunit.client.1.smithi044.stdout:6/465: dread d2/d23/d55/d5f/f6e [0,4194304] 0 2024-06-03T01:20:52.862 INFO:tasks.workunit.client.1.smithi044.stdout:9/293: truncate d6/dd/d51/f9 7210820 0 2024-06-03T01:20:52.863 INFO:tasks.workunit.client.1.smithi044.stdout:6/466: mkdir d2/d23/d30/d1e/d34/d39/d7d/da7 0 2024-06-03T01:20:52.865 INFO:tasks.workunit.client.1.smithi044.stdout:9/294: creat d6/dd/d51/d16/d4d/d55/f5b x:0 0 0 2024-06-03T01:20:52.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/467: mkdir d2/d23/d30/d1e/d34/d39/da8 0 2024-06-03T01:20:52.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/468: fdatasync d2/d23/f33 0 2024-06-03T01:20:52.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/469: creat d2/d23/d55/d5f/fa9 x:0 0 0 2024-06-03T01:20:52.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/470: chown d2/c97 1235754 1 2024-06-03T01:20:52.868 INFO:tasks.workunit.client.1.smithi044.stdout:6/471: creat d2/d23/d30/d1e/d34/d39/d7d/da7/faa x:0 0 0 2024-06-03T01:20:52.868 INFO:tasks.workunit.client.1.smithi044.stdout:6/472: dread - d2/d23/d30/d1e/d38/f72 zero size 2024-06-03T01:20:52.868 INFO:tasks.workunit.client.1.smithi044.stdout:6/473: chown d2/d23/d30/d1e/d38/d6c/f21 6580 1 2024-06-03T01:20:52.869 INFO:tasks.workunit.client.1.smithi044.stdout:6/474: unlink d2/d23/d30/f19 0 2024-06-03T01:20:52.869 INFO:tasks.workunit.client.1.smithi044.stdout:6/475: write d2/d23/d30/f2a [2263810,84830] 0 2024-06-03T01:20:52.870 INFO:tasks.workunit.client.1.smithi044.stdout:6/476: creat d2/d23/d30/d1e/d38/d6c/d5c/fab x:0 0 0 2024-06-03T01:20:52.871 INFO:tasks.workunit.client.1.smithi044.stdout:6/477: creat d2/d23/d30/d1e/d34/d39/d7d/fac x:0 0 0 2024-06-03T01:20:52.872 INFO:tasks.workunit.client.1.smithi044.stdout:6/478: getdents d2/d6 0 2024-06-03T01:20:52.872 INFO:tasks.workunit.client.1.smithi044.stdout:6/479: dread - d2/d23/d30/f74 zero size 2024-06-03T01:20:52.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/480: getdents d2/d23/d30/d1e/d34/d39/d9b 0 2024-06-03T01:20:52.874 INFO:tasks.workunit.client.1.smithi044.stdout:4/276: sync 2024-06-03T01:20:52.875 INFO:tasks.workunit.client.1.smithi044.stdout:4/277: chown d17/d23/d28/f2c 237 1 2024-06-03T01:20:52.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/481: truncate d2/d23/f2e 3748705 0 2024-06-03T01:20:52.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/482: chown d2/d4 315 1 2024-06-03T01:20:52.876 INFO:tasks.workunit.client.1.smithi044.stdout:4/278: symlink d17/l60 0 2024-06-03T01:20:52.883 INFO:tasks.workunit.client.1.smithi044.stdout:6/483: getdents d2/d23/d30/d1e/d34/d39/d7d/da7 0 2024-06-03T01:20:52.885 INFO:tasks.workunit.client.1.smithi044.stdout:4/279: read fd [2564971,49938] 0 2024-06-03T01:20:52.885 INFO:tasks.workunit.client.1.smithi044.stdout:4/280: getdents d17/d23/d58 0 2024-06-03T01:20:52.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/484: rename d2/d23/d30/d1e/d34/d39/d64/f87 to d2/d23/d30/d1e/d34/d39/fad 0 2024-06-03T01:20:52.887 INFO:tasks.workunit.client.1.smithi044.stdout:4/281: stat d17/d23/l1f 0 2024-06-03T01:20:52.889 INFO:tasks.workunit.client.1.smithi044.stdout:4/282: mknod d17/d1c/c61 0 2024-06-03T01:20:52.889 INFO:tasks.workunit.client.1.smithi044.stdout:4/283: read d17/d23/d28/d2b/f21 [12030,127535] 0 2024-06-03T01:20:52.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/485: write d2/d23/d55/d5f/d53/d5a/f93 [2158152,9543] 0 2024-06-03T01:20:52.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/486: dread - d2/d23/d30/d1e/d34/d39/d7d/da7/faa zero size 2024-06-03T01:20:52.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/487: fsync d2/d23/d55/d5f/f62 0 2024-06-03T01:20:52.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/488: creat d2/d23/d44/d4d/da0/fae x:0 0 0 2024-06-03T01:20:52.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/489: chown d2/d23/d30/ca2 1026 1 2024-06-03T01:20:52.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/490: dread - d2/d23/d30/d1e/f47 zero size 2024-06-03T01:20:52.903 INFO:tasks.workunit.client.1.smithi044.stdout:6/491: unlink d2/d23/d30/d1e/f66 0 2024-06-03T01:20:52.904 INFO:tasks.workunit.client.1.smithi044.stdout:6/492: rmdir d2/d23/d30/d1e/d34/d39/d9b 39 2024-06-03T01:20:52.904 INFO:tasks.workunit.client.1.smithi044.stdout:6/493: write d2/d23/d55/d5f/fa9 [1021953,97972] 0 2024-06-03T01:20:52.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/284: dwrite d17/d1c/f3a [0,4194304] 0 2024-06-03T01:20:52.915 INFO:tasks.workunit.client.1.smithi044.stdout:6/494: write d2/d23/d30/f1c [7709395,82367] 0 2024-06-03T01:20:52.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/285: dwrite d17/d1c/f49 [0,4194304] 0 2024-06-03T01:20:52.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/286: link d17/d23/d28/d2b/f3d d17/d23/d28/d39/f62 0 2024-06-03T01:20:52.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/287: stat d17/d23/d28/f33 0 2024-06-03T01:20:52.932 INFO:tasks.workunit.client.1.smithi044.stdout:4/288: mkdir d17/d63 0 2024-06-03T01:20:52.933 INFO:tasks.workunit.client.1.smithi044.stdout:4/289: mknod d17/d23/d28/d39/c64 0 2024-06-03T01:20:52.942 INFO:tasks.workunit.client.1.smithi044.stdout:4/290: dread d17/d23/d28/d39/f4b [0,4194304] 0 2024-06-03T01:20:52.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/291: creat d17/d63/f65 x:0 0 0 2024-06-03T01:20:52.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/292: stat d17/d23/d28/d2b/f3d 0 2024-06-03T01:20:52.945 INFO:tasks.workunit.client.1.smithi044.stdout:4/293: symlink d17/d1c/l66 0 2024-06-03T01:20:52.945 INFO:tasks.workunit.client.1.smithi044.stdout:4/294: creat d17/d23/f67 x:0 0 0 2024-06-03T01:20:52.945 INFO:tasks.workunit.client.1.smithi044.stdout:4/295: creat d17/d23/f68 x:0 0 0 2024-06-03T01:20:52.948 INFO:tasks.workunit.client.1.smithi044.stdout:4/296: creat d17/f69 x:0 0 0 2024-06-03T01:20:52.959 INFO:tasks.workunit.client.1.smithi044.stdout:4/297: write d17/d23/f4d [1147245,34199] 0 2024-06-03T01:20:52.969 INFO:tasks.workunit.client.1.smithi044.stdout:6/495: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/f83 [0,4194304] 0 2024-06-03T01:20:52.974 INFO:tasks.workunit.client.1.smithi044.stdout:6/496: mknod d2/d4/d3f/caf 0 2024-06-03T01:20:52.974 INFO:tasks.workunit.client.1.smithi044.stdout:6/497: creat d2/d23/d44/d4d/da0/fb0 x:0 0 0 2024-06-03T01:20:52.975 INFO:tasks.workunit.client.1.smithi044.stdout:6/498: chown d2/d23/d30/d1e/d34/d39/d7d/da7/faa 63467070 1 2024-06-03T01:20:52.975 INFO:tasks.workunit.client.1.smithi044.stdout:4/298: dwrite d17/d23/f68 [0,4194304] 0 2024-06-03T01:20:52.977 INFO:tasks.workunit.client.1.smithi044.stdout:6/499: mkdir d2/d23/d30/d1e/d34/d39/d7d/da7/db1 0 2024-06-03T01:20:52.982 INFO:tasks.workunit.client.1.smithi044.stdout:4/299: mkdir d17/d23/d6a 0 2024-06-03T01:20:52.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/300: dread d17/d23/f4d [0,4194304] 0 2024-06-03T01:20:52.989 INFO:tasks.workunit.client.1.smithi044.stdout:4/301: getdents d17/d23/d28/d39 0 2024-06-03T01:20:52.990 INFO:tasks.workunit.client.1.smithi044.stdout:4/302: symlink d17/d23/d35/l6b 0 2024-06-03T01:20:52.991 INFO:tasks.workunit.client.1.smithi044.stdout:4/303: mknod d17/d23/d28/c6c 0 2024-06-03T01:20:52.993 INFO:tasks.workunit.client.1.smithi044.stdout:6/500: rename d2/d4 to d2/d23/d55/d5f/db2 0 2024-06-03T01:20:52.995 INFO:tasks.workunit.client.1.smithi044.stdout:6/501: symlink d2/d23/d30/d1e/d38/lb3 0 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/502: rename d2/d23/d55/d5f/d53/d5a to d2/d23/d30/d1e/d34/d39/d9b/db4 0 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:5/322: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:0/471: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:1/408: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/454: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/455: stat d0/f7c 0 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/296: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:1/409: fsync d0/d3/d20/d40/d45/d4b/d5a/f72 0 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:8/391: sync 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:1/410: dread - d0/d16/f3a zero size 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:8/392: chown d5/d16/f5e 5 1 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/456: truncate d0/d3/d8/d95/d27/d6e/d76/f80 277549 0 2024-06-03T01:20:53.000 INFO:tasks.workunit.client.1.smithi044.stdout:2/264: sync 2024-06-03T01:20:53.001 INFO:tasks.workunit.client.1.smithi044.stdout:4/304: write d17/d23/d28/d2b/f4a [925578,27759] 0 2024-06-03T01:20:53.001 INFO:tasks.workunit.client.1.smithi044.stdout:5/323: write d2/dc/f49 [497296,83885] 0 2024-06-03T01:20:53.001 INFO:tasks.workunit.client.1.smithi044.stdout:6/503: mkdir d2/d23/d55/d5f/db2/db5 0 2024-06-03T01:20:53.001 INFO:tasks.workunit.client.1.smithi044.stdout:6/504: write d2/d23/d30/d1e/d34/d39/d9b/db4/f93 [2428748,15271] 0 2024-06-03T01:20:53.003 INFO:tasks.workunit.client.1.smithi044.stdout:0/472: creat d3/da6/d56/d62/fa9 x:0 0 0 2024-06-03T01:20:53.004 INFO:tasks.workunit.client.1.smithi044.stdout:5/324: dread d2/fa [0,4194304] 0 2024-06-03T01:20:53.004 INFO:tasks.workunit.client.1.smithi044.stdout:5/325: fsync d2/d11/d1e/f51 0 2024-06-03T01:20:53.004 INFO:tasks.workunit.client.1.smithi044.stdout:1/411: mknod d0/d3/d20/d40/d45/d4b/d5c/c89 0 2024-06-03T01:20:53.005 INFO:tasks.workunit.client.1.smithi044.stdout:8/393: mknod d5/d10/d5f/c77 0 2024-06-03T01:20:53.005 INFO:tasks.workunit.client.1.smithi044.stdout:8/394: readlink d5/d16/l1e 0 2024-06-03T01:20:53.006 INFO:tasks.workunit.client.1.smithi044.stdout:7/457: truncate d0/d3/d8/d32/f38 3366850 0 2024-06-03T01:20:53.007 INFO:tasks.workunit.client.1.smithi044.stdout:3/297: symlink df/d1e/l5d 0 2024-06-03T01:20:53.010 INFO:tasks.workunit.client.1.smithi044.stdout:4/305: creat d17/d23/d59/f6d x:0 0 0 2024-06-03T01:20:53.011 INFO:tasks.workunit.client.1.smithi044.stdout:5/326: creat d2/d11/d1e/d21/d24/f69 x:0 0 0 2024-06-03T01:20:53.012 INFO:tasks.workunit.client.1.smithi044.stdout:0/473: dread d3/d9/f1d [0,4194304] 0 2024-06-03T01:20:53.012 INFO:tasks.workunit.client.1.smithi044.stdout:0/474: stat d3/d5c 0 2024-06-03T01:20:53.013 INFO:tasks.workunit.client.1.smithi044.stdout:8/395: creat d5/d55/d74/f78 x:0 0 0 2024-06-03T01:20:53.015 INFO:tasks.workunit.client.1.smithi044.stdout:3/298: truncate df/d13/f25 546622 0 2024-06-03T01:20:53.015 INFO:tasks.workunit.client.1.smithi044.stdout:3/299: creat df/d13/d1b/f5e x:0 0 0 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:6/505: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/f81 [0,4194304] 0 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:3/300: creat df/d13/d1b/d56/d5c/f5f x:0 0 0 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:8/396: mkdir d5/d50/d79 0 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:7/458: rename d0/d3/d8/d95/d27 to d0/d3/d8/d95/d63/d7f/d9c 0 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:7/459: chown d0/d3/d8/d32/d33/f42 3758069 1 2024-06-03T01:20:53.023 INFO:tasks.workunit.client.1.smithi044.stdout:8/397: creat d5/d16/d1d/d32/f7a x:0 0 0 2024-06-03T01:20:53.024 INFO:tasks.workunit.client.1.smithi044.stdout:0/475: read d3/d9/f53 [3808677,24971] 0 2024-06-03T01:20:53.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/265: dwrite db/d1b/d27/f50 [0,4194304] 0 2024-06-03T01:20:53.026 INFO:tasks.workunit.client.1.smithi044.stdout:5/327: write d2/d11/d1e/f51 [6968907,9072] 0 2024-06-03T01:20:53.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/306: unlink c13 0 2024-06-03T01:20:53.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/307: readlink l9 0 2024-06-03T01:20:53.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/308: stat d17/l5a 0 2024-06-03T01:20:53.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/309: readlink d17/d23/d28/l38 0 2024-06-03T01:20:53.028 INFO:tasks.workunit.client.1.smithi044.stdout:3/301: mknod df/d13/d1b/d4b/c60 0 2024-06-03T01:20:53.029 INFO:tasks.workunit.client.1.smithi044.stdout:6/506: dread d2/d23/d30/d1e/d38/d6c/fa3 [4194304,4194304] 0 2024-06-03T01:20:53.034 INFO:tasks.workunit.client.1.smithi044.stdout:0/476: creat d3/da6/d20/faa x:0 0 0 2024-06-03T01:20:53.037 INFO:tasks.workunit.client.1.smithi044.stdout:2/266: dread db/d1b/f22 [0,4194304] 0 2024-06-03T01:20:53.037 INFO:tasks.workunit.client.1.smithi044.stdout:2/267: dread - db/d1b/d2a/f43 zero size 2024-06-03T01:20:53.038 INFO:tasks.workunit.client.1.smithi044.stdout:1/412: dwrite d0/d3/d20/d7/d13/d2d/f79 [0,4194304] 0 2024-06-03T01:20:53.039 INFO:tasks.workunit.client.1.smithi044.stdout:5/328: creat d2/d11/f6a x:0 0 0 2024-06-03T01:20:53.040 INFO:tasks.workunit.client.1.smithi044.stdout:5/329: readlink d2/d11/l4d 0 2024-06-03T01:20:53.042 INFO:tasks.workunit.client.1.smithi044.stdout:4/310: rmdir d17/d23/d55 0 2024-06-03T01:20:53.044 INFO:tasks.workunit.client.1.smithi044.stdout:6/507: rmdir d2/d23/d55/d5f/db2/db5 0 2024-06-03T01:20:53.045 INFO:tasks.workunit.client.1.smithi044.stdout:7/460: mknod d0/d3/d8/d95/d63/d7f/d9c/c9d 0 2024-06-03T01:20:53.046 INFO:tasks.workunit.client.1.smithi044.stdout:0/477: mknod d3/d9/d55/d5a/cab 0 2024-06-03T01:20:53.046 INFO:tasks.workunit.client.1.smithi044.stdout:0/478: truncate d3/d9/d55/d5a/f38 1317590 0 2024-06-03T01:20:53.047 INFO:tasks.workunit.client.1.smithi044.stdout:2/268: mknod db/d45/d4e/c5a 0 2024-06-03T01:20:53.049 INFO:tasks.workunit.client.1.smithi044.stdout:8/398: dwrite d5/f2d [0,4194304] 0 2024-06-03T01:20:53.049 INFO:tasks.workunit.client.1.smithi044.stdout:8/399: write d5/d16/d1d/d59/f6f [8643,75064] 0 2024-06-03T01:20:53.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/400: stat d5/d50 0 2024-06-03T01:20:53.050 INFO:tasks.workunit.client.1.smithi044.stdout:1/413: creat d0/d3/d20/d40/d45/d4b/d5c/f8a x:0 0 0 2024-06-03T01:20:53.050 INFO:tasks.workunit.client.1.smithi044.stdout:1/414: fdatasync d0/d3/d14/f5e 0 2024-06-03T01:20:53.050 INFO:tasks.workunit.client.1.smithi044.stdout:5/330: creat d2/d11/d12/d2b/d4a/d57/f6b x:0 0 0 2024-06-03T01:20:53.050 INFO:tasks.workunit.client.1.smithi044.stdout:5/331: creat d2/d11/d12/f6c x:0 0 0 2024-06-03T01:20:53.051 INFO:tasks.workunit.client.1.smithi044.stdout:5/332: write d2/dc/f22 [3441119,125079] 0 2024-06-03T01:20:53.051 INFO:tasks.workunit.client.1.smithi044.stdout:4/311: mknod d17/d23/d59/c6e 0 2024-06-03T01:20:53.059 INFO:tasks.workunit.client.1.smithi044.stdout:0/479: dread d3/da6/d20/f71 [0,4194304] 0 2024-06-03T01:20:53.059 INFO:tasks.workunit.client.1.smithi044.stdout:0/480: getdents d3/d9/d55/d73/da0 0 2024-06-03T01:20:53.059 INFO:tasks.workunit.client.1.smithi044.stdout:0/481: truncate d3/f3a 722619 0 2024-06-03T01:20:53.059 INFO:tasks.workunit.client.1.smithi044.stdout:3/302: rmdir df/d13 39 2024-06-03T01:20:53.060 INFO:tasks.workunit.client.1.smithi044.stdout:3/303: dread - df/d13/d39/d3d/d43/f55 zero size 2024-06-03T01:20:53.060 INFO:tasks.workunit.client.1.smithi044.stdout:3/304: chown df/d1e/d2a 92742275 1 2024-06-03T01:20:53.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/508: dwrite d2/d23/d30/d1e/f47 [0,4194304] 0 2024-06-03T01:20:53.062 INFO:tasks.workunit.client.1.smithi044.stdout:7/461: creat d0/d3/d8/d32/d51/f9e x:0 0 0 2024-06-03T01:20:53.062 INFO:tasks.workunit.client.1.smithi044.stdout:0/482: dread d3/d9/d55/d5a/f34 [0,4194304] 0 2024-06-03T01:20:53.064 INFO:tasks.workunit.client.1.smithi044.stdout:2/269: mknod db/d1b/d2a/d2f/c5b 0 2024-06-03T01:20:53.071 INFO:tasks.workunit.client.1.smithi044.stdout:1/415: rmdir d0/d3/d14/d30 39 2024-06-03T01:20:53.075 INFO:tasks.workunit.client.1.smithi044.stdout:6/509: symlink d2/d23/d30/d1e/d34/d39/d64/lb6 0 2024-06-03T01:20:53.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/333: dwrite d2/d11/d12/d2b/f34 [0,4194304] 0 2024-06-03T01:20:53.076 INFO:tasks.workunit.client.1.smithi044.stdout:7/462: creat d0/f9f x:0 0 0 2024-06-03T01:20:53.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/334: dread d2/d11/d12/f25 [0,4194304] 0 2024-06-03T01:20:53.076 INFO:tasks.workunit.client.1.smithi044.stdout:2/270: creat db/d15/d49/f5c x:0 0 0 2024-06-03T01:20:53.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/510: mknod d2/d23/d30/d1e/d34/d39/d7d/da7/db1/cb7 0 2024-06-03T01:20:53.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/511: chown d2/d23/d30/f1c 5032350 1 2024-06-03T01:20:53.081 INFO:tasks.workunit.client.1.smithi044.stdout:6/512: getdents d2 0 2024-06-03T01:20:53.089 INFO:tasks.workunit.client.1.smithi044.stdout:1/416: dread d0/d3/d20/d7/d13/f4a [0,4194304] 0 2024-06-03T01:20:53.089 INFO:tasks.workunit.client.1.smithi044.stdout:1/417: write d0/d16/f26 [2968547,68592] 0 2024-06-03T01:20:53.089 INFO:tasks.workunit.client.1.smithi044.stdout:1/418: readlink d0/l10 0 2024-06-03T01:20:53.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/419: symlink d0/d3/d20/d7/d13/d2d/l8b 0 2024-06-03T01:20:53.092 INFO:tasks.workunit.client.1.smithi044.stdout:1/420: chown d0/d3/d14/d30/c3b 911 1 2024-06-03T01:20:53.094 INFO:tasks.workunit.client.1.smithi044.stdout:1/421: mkdir d0/d3/d20/d40/d45/d6f/d8c 0 2024-06-03T01:20:53.109 INFO:tasks.workunit.client.1.smithi044.stdout:1/422: read d0/d3/d20/d7/d4e/f78 [2147634,93964] 0 2024-06-03T01:20:53.109 INFO:tasks.workunit.client.1.smithi044.stdout:3/305: dwrite df/d13/d1b/d4b/f4f [0,4194304] 0 2024-06-03T01:20:53.111 INFO:tasks.workunit.client.1.smithi044.stdout:1/423: creat d0/d3/d14/f8d x:0 0 0 2024-06-03T01:20:53.113 INFO:tasks.workunit.client.1.smithi044.stdout:1/424: creat d0/d3/d20/d7/d4e/f8e x:0 0 0 2024-06-03T01:20:53.113 INFO:tasks.workunit.client.1.smithi044.stdout:3/306: write df/d13/d1b/f20 [113778,30784] 0 2024-06-03T01:20:53.113 INFO:tasks.workunit.client.1.smithi044.stdout:3/307: write df/d13/f24 [4532586,74466] 0 2024-06-03T01:20:53.113 INFO:tasks.workunit.client.1.smithi044.stdout:7/463: dwrite d0/d3/d8/d32/d33/f42 [0,4194304] 0 2024-06-03T01:20:53.114 INFO:tasks.workunit.client.1.smithi044.stdout:3/308: link df/d13/d1b/d4b/c60 df/d13/d1b/d4b/c61 0 2024-06-03T01:20:53.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/464: mkdir d0/d3/d8/da0 0 2024-06-03T01:20:53.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/465: stat d0/d3/d8/d95/f46 0 2024-06-03T01:20:53.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/466: creat d0/d3/d8/d95/d63/d7f/fa1 x:0 0 0 2024-06-03T01:20:53.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/467: chown d0/d3/d8/d95/d84/l85 165140513 1 2024-06-03T01:20:53.117 INFO:tasks.workunit.client.1.smithi044.stdout:7/468: dread - d0/d3/d8/d95/d63/d7f/f99 zero size 2024-06-03T01:20:53.117 INFO:tasks.workunit.client.1.smithi044.stdout:7/469: fdatasync d0/d3/d8/f10 0 2024-06-03T01:20:53.117 INFO:tasks.workunit.client.1.smithi044.stdout:7/470: stat d0/d3/d8/d32 0 2024-06-03T01:20:53.120 INFO:tasks.workunit.client.1.smithi044.stdout:0/483: dwrite d3/f6e [0,4194304] 0 2024-06-03T01:20:53.120 INFO:tasks.workunit.client.1.smithi044.stdout:5/335: dwrite d2/d11/d1e/d3b/f5e [0,4194304] 0 2024-06-03T01:20:53.121 INFO:tasks.workunit.client.1.smithi044.stdout:9/295: sync 2024-06-03T01:20:53.123 INFO:tasks.workunit.client.1.smithi044.stdout:8/401: dwrite d5/ff [4194304,4194304] 0 2024-06-03T01:20:53.123 INFO:tasks.workunit.client.1.smithi044.stdout:3/309: dread f9 [4194304,4194304] 0 2024-06-03T01:20:53.123 INFO:tasks.workunit.client.1.smithi044.stdout:8/402: write d5/f15 [2723885,10149] 0 2024-06-03T01:20:53.124 INFO:tasks.workunit.client.1.smithi044.stdout:8/403: write d5/d55/d56/d5c/f72 [580963,71588] 0 2024-06-03T01:20:53.125 INFO:tasks.workunit.client.1.smithi044.stdout:0/484: creat d3/da6/fac x:0 0 0 2024-06-03T01:20:53.126 INFO:tasks.workunit.client.1.smithi044.stdout:2/271: dwrite db/dd/fe [0,4194304] 0 2024-06-03T01:20:53.128 INFO:tasks.workunit.client.1.smithi044.stdout:6/513: dwrite d2/d23/d55/d5f/d53/f61 [0,4194304] 0 2024-06-03T01:20:53.129 INFO:tasks.workunit.client.1.smithi044.stdout:1/425: dwrite d0/d16/d64/d6a/f74 [0,4194304] 0 2024-06-03T01:20:53.130 INFO:tasks.workunit.client.1.smithi044.stdout:7/471: read d0/d3/d8/d95/d63/f6b [1458998,27960] 0 2024-06-03T01:20:53.131 INFO:tasks.workunit.client.1.smithi044.stdout:0/485: mkdir d3/da6/d21/d94/da3/dad 0 2024-06-03T01:20:53.131 INFO:tasks.workunit.client.1.smithi044.stdout:9/296: creat d6/dd/d51/d46/f5c x:0 0 0 2024-06-03T01:20:53.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/336: read d2/d11/d12/f1b [404836,95584] 0 2024-06-03T01:20:53.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/337: fsync d2/dc/f62 0 2024-06-03T01:20:53.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/338: write d2/d11/d12/d2b/d44/f5c [1539553,77836] 0 2024-06-03T01:20:53.135 INFO:tasks.workunit.client.1.smithi044.stdout:2/272: dread db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:20:53.142 INFO:tasks.workunit.client.1.smithi044.stdout:7/472: truncate d0/d3/d8/d32/f48 331976 0 2024-06-03T01:20:53.142 INFO:tasks.workunit.client.1.smithi044.stdout:5/339: rename d2/lf to d2/d11/d1e/l6d 0 2024-06-03T01:20:53.143 INFO:tasks.workunit.client.1.smithi044.stdout:7/473: readlink d0/l15 0 2024-06-03T01:20:53.143 INFO:tasks.workunit.client.1.smithi044.stdout:2/273: mkdir db/d1b/d5d 0 2024-06-03T01:20:53.146 INFO:tasks.workunit.client.1.smithi044.stdout:5/340: read d2/d11/d12/d2b/d4a/d57/f50 [668512,65175] 0 2024-06-03T01:20:53.149 INFO:tasks.workunit.client.1.smithi044.stdout:5/341: link d2/d11/d12/d2b/d44/f5c d2/d11/d1e/d2f/f6e 0 2024-06-03T01:20:53.153 INFO:tasks.workunit.client.1.smithi044.stdout:5/342: rename d2/d11/d12/l48 to d2/d11/d12/d2b/l6f 0 2024-06-03T01:20:53.154 INFO:tasks.workunit.client.1.smithi044.stdout:5/343: rmdir d2/dc/d28 39 2024-06-03T01:20:53.157 INFO:tasks.workunit.client.1.smithi044.stdout:5/344: truncate d2/d11/d12/d2b/f58 464015 0 2024-06-03T01:20:53.157 INFO:tasks.workunit.client.1.smithi044.stdout:5/345: rename d2 to d2/d11/d12/d70 22 2024-06-03T01:20:53.159 INFO:tasks.workunit.client.1.smithi044.stdout:5/346: mknod d2/d11/d12/c71 0 2024-06-03T01:20:53.159 INFO:tasks.workunit.client.1.smithi044.stdout:8/404: dwrite d5/d20/f3d [0,4194304] 0 2024-06-03T01:20:53.160 INFO:tasks.workunit.client.1.smithi044.stdout:8/405: chown d5/d55/d56 353 1 2024-06-03T01:20:53.160 INFO:tasks.workunit.client.1.smithi044.stdout:7/474: dread d0/d3/d8/d32/f6c [0,4194304] 0 2024-06-03T01:20:53.160 INFO:tasks.workunit.client.1.smithi044.stdout:5/347: creat d2/d11/d12/d2b/f72 x:0 0 0 2024-06-03T01:20:53.161 INFO:tasks.workunit.client.1.smithi044.stdout:1/426: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-03T01:20:53.161 INFO:tasks.workunit.client.1.smithi044.stdout:1/427: chown d0/d3/d14/d81/f56 89782647 1 2024-06-03T01:20:53.163 INFO:tasks.workunit.client.1.smithi044.stdout:9/297: dwrite d6/dd/d51/db/f22 [0,4194304] 0 2024-06-03T01:20:53.163 INFO:tasks.workunit.client.1.smithi044.stdout:9/298: read - d6/dd/d45/f56 zero size 2024-06-03T01:20:53.164 INFO:tasks.workunit.client.1.smithi044.stdout:8/406: unlink d5/d20/f4a 0 2024-06-03T01:20:53.165 INFO:tasks.workunit.client.1.smithi044.stdout:1/428: getdents d0/d3/d20/d7/d13 0 2024-06-03T01:20:53.169 INFO:tasks.workunit.client.1.smithi044.stdout:0/486: dread d3/da6/f8b [0,4194304] 0 2024-06-03T01:20:53.169 INFO:tasks.workunit.client.1.smithi044.stdout:3/310: dwrite df/d1e/f37 [0,4194304] 0 2024-06-03T01:20:53.171 INFO:tasks.workunit.client.1.smithi044.stdout:9/299: creat d6/dd/d51/d3b/f5d x:0 0 0 2024-06-03T01:20:53.171 INFO:tasks.workunit.client.1.smithi044.stdout:9/300: fdatasync d6/dd/d51/db/f32 0 2024-06-03T01:20:53.172 INFO:tasks.workunit.client.1.smithi044.stdout:8/407: rmdir d5/d20/d47/d64 39 2024-06-03T01:20:53.172 INFO:tasks.workunit.client.1.smithi044.stdout:8/408: fsync d5/d20/d33/f63 0 2024-06-03T01:20:53.174 INFO:tasks.workunit.client.1.smithi044.stdout:1/429: mknod d0/d3/d20/d40/d45/d4b/c8f 0 2024-06-03T01:20:53.175 INFO:tasks.workunit.client.1.smithi044.stdout:7/475: dread d0/d3/d8/f58 [0,4194304] 0 2024-06-03T01:20:53.175 INFO:tasks.workunit.client.1.smithi044.stdout:0/487: dread d3/da6/d21/f76 [0,4194304] 0 2024-06-03T01:20:53.175 INFO:tasks.workunit.client.1.smithi044.stdout:7/476: creat d0/d3/d8/d32/fa2 x:0 0 0 2024-06-03T01:20:53.177 INFO:tasks.workunit.client.1.smithi044.stdout:2/274: write db/f1f [2848140,25607] 0 2024-06-03T01:20:53.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/514: dwrite d2/d23/d30/f1f [4194304,4194304] 0 2024-06-03T01:20:53.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/515: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/fb8 x:0 0 0 2024-06-03T01:20:53.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/516: creat d2/d23/d55/d5f/d53/fb9 x:0 0 0 2024-06-03T01:20:53.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/517: stat d2/d23/d30/f2a 0 2024-06-03T01:20:53.178 INFO:tasks.workunit.client.1.smithi044.stdout:3/311: truncate df/f19 1515896 0 2024-06-03T01:20:53.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/301: mkdir d6/dd/d51/d16/d27/d5e 0 2024-06-03T01:20:53.180 INFO:tasks.workunit.client.1.smithi044.stdout:1/430: creat d0/d3/d20/d40/d45/d4b/f90 x:0 0 0 2024-06-03T01:20:53.182 INFO:tasks.workunit.client.1.smithi044.stdout:3/312: write df/f1c [3560058,121045] 0 2024-06-03T01:20:53.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/488: mkdir d3/d6b/d95/dae 0 2024-06-03T01:20:53.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/489: write d3/f3a [1627276,68815] 0 2024-06-03T01:20:53.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/490: truncate d3/da6/d20/f5b 673157 0 2024-06-03T01:20:53.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/491: fsync d3/f19 0 2024-06-03T01:20:53.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/492: creat d3/da6/d20/faf x:0 0 0 2024-06-03T01:20:53.184 INFO:tasks.workunit.client.1.smithi044.stdout:7/477: getdents d0/d3/d8/d95/d63/d7f/d9c/d6e 0 2024-06-03T01:20:53.184 INFO:tasks.workunit.client.1.smithi044.stdout:7/478: chown d0/d3/d8/d95/f28 11097 1 2024-06-03T01:20:53.185 INFO:tasks.workunit.client.1.smithi044.stdout:7/479: write d0/f9f [451714,51271] 0 2024-06-03T01:20:53.185 INFO:tasks.workunit.client.1.smithi044.stdout:7/480: read d0/d3/d8/f6d [141787,126702] 0 2024-06-03T01:20:53.194 INFO:tasks.workunit.client.1.smithi044.stdout:5/348: dwrite d2/d11/d12/f6c [0,4194304] 0 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:2/275: dwrite db/d1b/d2a/f33 [0,4194304] 0 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:8/409: dwrite d5/d20/d33/f3f [0,4194304] 0 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:2/276: fsync f2 0 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:8/410: creat d5/d16/d1d/d6b/d6d/f7b x:0 0 0 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:8/411: dread - d5/d16/d1d/d32/f62 zero size 2024-06-03T01:20:53.196 INFO:tasks.workunit.client.1.smithi044.stdout:8/412: write d5/d16/d1d/f42 [709300,71965] 0 2024-06-03T01:20:53.202 INFO:tasks.workunit.client.1.smithi044.stdout:6/518: mkdir d2/d23/d30/d1e/d34/d39/d9b/db4/dba 0 2024-06-03T01:20:53.204 INFO:tasks.workunit.client.1.smithi044.stdout:9/302: mkdir d6/dd/d51/d16/d5f 0 2024-06-03T01:20:53.207 INFO:tasks.workunit.client.1.smithi044.stdout:1/431: truncate d0/d3/f62 416584 0 2024-06-03T01:20:53.210 INFO:tasks.workunit.client.1.smithi044.stdout:3/313: mknod df/d13/d1b/d56/d5c/c62 0 2024-06-03T01:20:53.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/493: getdents d3/da6/d56 0 2024-06-03T01:20:53.214 INFO:tasks.workunit.client.1.smithi044.stdout:7/481: unlink d0/d16/d23/c64 0 2024-06-03T01:20:53.214 INFO:tasks.workunit.client.1.smithi044.stdout:7/482: chown d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c 3 1 2024-06-03T01:20:53.219 INFO:tasks.workunit.client.1.smithi044.stdout:4/312: sync 2024-06-03T01:20:53.219 INFO:tasks.workunit.client.1.smithi044.stdout:4/313: chown d17/d23/c44 367 1 2024-06-03T01:20:53.219 INFO:tasks.workunit.client.1.smithi044.stdout:4/314: stat d17/d1c 0 2024-06-03T01:20:53.220 INFO:tasks.workunit.client.1.smithi044.stdout:1/432: write d0/d3/d14/f38 [7112308,53230] 0 2024-06-03T01:20:53.220 INFO:tasks.workunit.client.1.smithi044.stdout:1/433: truncate d0/d16/d64/d6a/f74 5093955 0 2024-06-03T01:20:53.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/277: dwrite f2 [4194304,4194304] 0 2024-06-03T01:20:53.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/349: dwrite d2/d11/d12/d2b/d4a/d57/f50 [0,4194304] 0 2024-06-03T01:20:53.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/350: stat d2/dc 0 2024-06-03T01:20:53.224 INFO:tasks.workunit.client.1.smithi044.stdout:8/413: rename d5/d20/f3d to d5/d16/f7c 0 2024-06-03T01:20:53.225 INFO:tasks.workunit.client.1.smithi044.stdout:9/303: mknod d6/dd/d45/c60 0 2024-06-03T01:20:53.225 INFO:tasks.workunit.client.1.smithi044.stdout:9/304: chown d6/f44 186 1 2024-06-03T01:20:53.226 INFO:tasks.workunit.client.1.smithi044.stdout:9/305: dread - d6/dd/d51/d16/f5a zero size 2024-06-03T01:20:53.227 INFO:tasks.workunit.client.1.smithi044.stdout:3/314: symlink df/d13/d39/d3d/d43/l63 0 2024-06-03T01:20:53.228 INFO:tasks.workunit.client.1.smithi044.stdout:0/494: creat d3/d9/d55/d7b/fb0 x:0 0 0 2024-06-03T01:20:53.228 INFO:tasks.workunit.client.1.smithi044.stdout:0/495: creat d3/da6/fb1 x:0 0 0 2024-06-03T01:20:53.228 INFO:tasks.workunit.client.1.smithi044.stdout:0/496: fdatasync d3/da6/f3e 0 2024-06-03T01:20:53.229 INFO:tasks.workunit.client.1.smithi044.stdout:0/497: chown d3/d9/d36/d82 0 1 2024-06-03T01:20:53.230 INFO:tasks.workunit.client.1.smithi044.stdout:1/434: mknod d0/d3/c91 0 2024-06-03T01:20:53.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/278: dread db/d1b/d2a/f33 [0,4194304] 0 2024-06-03T01:20:53.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/279: fsync db/dd/fe 0 2024-06-03T01:20:53.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/280: readlink db/l13 0 2024-06-03T01:20:53.236 INFO:tasks.workunit.client.1.smithi044.stdout:5/351: creat d2/d11/d3e/f73 x:0 0 0 2024-06-03T01:20:53.236 INFO:tasks.workunit.client.1.smithi044.stdout:5/352: chown d2/d11/c47 28273 1 2024-06-03T01:20:53.238 INFO:tasks.workunit.client.1.smithi044.stdout:8/414: rename d5/d50/l51 to d5/d20/d33/l7d 0 2024-06-03T01:20:53.239 INFO:tasks.workunit.client.1.smithi044.stdout:6/519: dwrite d2/f31 [0,4194304] 0 2024-06-03T01:20:53.239 INFO:tasks.workunit.client.1.smithi044.stdout:6/520: chown d2/d23/d30/d1e/d34/d39/d64/l67 114 1 2024-06-03T01:20:53.240 INFO:tasks.workunit.client.1.smithi044.stdout:9/306: mkdir d6/dd/d51/d16/d61 0 2024-06-03T01:20:53.241 INFO:tasks.workunit.client.1.smithi044.stdout:4/315: dwrite d17/d23/f4d [4194304,4194304] 0 2024-06-03T01:20:53.244 INFO:tasks.workunit.client.1.smithi044.stdout:0/498: truncate d3/da6/f3b 585857 0 2024-06-03T01:20:53.244 INFO:tasks.workunit.client.1.smithi044.stdout:1/435: getdents d0/d3/d20/d7 0 2024-06-03T01:20:53.245 INFO:tasks.workunit.client.1.smithi044.stdout:0/499: chown d3/da6/d21/f9b 5845 1 2024-06-03T01:20:53.245 INFO:tasks.workunit.client.1.smithi044.stdout:0/500: read - d3/f81 zero size 2024-06-03T01:20:53.245 INFO:tasks.workunit.client.1.smithi044.stdout:0/501: write d3/da6/d56/f5d [1461845,85866] 0 2024-06-03T01:20:53.245 INFO:tasks.workunit.client.1.smithi044.stdout:2/281: mknod db/d15/c5e 0 2024-06-03T01:20:53.246 INFO:tasks.workunit.client.1.smithi044.stdout:5/353: dread d2/d11/d12/d2b/f34 [0,4194304] 0 2024-06-03T01:20:53.247 INFO:tasks.workunit.client.1.smithi044.stdout:3/315: rename df/f32 to df/d13/d1b/f64 0 2024-06-03T01:20:53.250 INFO:tasks.workunit.client.1.smithi044.stdout:6/521: read d2/d23/d30/d1e/d38/d6c/f12 [1482065,127579] 0 2024-06-03T01:20:53.256 INFO:tasks.workunit.client.1.smithi044.stdout:9/307: creat d6/dd/d51/f62 x:0 0 0 2024-06-03T01:20:53.261 INFO:tasks.workunit.client.1.smithi044.stdout:5/354: dwrite d2/d11/d12/d2b/f72 [0,4194304] 0 2024-06-03T01:20:53.262 INFO:tasks.workunit.client.1.smithi044.stdout:5/355: chown d2/d11/d1e/f51 1 1 2024-06-03T01:20:53.268 INFO:tasks.workunit.client.1.smithi044.stdout:2/282: stat l4 0 2024-06-03T01:20:53.268 INFO:tasks.workunit.client.1.smithi044.stdout:2/283: creat db/d1b/d27/f5f x:0 0 0 2024-06-03T01:20:53.271 INFO:tasks.workunit.client.1.smithi044.stdout:3/316: unlink df/d13/d1b/f33 0 2024-06-03T01:20:53.271 INFO:tasks.workunit.client.1.smithi044.stdout:3/317: fsync df/d13/f26 0 2024-06-03T01:20:53.273 INFO:tasks.workunit.client.1.smithi044.stdout:6/522: creat d2/d23/d30/d1e/d34/d39/da8/fbb x:0 0 0 2024-06-03T01:20:53.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/415: rmdir d5/d20/d33 39 2024-06-03T01:20:53.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/416: read - d5/d10/f73 zero size 2024-06-03T01:20:53.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/308: symlink d6/dd/d51/d16/d27/l63 0 2024-06-03T01:20:53.281 INFO:tasks.workunit.client.1.smithi044.stdout:5/356: creat d2/dc/d28/f74 x:0 0 0 2024-06-03T01:20:53.281 INFO:tasks.workunit.client.1.smithi044.stdout:5/357: chown d2/ld 9259856 1 2024-06-03T01:20:53.282 INFO:tasks.workunit.client.1.smithi044.stdout:3/318: dread df/d13/f3f [0,4194304] 0 2024-06-03T01:20:53.284 INFO:tasks.workunit.client.1.smithi044.stdout:4/316: rmdir d17/d23/d59 39 2024-06-03T01:20:53.284 INFO:tasks.workunit.client.1.smithi044.stdout:4/317: stat d17/d23/d35/l6b 0 2024-06-03T01:20:53.284 INFO:tasks.workunit.client.1.smithi044.stdout:4/318: write d17/d23/f20 [1161138,7603] 0 2024-06-03T01:20:53.286 INFO:tasks.workunit.client.1.smithi044.stdout:2/284: dwrite db/f3e [0,4194304] 0 2024-06-03T01:20:53.287 INFO:tasks.workunit.client.1.smithi044.stdout:1/436: truncate d0/f29 2794821 0 2024-06-03T01:20:53.288 INFO:tasks.workunit.client.1.smithi044.stdout:0/502: truncate d3/f19 1999887 0 2024-06-03T01:20:53.289 INFO:tasks.workunit.client.1.smithi044.stdout:4/319: dread d17/f31 [0,4194304] 0 2024-06-03T01:20:53.289 INFO:tasks.workunit.client.1.smithi044.stdout:4/320: chown d17/d1c/f52 223711484 1 2024-06-03T01:20:53.289 INFO:tasks.workunit.client.1.smithi044.stdout:4/321: readlink l10 0 2024-06-03T01:20:53.289 INFO:tasks.workunit.client.1.smithi044.stdout:4/322: creat d17/d23/d28/f6f x:0 0 0 2024-06-03T01:20:53.292 INFO:tasks.workunit.client.1.smithi044.stdout:8/417: rename d5/d10/l36 to d5/d10/l7e 0 2024-06-03T01:20:53.292 INFO:tasks.workunit.client.1.smithi044.stdout:8/418: creat d5/d50/f7f x:0 0 0 2024-06-03T01:20:53.294 INFO:tasks.workunit.client.1.smithi044.stdout:9/309: creat d6/dd/d51/d16/d27/f64 x:0 0 0 2024-06-03T01:20:53.294 INFO:tasks.workunit.client.1.smithi044.stdout:9/310: creat d6/f65 x:0 0 0 2024-06-03T01:20:53.298 INFO:tasks.workunit.client.1.smithi044.stdout:6/523: dwrite d2/d23/d30/d1e/d38/f72 [0,4194304] 0 2024-06-03T01:20:53.298 INFO:tasks.workunit.client.1.smithi044.stdout:6/524: dread - d2/d23/d30/f45 zero size 2024-06-03T01:20:53.300 INFO:tasks.workunit.client.1.smithi044.stdout:0/503: creat d3/d6b/d95/fb2 x:0 0 0 2024-06-03T01:20:53.303 INFO:tasks.workunit.client.1.smithi044.stdout:5/358: dwrite d2/dc/f65 [0,4194304] 0 2024-06-03T01:20:53.303 INFO:tasks.workunit.client.1.smithi044.stdout:4/323: mkdir d17/d1c/d70 0 2024-06-03T01:20:53.310 INFO:tasks.workunit.client.1.smithi044.stdout:9/311: mkdir d6/d38/d66 0 2024-06-03T01:20:53.310 INFO:tasks.workunit.client.1.smithi044.stdout:9/312: dread - d6/d26/f4a zero size 2024-06-03T01:20:53.313 INFO:tasks.workunit.client.1.smithi044.stdout:6/525: rename d2/d23/d55/d5f/d53/f73 to d2/d23/d30/d1e/fbc 0 2024-06-03T01:20:53.313 INFO:tasks.workunit.client.1.smithi044.stdout:6/526: write d2/d23/d30/d1e/d34/d39/fad [809609,25920] 0 2024-06-03T01:20:53.314 INFO:tasks.workunit.client.1.smithi044.stdout:0/504: unlink d3/d9/d36/l87 0 2024-06-03T01:20:53.315 INFO:tasks.workunit.client.1.smithi044.stdout:1/437: dwrite d0/d16/f26 [0,4194304] 0 2024-06-03T01:20:53.315 INFO:tasks.workunit.client.1.smithi044.stdout:2/285: dwrite db/f26 [8388608,4194304] 0 2024-06-03T01:20:53.315 INFO:tasks.workunit.client.1.smithi044.stdout:2/286: readlink db/d1b/d2a/d2f/l58 0 2024-06-03T01:20:53.317 INFO:tasks.workunit.client.1.smithi044.stdout:9/313: dread d6/dd/f21 [0,4194304] 0 2024-06-03T01:20:53.317 INFO:tasks.workunit.client.1.smithi044.stdout:9/314: stat d6/d26/f4f 0 2024-06-03T01:20:53.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/324: rename d17/d23/d28/f54 to d17/d23/f71 0 2024-06-03T01:20:53.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/527: creat d2/d23/d30/d1e/d38/d77/fbd x:0 0 0 2024-06-03T01:20:53.321 INFO:tasks.workunit.client.1.smithi044.stdout:0/505: mkdir d3/d6b/d95/db3 0 2024-06-03T01:20:53.323 INFO:tasks.workunit.client.1.smithi044.stdout:1/438: creat d0/d3/d20/d40/d45/d4b/d80/f92 x:0 0 0 2024-06-03T01:20:53.324 INFO:tasks.workunit.client.1.smithi044.stdout:9/315: mkdir d6/dd/d51/d16/d4d/d67 0 2024-06-03T01:20:53.324 INFO:tasks.workunit.client.1.smithi044.stdout:9/316: chown d6/dd/f14 25535 1 2024-06-03T01:20:53.325 INFO:tasks.workunit.client.1.smithi044.stdout:5/359: rmdir d2/d11/d1e/d3b 39 2024-06-03T01:20:53.325 INFO:tasks.workunit.client.1.smithi044.stdout:5/360: dread - d2/dc/d28/f74 zero size 2024-06-03T01:20:53.325 INFO:tasks.workunit.client.1.smithi044.stdout:5/361: read - d2/dc/d28/f74 zero size 2024-06-03T01:20:53.329 INFO:tasks.workunit.client.1.smithi044.stdout:0/506: creat d3/d9/d36/fb4 x:0 0 0 2024-06-03T01:20:53.329 INFO:tasks.workunit.client.1.smithi044.stdout:0/507: stat d3/d9/f13 0 2024-06-03T01:20:53.332 INFO:tasks.workunit.client.1.smithi044.stdout:1/439: unlink d0/d3/d20/d40/d45/c46 0 2024-06-03T01:20:53.333 INFO:tasks.workunit.client.1.smithi044.stdout:4/325: dwrite d17/d1c/f52 [0,4194304] 0 2024-06-03T01:20:53.334 INFO:tasks.workunit.client.1.smithi044.stdout:5/362: creat d2/d11/d1e/d21/f75 x:0 0 0 2024-06-03T01:20:53.335 INFO:tasks.workunit.client.1.smithi044.stdout:5/363: chown d2/d11/d12/d2b/d4a/f5d 6869913 1 2024-06-03T01:20:53.335 INFO:tasks.workunit.client.1.smithi044.stdout:1/440: dread d0/d3/d20/d40/f55 [0,4194304] 0 2024-06-03T01:20:53.335 INFO:tasks.workunit.client.1.smithi044.stdout:5/364: creat d2/d11/d12/d2b/d4a/d57/f76 x:0 0 0 2024-06-03T01:20:53.335 INFO:tasks.workunit.client.1.smithi044.stdout:1/441: truncate d0/d3/d20/d40/d45/d4b/d5c/f8a 896277 0 2024-06-03T01:20:53.335 INFO:tasks.workunit.client.1.smithi044.stdout:5/365: stat d2/d11/d3e/f59 0 2024-06-03T01:20:53.336 INFO:tasks.workunit.client.1.smithi044.stdout:0/508: mknod d3/d6b/d95/db3/cb5 0 2024-06-03T01:20:53.336 INFO:tasks.workunit.client.1.smithi044.stdout:0/509: chown d3/da6/d56/d62/fa9 104 1 2024-06-03T01:20:53.340 INFO:tasks.workunit.client.1.smithi044.stdout:4/326: symlink d17/d23/d58/l72 0 2024-06-03T01:20:53.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/317: dwrite d6/dd/f17 [0,4194304] 0 2024-06-03T01:20:53.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/318: chown d6/l19 67235 1 2024-06-03T01:20:53.343 INFO:tasks.workunit.client.1.smithi044.stdout:1/442: truncate d0/d3/d20/d7/f36 2769072 0 2024-06-03T01:20:53.345 INFO:tasks.workunit.client.1.smithi044.stdout:4/327: link d17/d1c/f52 d17/d23/d6a/f73 0 2024-06-03T01:20:53.345 INFO:tasks.workunit.client.1.smithi044.stdout:4/328: chown d17/d23/f57 80 1 2024-06-03T01:20:53.346 INFO:tasks.workunit.client.1.smithi044.stdout:4/329: dread d17/d23/d28/d2b/f21 [0,4194304] 0 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:9/319: mkdir d6/dd/d51/d16/d4d/d55/d68 0 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:9/320: chown d6/dd/f3d 45 1 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:4/330: dread d17/d23/d35/f41 [0,4194304] 0 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:4/331: rename d17/d23/d35 to d17/d23/d35/d74 22 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:4/332: chown d17/d23/d6a/f73 213518017 1 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:4/333: chown d17/d23/d28/d2b/f4a 2442178 1 2024-06-03T01:20:53.349 INFO:tasks.workunit.client.1.smithi044.stdout:1/443: creat d0/d16/f93 x:0 0 0 2024-06-03T01:20:53.350 INFO:tasks.workunit.client.1.smithi044.stdout:5/366: getdents d2/d11/d12/d2b/d4a 0 2024-06-03T01:20:53.351 INFO:tasks.workunit.client.1.smithi044.stdout:9/321: symlink d6/dd/d51/d16/d27/d5e/l69 0 2024-06-03T01:20:53.353 INFO:tasks.workunit.client.1.smithi044.stdout:4/334: rename d17/d23/d28/d39 to d17/d1c/d75 0 2024-06-03T01:20:53.354 INFO:tasks.workunit.client.1.smithi044.stdout:1/444: mkdir d0/d3/d20/d40/d45/d4b/d5c/d94 0 2024-06-03T01:20:53.356 INFO:tasks.workunit.client.1.smithi044.stdout:5/367: unlink d2/d11/f6a 0 2024-06-03T01:20:53.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/322: rmdir d6/d38/d66 0 2024-06-03T01:20:53.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/323: chown d6/dd/d51/c24 203280613 1 2024-06-03T01:20:53.359 INFO:tasks.workunit.client.1.smithi044.stdout:4/335: creat d17/d1c/d75/f76 x:0 0 0 2024-06-03T01:20:53.359 INFO:tasks.workunit.client.1.smithi044.stdout:4/336: chown d17/d23/d28/d2b/f21 57520819 1 2024-06-03T01:20:53.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/445: rename d0/d3/d20/d7/d2f/c6e to d0/d3/d20/d40/d45/d6f/c95 0 2024-06-03T01:20:53.362 INFO:tasks.workunit.client.1.smithi044.stdout:1/446: read - d0/d3/d14/f8d zero size 2024-06-03T01:20:53.365 INFO:tasks.workunit.client.1.smithi044.stdout:4/337: mknod d17/d23/d6a/c77 0 2024-06-03T01:20:53.366 INFO:tasks.workunit.client.1.smithi044.stdout:1/447: creat d0/d3/d20/d7/d13/f96 x:0 0 0 2024-06-03T01:20:53.369 INFO:tasks.workunit.client.1.smithi044.stdout:4/338: rename d17/d23/d28/c2a to d17/d1c/d70/c78 0 2024-06-03T01:20:53.369 INFO:tasks.workunit.client.1.smithi044.stdout:4/339: stat d17/d1c/d75/f5c 0 2024-06-03T01:20:53.369 INFO:tasks.workunit.client.1.smithi044.stdout:4/340: truncate d17/d23/d35/f3c 449830 0 2024-06-03T01:20:53.370 INFO:tasks.workunit.client.1.smithi044.stdout:1/448: symlink d0/d3/d20/d40/d45/d4b/d5a/l97 0 2024-06-03T01:20:53.370 INFO:tasks.workunit.client.1.smithi044.stdout:1/449: write d0/d3/d14/f50 [892876,80143] 0 2024-06-03T01:20:53.371 INFO:tasks.workunit.client.1.smithi044.stdout:4/341: symlink d17/d23/d58/l79 0 2024-06-03T01:20:53.371 INFO:tasks.workunit.client.1.smithi044.stdout:9/324: dwrite d6/dd/d51/d16/f34 [0,4194304] 0 2024-06-03T01:20:53.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/342: mkdir d17/d1c/d75/d7a 0 2024-06-03T01:20:53.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/343: write d17/d23/d35/f41 [378704,110012] 0 2024-06-03T01:20:53.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/344: readlink d17/l26 0 2024-06-03T01:20:53.377 INFO:tasks.workunit.client.1.smithi044.stdout:4/345: creat d17/d23/d59/f7b x:0 0 0 2024-06-03T01:20:53.384 INFO:tasks.workunit.client.1.smithi044.stdout:5/368: dwrite d2/d11/d3e/f59 [0,4194304] 0 2024-06-03T01:20:53.384 INFO:tasks.workunit.client.1.smithi044.stdout:5/369: truncate d2/d11/d1e/d21/f75 72512 0 2024-06-03T01:20:53.388 INFO:tasks.workunit.client.1.smithi044.stdout:5/370: truncate d2/fa 1970562 0 2024-06-03T01:20:53.397 INFO:tasks.workunit.client.1.smithi044.stdout:4/346: dwrite d17/d23/f67 [0,4194304] 0 2024-06-03T01:20:53.398 INFO:tasks.workunit.client.1.smithi044.stdout:1/450: dwrite d0/d3/f62 [0,4194304] 0 2024-06-03T01:20:53.405 INFO:tasks.workunit.client.1.smithi044.stdout:1/451: creat d0/d3/d20/d40/d45/d6f/f98 x:0 0 0 2024-06-03T01:20:53.407 INFO:tasks.workunit.client.1.smithi044.stdout:5/371: dwrite d2/dc/f65 [0,4194304] 0 2024-06-03T01:20:53.411 INFO:tasks.workunit.client.1.smithi044.stdout:5/372: rename d2/dc/f65 to d2/d11/d1e/d21/d24/f77 0 2024-06-03T01:20:53.416 INFO:tasks.workunit.client.1.smithi044.stdout:1/452: write d0/d3/f62 [1899716,28406] 0 2024-06-03T01:20:53.417 INFO:tasks.workunit.client.1.smithi044.stdout:1/453: stat d0/d3/d14/d30/c39 0 2024-06-03T01:20:53.417 INFO:tasks.workunit.client.1.smithi044.stdout:1/454: write d0/d3/d20/d7/d4e/f8e [1035283,1102] 0 2024-06-03T01:20:53.418 INFO:tasks.workunit.client.1.smithi044.stdout:4/347: dwrite d17/d23/d35/f41 [0,4194304] 0 2024-06-03T01:20:53.418 INFO:tasks.workunit.client.1.smithi044.stdout:4/348: creat d17/f7c x:0 0 0 2024-06-03T01:20:53.418 INFO:tasks.workunit.client.1.smithi044.stdout:4/349: write d17/d23/d28/d2b/f2e [451393,29251] 0 2024-06-03T01:20:53.419 INFO:tasks.workunit.client.1.smithi044.stdout:1/455: mknod d0/d3/d20/d7/d13/d7a/c99 0 2024-06-03T01:20:53.419 INFO:tasks.workunit.client.1.smithi044.stdout:1/456: fsync d0/d3/d20/d40/d45/d4b/f90 0 2024-06-03T01:20:53.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/457: dread d0/f3f [0,4194304] 0 2024-06-03T01:20:53.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/458: readlink d0/l10 0 2024-06-03T01:20:53.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/373: write d2/d11/d3e/f40 [4230335,16742] 0 2024-06-03T01:20:53.421 INFO:tasks.workunit.client.1.smithi044.stdout:1/459: getdents d0/d66 0 2024-06-03T01:20:53.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/374: rmdir d2/d11/d12/d2b/d44 39 2024-06-03T01:20:53.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/375: fsync d2/dc/d28/d4c/f4e 0 2024-06-03T01:20:53.427 INFO:tasks.workunit.client.1.smithi044.stdout:4/350: read d17/d23/f67 [1879450,18470] 0 2024-06-03T01:20:53.428 INFO:tasks.workunit.client.1.smithi044.stdout:4/351: fsync d17/d63/f65 0 2024-06-03T01:20:53.428 INFO:tasks.workunit.client.1.smithi044.stdout:4/352: read d17/d23/d59/f5f [109317,42928] 0 2024-06-03T01:20:53.429 INFO:tasks.workunit.client.1.smithi044.stdout:4/353: mkdir d17/d1c/d70/d7d 0 2024-06-03T01:20:53.430 INFO:tasks.workunit.client.1.smithi044.stdout:4/354: mkdir d17/d1c/d7e 0 2024-06-03T01:20:53.436 INFO:tasks.workunit.client.1.smithi044.stdout:4/355: symlink d17/d1c/d70/l7f 0 2024-06-03T01:20:53.436 INFO:tasks.workunit.client.1.smithi044.stdout:4/356: chown d17/d23/d59/f5f 457 1 2024-06-03T01:20:53.436 INFO:tasks.workunit.client.1.smithi044.stdout:4/357: write fd [3635339,26402] 0 2024-06-03T01:20:53.438 INFO:tasks.workunit.client.1.smithi044.stdout:1/460: dwrite d0/d3/d14/f50 [0,4194304] 0 2024-06-03T01:20:53.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/461: dread d0/d3/d20/d7/d2f/f59 [0,4194304] 0 2024-06-03T01:20:53.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/462: creat d0/d3/d20/d40/d45/d4b/d5c/f9a x:0 0 0 2024-06-03T01:20:53.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/463: readlink d0/l10 0 2024-06-03T01:20:53.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/464: chown d0/d3/d20/d40/d45/d4b/d80 116669636 1 2024-06-03T01:20:53.446 INFO:tasks.workunit.client.1.smithi044.stdout:4/358: dread f2 [0,4194304] 0 2024-06-03T01:20:53.447 INFO:tasks.workunit.client.1.smithi044.stdout:1/465: truncate d0/d3/d20/d40/d45/d6b/f77 104054 0 2024-06-03T01:20:53.447 INFO:tasks.workunit.client.1.smithi044.stdout:4/359: symlink d17/d1c/d70/l80 0 2024-06-03T01:20:53.447 INFO:tasks.workunit.client.1.smithi044.stdout:4/360: write d17/d23/f71 [3163078,116647] 0 2024-06-03T01:20:53.448 INFO:tasks.workunit.client.1.smithi044.stdout:1/466: symlink d0/d3/d20/d40/d45/l9b 0 2024-06-03T01:20:53.462 INFO:tasks.workunit.client.1.smithi044.stdout:7/483: sync 2024-06-03T01:20:53.462 INFO:tasks.workunit.client.1.smithi044.stdout:1/467: dwrite d0/d3/d20/d40/d45/d4b/d80/f92 [0,4194304] 0 2024-06-03T01:20:53.462 INFO:tasks.workunit.client.1.smithi044.stdout:1/468: creat d0/d3/d20/f9c x:0 0 0 2024-06-03T01:20:53.464 INFO:tasks.workunit.client.1.smithi044.stdout:1/469: mknod d0/d3/d20/d40/d45/d4b/c9d 0 2024-06-03T01:20:53.465 INFO:tasks.workunit.client.1.smithi044.stdout:1/470: unlink d0/d16/d64/f82 0 2024-06-03T01:20:53.467 INFO:tasks.workunit.client.1.smithi044.stdout:1/471: mkdir d0/d16/d64/d6a/d9e 0 2024-06-03T01:20:53.467 INFO:tasks.workunit.client.1.smithi044.stdout:1/472: dread - d0/d3/d20/d40/d45/d4b/d5c/f9a zero size 2024-06-03T01:20:53.467 INFO:tasks.workunit.client.1.smithi044.stdout:1/473: dread - d0/d3/d20/f54 zero size 2024-06-03T01:20:53.467 INFO:tasks.workunit.client.1.smithi044.stdout:1/474: symlink d0/d3/d14/d30/l9f 0 2024-06-03T01:20:53.471 INFO:tasks.workunit.client.1.smithi044.stdout:7/484: dread d0/d16/d23/f24 [0,4194304] 0 2024-06-03T01:20:53.471 INFO:tasks.workunit.client.1.smithi044.stdout:7/485: write d0/d3/d8/d32/f4a [1169196,107081] 0 2024-06-03T01:20:53.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/319: sync 2024-06-03T01:20:53.548 INFO:tasks.workunit.client.1.smithi044.stdout:3/320: dwrite df/d13/d39/d3d/d43/f5b [0,4194304] 0 2024-06-03T01:20:53.548 INFO:tasks.workunit.client.1.smithi044.stdout:3/321: fsync df/f47 0 2024-06-03T01:20:53.553 INFO:tasks.workunit.client.1.smithi044.stdout:3/322: dread fe [0,4194304] 0 2024-06-03T01:20:53.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/323: read f9 [3474132,125508] 0 2024-06-03T01:20:53.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/324: chown df/c42 15797 1 2024-06-03T01:20:53.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/325: write df/d13/d1b/f5e [138561,4849] 0 2024-06-03T01:20:53.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/326: creat df/f65 x:0 0 0 2024-06-03T01:20:53.555 INFO:tasks.workunit.client.1.smithi044.stdout:3/327: rmdir df/d13/d1b 39 2024-06-03T01:20:53.556 INFO:tasks.workunit.client.1.smithi044.stdout:3/328: dread df/d13/f25 [0,4194304] 0 2024-06-03T01:20:53.557 INFO:tasks.workunit.client.1.smithi044.stdout:3/329: link df/d13/d39/c48 df/d13/d1b/d56/d5c/c66 0 2024-06-03T01:20:53.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/330: rename df/f2f to df/d13/d1b/d56/f67 0 2024-06-03T01:20:53.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/331: stat df/d13/f1a 0 2024-06-03T01:20:53.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/332: truncate df/d13/d39/d3d/f44 633558 0 2024-06-03T01:20:53.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/333: chown df/f65 3516 1 2024-06-03T01:20:53.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/334: fdatasync df/f23 0 2024-06-03T01:20:53.560 INFO:tasks.workunit.client.1.smithi044.stdout:3/335: creat df/d13/d39/d3d/f68 x:0 0 0 2024-06-03T01:20:53.561 INFO:tasks.workunit.client.1.smithi044.stdout:3/336: unlink df/f31 0 2024-06-03T01:20:53.562 INFO:tasks.workunit.client.1.smithi044.stdout:3/337: mkdir df/d13/d1b/d4b/d69 0 2024-06-03T01:20:53.575 INFO:tasks.workunit.client.1.smithi044.stdout:3/338: dwrite df/d13/d1b/d4b/f54 [0,4194304] 0 2024-06-03T01:20:53.575 INFO:tasks.workunit.client.1.smithi044.stdout:3/339: read df/d13/d1b/f64 [121342,9237] 0 2024-06-03T01:20:53.576 INFO:tasks.workunit.client.1.smithi044.stdout:3/340: dread df/d13/d1b/f64 [0,4194304] 0 2024-06-03T01:20:53.577 INFO:tasks.workunit.client.1.smithi044.stdout:3/341: mkdir df/d13/d39/d3d/d6a 0 2024-06-03T01:20:53.578 INFO:tasks.workunit.client.1.smithi044.stdout:3/342: stat df/d13/d39/d3d/f68 0 2024-06-03T01:20:53.641 INFO:tasks.workunit.client.1.smithi044.stdout:8/419: sync 2024-06-03T01:20:53.642 INFO:tasks.workunit.client.1.smithi044.stdout:8/420: creat d5/d16/d1d/d6b/f80 x:0 0 0 2024-06-03T01:20:53.643 INFO:tasks.workunit.client.1.smithi044.stdout:8/421: rmdir d5/d20/d2b/d58 39 2024-06-03T01:20:53.658 INFO:tasks.workunit.client.1.smithi044.stdout:8/422: dwrite d5/d20/d33/f4b [0,4194304] 0 2024-06-03T01:20:53.658 INFO:tasks.workunit.client.1.smithi044.stdout:8/423: fdatasync d5/f25 0 2024-06-03T01:20:53.661 INFO:tasks.workunit.client.1.smithi044.stdout:8/424: mkdir d5/d16/d81 0 2024-06-03T01:20:53.664 INFO:tasks.workunit.client.1.smithi044.stdout:8/425: rename d5/d10/d5f to d5/d55/d74/d82 0 2024-06-03T01:20:53.666 INFO:tasks.workunit.client.1.smithi044.stdout:8/426: rename d5/ff to d5/d16/d81/f83 0 2024-06-03T01:20:53.666 INFO:tasks.workunit.client.1.smithi044.stdout:8/427: chown d5/d20/d47/f71 34448689 1 2024-06-03T01:20:53.666 INFO:tasks.workunit.client.1.smithi044.stdout:8/428: truncate d5/f6a 843823 0 2024-06-03T01:20:53.667 INFO:tasks.workunit.client.1.smithi044.stdout:8/429: rename d5/d16/d1d/d32/f62 to d5/d20/d2b/d58/f84 0 2024-06-03T01:20:53.667 INFO:tasks.workunit.client.1.smithi044.stdout:8/430: read d5/d16/f3a [541624,91536] 0 2024-06-03T01:20:53.668 INFO:tasks.workunit.client.1.smithi044.stdout:8/431: symlink d5/d16/l85 0 2024-06-03T01:20:53.668 INFO:tasks.workunit.client.1.smithi044.stdout:8/432: chown d5/d16/d1d/f1f 93244 1 2024-06-03T01:20:53.669 INFO:tasks.workunit.client.1.smithi044.stdout:8/433: write d5/d20/d47/f71 [370084,76507] 0 2024-06-03T01:20:53.669 INFO:tasks.workunit.client.1.smithi044.stdout:8/434: creat d5/d55/f86 x:0 0 0 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/361: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/510: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:9/325: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:2/287: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/511: chown d3/d9/d55/d73/da0 9567 1 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/486: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/512: read - d3/d5c/fa2 zero size 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:6/528: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/343: sync 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:9/326: write d6/dd/d51/d16/f34 [4684917,42014] 0 2024-06-03T01:20:53.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/487: write d0/d3/f7 [5107207,13474] 0 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:9/327: chown d6/dd/d51/d16/c3a 306031296 1 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:0/513: write d3/da6/d56/d62/fa9 [973310,31369] 0 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:0/514: readlink d3/da6/l6c 0 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:5/376: sync 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:1/475: sync 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:5/377: chown d2/d11/d3e/f40 30 1 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:5/378: stat d2/d11/d12/d2b/f34 0 2024-06-03T01:20:53.715 INFO:tasks.workunit.client.1.smithi044.stdout:4/362: link d17/d23/f20 d17/d23/d59/f81 0 2024-06-03T01:20:53.716 INFO:tasks.workunit.client.1.smithi044.stdout:4/363: truncate d17/d23/d59/f5f 1464364 0 2024-06-03T01:20:53.716 INFO:tasks.workunit.client.1.smithi044.stdout:4/364: creat d17/d23/d59/f82 x:0 0 0 2024-06-03T01:20:53.716 INFO:tasks.workunit.client.1.smithi044.stdout:2/288: creat db/dd/f60 x:0 0 0 2024-06-03T01:20:53.719 INFO:tasks.workunit.client.1.smithi044.stdout:4/365: write d17/f31 [3768242,70894] 0 2024-06-03T01:20:53.719 INFO:tasks.workunit.client.1.smithi044.stdout:9/328: unlink d6/dd/d51/d3b/l43 0 2024-06-03T01:20:53.719 INFO:tasks.workunit.client.1.smithi044.stdout:9/329: readlink d6/dd/l1a 0 2024-06-03T01:20:53.720 INFO:tasks.workunit.client.1.smithi044.stdout:9/330: creat d6/dd/d51/d3b/f6a x:0 0 0 2024-06-03T01:20:53.721 INFO:tasks.workunit.client.1.smithi044.stdout:1/476: mknod d0/d16/d64/d6a/d9e/ca0 0 2024-06-03T01:20:53.721 INFO:tasks.workunit.client.1.smithi044.stdout:1/477: fsync d0/d3/d20/d40/d45/d4b/d80/f92 0 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/2: sync 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/3: chown d0 2 1 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/4: getdents d0 0 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/5: truncate - no filename 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/6: chown d0 40183 1 2024-06-03T01:20:53.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/7: fdatasync - no filename 2024-06-03T01:20:53.724 INFO:tasks.workunit.client.0.smithi032.stdout:0/8: truncate - no filename 2024-06-03T01:20:53.725 INFO:tasks.workunit.client.1.smithi044.stdout:2/289: truncate db/d1b/f36 385161 0 2024-06-03T01:20:53.726 INFO:tasks.workunit.client.1.smithi044.stdout:7/488: getdents d0/d3/d8 0 2024-06-03T01:20:53.728 INFO:tasks.workunit.client.1.smithi044.stdout:4/366: read d17/d23/d6a/f73 [102288,49209] 0 2024-06-03T01:20:53.728 INFO:tasks.workunit.client.1.smithi044.stdout:4/367: write d17/d23/d59/f7b [1042346,91682] 0 2024-06-03T01:20:53.728 INFO:tasks.workunit.client.0.smithi032.stdout:0/9: rmdir d0 0 2024-06-03T01:20:53.730 INFO:tasks.workunit.client.1.smithi044.stdout:6/529: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/f83 [0,4194304] 0 2024-06-03T01:20:53.730 INFO:tasks.workunit.client.0.smithi032.stdout:0/10: mkdir d1 0 2024-06-03T01:20:53.730 INFO:tasks.workunit.client.0.smithi032.stdout:0/11: dwrite - no filename 2024-06-03T01:20:53.731 INFO:tasks.workunit.client.0.smithi032.stdout:0/12: truncate - no filename 2024-06-03T01:20:53.733 INFO:tasks.workunit.client.1.smithi044.stdout:9/331: rename d6/l49 to d6/dd/d51/d16/d4d/d55/l6b 0 2024-06-03T01:20:53.736 INFO:tasks.workunit.client.1.smithi044.stdout:4/368: symlink d17/d1c/d70/d7d/l83 0 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/530: mknod d2/d23/d30/d1e/d34/d39/cbe 0 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/531: chown d2/d6/c29 260 1 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/532: chown d2/d23/d30/d1e/d34/d39/c98 9 1 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/533: fsync d2/d23/d55/d5f/fa9 0 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/534: write d2/d23/d55/d5f/f62 [1024683,39635] 0 2024-06-03T01:20:53.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/535: creat d2/d23/d30/d1e/d38/d6c/fbf x:0 0 0 2024-06-03T01:20:53.739 INFO:tasks.workunit.client.1.smithi044.stdout:2/290: getdents db/d1b/d27 0 2024-06-03T01:20:53.740 INFO:tasks.workunit.client.1.smithi044.stdout:6/536: rmdir d2/d23/d30/d1e/d34/d39/d7d/da7/db1 39 2024-06-03T01:20:53.740 INFO:tasks.workunit.client.1.smithi044.stdout:6/537: write d2/d23/d30/d1e/d38/d6c/d5c/fab [241573,104668] 0 2024-06-03T01:20:53.740 INFO:tasks.workunit.client.1.smithi044.stdout:6/538: dread - d2/d23/d30/d1e/d34/d39/d9b/db4/f6d zero size 2024-06-03T01:20:53.742 INFO:tasks.workunit.client.1.smithi044.stdout:2/291: unlink db/c3c 0 2024-06-03T01:20:53.742 INFO:tasks.workunit.client.1.smithi044.stdout:6/539: mkdir d2/d23/d30/d1e/d34/d39/d9b/db4/dba/dc0 0 2024-06-03T01:20:53.744 INFO:tasks.workunit.client.1.smithi044.stdout:6/540: symlink d2/d23/d55/d5f/d85/lc1 0 2024-06-03T01:20:53.745 INFO:tasks.workunit.client.1.smithi044.stdout:6/541: mkdir d2/d23/d30/d1e/d38/d6c/dc2 0 2024-06-03T01:20:53.749 INFO:tasks.workunit.client.1.smithi044.stdout:2/292: dread db/d1b/d3f/f41 [0,4194304] 0 2024-06-03T01:20:53.749 INFO:tasks.workunit.client.1.smithi044.stdout:2/293: fdatasync db/d1b/d3f/f4a 0 2024-06-03T01:20:53.749 INFO:tasks.workunit.client.1.smithi044.stdout:2/294: dread - db/d1b/d27/f39 zero size 2024-06-03T01:20:53.815 INFO:tasks.workunit.client.1.smithi044.stdout:2/295: dread db/d1b/f36 [0,4194304] 0 2024-06-03T01:20:53.815 INFO:tasks.workunit.client.1.smithi044.stdout:2/296: creat db/d1b/d3f/f61 x:0 0 0 2024-06-03T01:20:53.817 INFO:tasks.workunit.client.1.smithi044.stdout:2/297: rename f7 to db/d1b/d5d/f62 0 2024-06-03T01:20:53.819 INFO:tasks.workunit.client.1.smithi044.stdout:7/489: dwrite d0/d3/d8/d95/d63/f7a [0,4194304] 0 2024-06-03T01:20:53.819 INFO:tasks.workunit.client.1.smithi044.stdout:7/490: write d0/d3/d8/d95/d84/f93 [281428,55494] 0 2024-06-03T01:20:53.819 INFO:tasks.workunit.client.1.smithi044.stdout:9/332: dwrite d6/dd/d51/d16/d27/f64 [0,4194304] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:3/344: dwrite df/d1e/d3a/f4a [0,4194304] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:4/369: dwrite d17/d1c/f3a [4194304,4194304] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/542: dwrite d2/d23/d30/d1e/d38/d6c/f3b [0,4194304] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/543: write d2/d23/d30/d1e/d38/d6c/d5c/fab [854310,96910] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/544: rename d2 to d2/d23/d30/d1e/d34/d39/da8/dc3 22 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:5/379: dwrite f1 [4194304,4194304] 0 2024-06-03T01:20:53.820 INFO:tasks.workunit.client.1.smithi044.stdout:5/380: fdatasync d2/dc/f62 0 2024-06-03T01:20:53.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/491: truncate d0/f4 887642 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/298: dread db/f1f [0,4194304] 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/515: dwrite d3/d9/f44 [4194304,4194304] 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/516: dread - d3/d9/d55/d5a/f93 zero size 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/517: creat d3/da6/fb6 x:0 0 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/299: write db/d1b/d3f/f41 [5122780,67487] 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/518: truncate d3/d5c/fa2 781520 0 2024-06-03T01:20:53.823 INFO:tasks.workunit.client.1.smithi044.stdout:9/333: truncate d6/dd/d51/db/f22 2232181 0 2024-06-03T01:20:53.824 INFO:tasks.workunit.client.1.smithi044.stdout:9/334: getdents d6/dd/d51/d16/d61 0 2024-06-03T01:20:53.829 INFO:tasks.workunit.client.1.smithi044.stdout:3/345: mknod df/d13/d1b/d59/c6b 0 2024-06-03T01:20:53.830 INFO:tasks.workunit.client.1.smithi044.stdout:4/370: mkdir d17/d23/d6a/d84 0 2024-06-03T01:20:53.830 INFO:tasks.workunit.client.1.smithi044.stdout:4/371: write d17/d23/f71 [2736444,60216] 0 2024-06-03T01:20:53.834 INFO:tasks.workunit.client.1.smithi044.stdout:6/545: truncate d2/d23/d30/d1e/d38/f72 3380580 0 2024-06-03T01:20:53.840 INFO:tasks.workunit.client.1.smithi044.stdout:5/381: dwrite d2/dc/f16 [0,4194304] 0 2024-06-03T01:20:53.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/382: write d2/d11/d1e/d21/d24/f69 [521404,37677] 0 2024-06-03T01:20:53.846 INFO:tasks.workunit.client.1.smithi044.stdout:0/519: symlink d3/d9/d55/d7b/lb7 0 2024-06-03T01:20:53.847 INFO:tasks.workunit.client.1.smithi044.stdout:2/300: creat db/d1b/d27/f63 x:0 0 0 2024-06-03T01:20:53.848 INFO:tasks.workunit.client.1.smithi044.stdout:9/335: creat d6/dd/d51/d16/d61/f6c x:0 0 0 2024-06-03T01:20:53.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/372: dwrite d17/d23/d59/f5f [0,4194304] 0 2024-06-03T01:20:53.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/373: chown d17/d1c/d7e 1955184 1 2024-06-03T01:20:53.849 INFO:tasks.workunit.client.1.smithi044.stdout:3/346: creat df/d13/d1b/d56/f6c x:0 0 0 2024-06-03T01:20:53.849 INFO:tasks.workunit.client.1.smithi044.stdout:3/347: chown df/c42 202 1 2024-06-03T01:20:53.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/546: link d2/f14 d2/d23/d30/d1e/d34/d39/d7d/da7/fc4 0 2024-06-03T01:20:53.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/547: readlink d2/d23/d55/d5f/db2/la 0 2024-06-03T01:20:53.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/548: fdatasync d2/d23/f33 0 2024-06-03T01:20:53.854 INFO:tasks.workunit.client.1.smithi044.stdout:5/383: symlink d2/d11/d12/d2b/l78 0 2024-06-03T01:20:53.854 INFO:tasks.workunit.client.1.smithi044.stdout:5/384: chown d2/dc/d28 2 1 2024-06-03T01:20:53.854 INFO:tasks.workunit.client.1.smithi044.stdout:0/520: mknod d3/d9/d36/cb8 0 2024-06-03T01:20:53.854 INFO:tasks.workunit.client.1.smithi044.stdout:0/521: chown d3/da6/d21/f76 3 1 2024-06-03T01:20:53.855 INFO:tasks.workunit.client.1.smithi044.stdout:9/336: symlink d6/dd/d51/l6d 0 2024-06-03T01:20:53.855 INFO:tasks.workunit.client.1.smithi044.stdout:9/337: write d6/f44 [501337,104526] 0 2024-06-03T01:20:53.855 INFO:tasks.workunit.client.1.smithi044.stdout:9/338: read - d6/dd/d51/f62 zero size 2024-06-03T01:20:53.856 INFO:tasks.workunit.client.1.smithi044.stdout:4/374: creat d17/d23/d58/f85 x:0 0 0 2024-06-03T01:20:53.857 INFO:tasks.workunit.client.1.smithi044.stdout:4/375: write d17/d23/f71 [4892165,113915] 0 2024-06-03T01:20:53.857 INFO:tasks.workunit.client.1.smithi044.stdout:3/348: mkdir df/d13/d6d 0 2024-06-03T01:20:53.859 INFO:tasks.workunit.client.1.smithi044.stdout:6/549: creat d2/d23/d55/d5f/db2/d3f/fc5 x:0 0 0 2024-06-03T01:20:53.859 INFO:tasks.workunit.client.1.smithi044.stdout:6/550: chown d2/d23/d30/d1e/d34/d39/d7d/da7/db1 4 1 2024-06-03T01:20:53.859 INFO:tasks.workunit.client.1.smithi044.stdout:6/551: creat d2/d23/d44/fc6 x:0 0 0 2024-06-03T01:20:53.859 INFO:tasks.workunit.client.1.smithi044.stdout:7/492: rename d0/d3/d8/d32/d51 to d0/d3/d8/d95/d63/d89/da3 0 2024-06-03T01:20:53.860 INFO:tasks.workunit.client.1.smithi044.stdout:5/385: creat d2/d11/d1e/d3b/f79 x:0 0 0 2024-06-03T01:20:53.861 INFO:tasks.workunit.client.1.smithi044.stdout:4/376: symlink d17/d23/d58/l86 0 2024-06-03T01:20:53.861 INFO:tasks.workunit.client.1.smithi044.stdout:4/377: creat d17/d23/d28/f87 x:0 0 0 2024-06-03T01:20:53.861 INFO:tasks.workunit.client.1.smithi044.stdout:4/378: fsync d17/d23/d28/f6f 0 2024-06-03T01:20:53.862 INFO:tasks.workunit.client.1.smithi044.stdout:3/349: rmdir df/d13 39 2024-06-03T01:20:53.864 INFO:tasks.workunit.client.1.smithi044.stdout:6/552: rename d2/d6 to d2/d23/d30/d1e/d34/d39/d64/dc7 0 2024-06-03T01:20:53.864 INFO:tasks.workunit.client.1.smithi044.stdout:6/553: chown d2/d23/d30/d1e/d38/d6c/f12 1321827 1 2024-06-03T01:20:53.867 INFO:tasks.workunit.client.1.smithi044.stdout:3/350: link fd df/d13/d39/d3d/d43/f6e 0 2024-06-03T01:20:53.867 INFO:tasks.workunit.client.1.smithi044.stdout:6/554: getdents d2/d23/d30/d1e/d34/d39 0 2024-06-03T01:20:53.867 INFO:tasks.workunit.client.1.smithi044.stdout:6/555: write d2/d23/d30/d1e/d34/d39/d64/f9f [135874,60304] 0 2024-06-03T01:20:53.868 INFO:tasks.workunit.client.1.smithi044.stdout:5/386: rmdir d2/dc/d28 39 2024-06-03T01:20:53.868 INFO:tasks.workunit.client.1.smithi044.stdout:5/387: write d2/d11/d1e/d2f/f42 [220302,57876] 0 2024-06-03T01:20:53.869 INFO:tasks.workunit.client.1.smithi044.stdout:3/351: rename df/d13/d1b/f46 to df/d13/d39/d3d/d50/f6f 0 2024-06-03T01:20:53.869 INFO:tasks.workunit.client.1.smithi044.stdout:3/352: readlink df/d1e/l5d 0 2024-06-03T01:20:53.870 INFO:tasks.workunit.client.1.smithi044.stdout:5/388: link d2/d11/l4d d2/d11/d12/d2b/d44/l7a 0 2024-06-03T01:20:53.872 INFO:tasks.workunit.client.1.smithi044.stdout:5/389: unlink d2/d11/d1e/d3b/f5e 0 2024-06-03T01:20:53.944 INFO:tasks.workunit.client.1.smithi044.stdout:2/301: dread db/f19 [0,4194304] 0 2024-06-03T01:20:53.947 INFO:tasks.workunit.client.1.smithi044.stdout:9/339: dwrite f4 [0,4194304] 0 2024-06-03T01:20:53.950 INFO:tasks.workunit.client.1.smithi044.stdout:9/340: mkdir d6/dd/d6e 0 2024-06-03T01:20:53.950 INFO:tasks.workunit.client.1.smithi044.stdout:9/341: fsync d6/dd/d51/d16/d27/f64 0 2024-06-03T01:20:53.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/493: dwrite d0/d3/d8/d32/fa2 [0,4194304] 0 2024-06-03T01:20:53.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/494: truncate d0/d3/d8/d32/f90 610334 0 2024-06-03T01:20:53.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/495: readlink d0/d3/d8/d95/d63/d7f/d9c/l60 0 2024-06-03T01:20:53.951 INFO:tasks.workunit.client.1.smithi044.stdout:6/556: write d2/d23/d55/d5f/db2/d3f/fa6 [3530498,126082] 0 2024-06-03T01:20:53.951 INFO:tasks.workunit.client.1.smithi044.stdout:2/302: dread db/d1b/f36 [0,4194304] 0 2024-06-03T01:20:53.951 INFO:tasks.workunit.client.1.smithi044.stdout:9/342: symlink d6/dd/d51/d16/d27/d5e/l6f 0 2024-06-03T01:20:53.951 INFO:tasks.workunit.client.1.smithi044.stdout:9/343: chown d6/f12 69900186 1 2024-06-03T01:20:53.952 INFO:tasks.workunit.client.1.smithi044.stdout:3/353: getdents df/d13/d39/d3d/d50 0 2024-06-03T01:20:53.953 INFO:tasks.workunit.client.1.smithi044.stdout:4/379: dwrite d17/d23/d28/f6f [0,4194304] 0 2024-06-03T01:20:53.958 INFO:tasks.workunit.client.1.smithi044.stdout:5/390: dwrite d2/d11/d12/d2b/d44/f5f [0,4194304] 0 2024-06-03T01:20:53.959 INFO:tasks.workunit.client.1.smithi044.stdout:1/478: sync 2024-06-03T01:20:53.959 INFO:tasks.workunit.client.1.smithi044.stdout:8/435: sync 2024-06-03T01:20:53.959 INFO:tasks.workunit.client.1.smithi044.stdout:8/436: chown f2 14128634 1 2024-06-03T01:20:53.959 INFO:tasks.workunit.client.1.smithi044.stdout:8/437: dread - d5/d20/d33/f65 zero size 2024-06-03T01:20:53.960 INFO:tasks.workunit.client.1.smithi044.stdout:8/438: truncate d5/d20/d2b/f30 355224 0 2024-06-03T01:20:53.962 INFO:tasks.workunit.client.1.smithi044.stdout:3/354: truncate df/d1e/d3a/f4a 290469 0 2024-06-03T01:20:53.967 INFO:tasks.workunit.client.1.smithi044.stdout:4/380: unlink d17/d23/d28/f6f 0 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/496: rmdir d0 39 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/497: dread - d0/d3/d8/d95/d63/d89/d8b/f97 zero size 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/498: fdatasync d0/d16/f4d 0 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/439: write d5/d16/f7c [2427914,51032] 0 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/440: dread - d5/d20/d47/f49 zero size 2024-06-03T01:20:53.975 INFO:tasks.workunit.client.1.smithi044.stdout:3/355: dread df/d13/f1d [0,4194304] 0 2024-06-03T01:20:53.976 INFO:tasks.workunit.client.1.smithi044.stdout:2/303: dwrite db/d1b/d2a/f33 [0,4194304] 0 2024-06-03T01:20:53.976 INFO:tasks.workunit.client.1.smithi044.stdout:1/479: mknod d0/d3/d20/d7/ca1 0 2024-06-03T01:20:53.976 INFO:tasks.workunit.client.1.smithi044.stdout:1/480: creat d0/d3/d20/d7/d13/fa2 x:0 0 0 2024-06-03T01:20:53.976 INFO:tasks.workunit.client.1.smithi044.stdout:9/344: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:53.978 INFO:tasks.workunit.client.1.smithi044.stdout:4/381: symlink d17/d23/d6a/l88 0 2024-06-03T01:20:53.980 INFO:tasks.workunit.client.1.smithi044.stdout:3/356: mkdir df/d1e/d2a/d70 0 2024-06-03T01:20:53.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/391: dwrite f1 [4194304,4194304] 0 2024-06-03T01:20:53.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/392: dread - d2/dc/d28/d4c/f61 zero size 2024-06-03T01:20:53.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/393: chown d2/d11/d12/c71 9375434 1 2024-06-03T01:20:53.988 INFO:tasks.workunit.client.1.smithi044.stdout:4/382: mknod d17/d23/d28/d2b/c89 0 2024-06-03T01:20:53.988 INFO:tasks.workunit.client.1.smithi044.stdout:4/383: dread - d17/d1c/d75/f76 zero size 2024-06-03T01:20:53.990 INFO:tasks.workunit.client.1.smithi044.stdout:7/499: mkdir d0/d3/d8/d95/d63/d7f/d9c/da4 0 2024-06-03T01:20:53.991 INFO:tasks.workunit.client.1.smithi044.stdout:7/500: getdents d0/d3/d8/d8a 0 2024-06-03T01:20:53.993 INFO:tasks.workunit.client.1.smithi044.stdout:1/481: rename d0/d3/d14/d30/l9f to d0/la3 0 2024-06-03T01:20:53.993 INFO:tasks.workunit.client.1.smithi044.stdout:1/482: fdatasync d0/d3/d20/d40/d45/d4b/d5c/f9a 0 2024-06-03T01:20:53.995 INFO:tasks.workunit.client.1.smithi044.stdout:5/394: link d2/d11/d12/d2b/d4a/l4f d2/d11/d12/d2b/d4a/l7b 0 2024-06-03T01:20:53.995 INFO:tasks.workunit.client.1.smithi044.stdout:5/395: write d2/fa [521968,60291] 0 2024-06-03T01:20:53.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/384: symlink d17/d63/l8a 0 2024-06-03T01:20:53.997 INFO:tasks.workunit.client.1.smithi044.stdout:1/483: mknod d0/d3/d20/d40/d45/d6f/d8c/ca4 0 2024-06-03T01:20:53.999 INFO:tasks.workunit.client.1.smithi044.stdout:5/396: creat d2/dc/d28/f7c x:0 0 0 2024-06-03T01:20:54.001 INFO:tasks.workunit.client.1.smithi044.stdout:5/397: creat d2/d11/d12/d2b/d44/f7d x:0 0 0 2024-06-03T01:20:54.003 INFO:tasks.workunit.client.1.smithi044.stdout:8/441: dwrite d5/d20/d33/f4c [0,4194304] 0 2024-06-03T01:20:54.004 INFO:tasks.workunit.client.1.smithi044.stdout:5/398: rename f1 to d2/dc/d28/d4c/f7e 0 2024-06-03T01:20:54.004 INFO:tasks.workunit.client.1.smithi044.stdout:5/399: chown d2/d11/d12/f2d 1964 1 2024-06-03T01:20:54.009 INFO:tasks.workunit.client.1.smithi044.stdout:8/442: rename d5/d16/d1d/d6b/d6d to d5/d55/d74/d87 0 2024-06-03T01:20:54.009 INFO:tasks.workunit.client.1.smithi044.stdout:8/443: dread - d5/d55/d74/d87/f7b zero size 2024-06-03T01:20:54.009 INFO:tasks.workunit.client.1.smithi044.stdout:8/444: chown d5/d10/f34 23550 1 2024-06-03T01:20:54.009 INFO:tasks.workunit.client.1.smithi044.stdout:8/445: chown d5/d20/d47/d64/l66 96250378 1 2024-06-03T01:20:54.014 INFO:tasks.workunit.client.1.smithi044.stdout:4/385: read f5 [3251664,73282] 0 2024-06-03T01:20:54.014 INFO:tasks.workunit.client.1.smithi044.stdout:4/386: rename d17 to d17/d23/d58/d8b 22 2024-06-03T01:20:54.022 INFO:tasks.workunit.client.1.smithi044.stdout:1/484: dwrite d0/d3/d20/d40/d45/d4b/d5c/f63 [0,4194304] 0 2024-06-03T01:20:54.023 INFO:tasks.workunit.client.1.smithi044.stdout:5/400: dwrite d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:20:54.024 INFO:tasks.workunit.client.1.smithi044.stdout:8/446: dread d5/d20/d33/f3f [0,4194304] 0 2024-06-03T01:20:54.026 INFO:tasks.workunit.client.1.smithi044.stdout:5/401: symlink d2/d11/d1e/d21/d24/l7f 0 2024-06-03T01:20:54.027 INFO:tasks.workunit.client.1.smithi044.stdout:8/447: mkdir d5/d50/d79/d88 0 2024-06-03T01:20:54.027 INFO:tasks.workunit.client.1.smithi044.stdout:8/448: dread - d5/d16/f5e zero size 2024-06-03T01:20:54.028 INFO:tasks.workunit.client.1.smithi044.stdout:5/402: mkdir d2/d11/d1e/d80 0 2024-06-03T01:20:54.030 INFO:tasks.workunit.client.1.smithi044.stdout:5/403: symlink d2/dc/l81 0 2024-06-03T01:20:54.031 INFO:tasks.workunit.client.1.smithi044.stdout:5/404: write d2/d11/d1e/d21/d24/f69 [320921,60606] 0 2024-06-03T01:20:54.031 INFO:tasks.workunit.client.1.smithi044.stdout:5/405: chown d2/d11/d12/d2b/f72 289549510 1 2024-06-03T01:20:54.031 INFO:tasks.workunit.client.1.smithi044.stdout:5/406: read - d2/d11/d12/d2b/d44/f7d zero size 2024-06-03T01:20:54.031 INFO:tasks.workunit.client.1.smithi044.stdout:5/407: creat d2/d11/d12/d2b/d44/f82 x:0 0 0 2024-06-03T01:20:54.031 INFO:tasks.workunit.client.1.smithi044.stdout:5/408: fdatasync d2/f2e 0 2024-06-03T01:20:54.033 INFO:tasks.workunit.client.1.smithi044.stdout:7/501: dwrite d0/d3/d8/d95/d63/d7f/d9c/f83 [0,4194304] 0 2024-06-03T01:20:54.033 INFO:tasks.workunit.client.1.smithi044.stdout:7/502: write d0/d3/d8/d32/f48 [865944,19648] 0 2024-06-03T01:20:54.034 INFO:tasks.workunit.client.1.smithi044.stdout:5/409: rename d2/d11/d1e/d21/d24/l7f to d2/d11/d12/d2b/d44/l83 0 2024-06-03T01:20:54.035 INFO:tasks.workunit.client.1.smithi044.stdout:2/304: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:54.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/387: dwrite d17/f31 [0,4194304] 0 2024-06-03T01:20:54.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/388: truncate d17/f7c 145096 0 2024-06-03T01:20:54.036 INFO:tasks.workunit.client.1.smithi044.stdout:5/410: stat d2/d11/d1e/l6d 0 2024-06-03T01:20:54.036 INFO:tasks.workunit.client.1.smithi044.stdout:9/345: dwrite d6/dd/d51/d16/f37 [0,4194304] 0 2024-06-03T01:20:54.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/346: creat d6/dd/d51/d16/d4d/d55/d68/f70 x:0 0 0 2024-06-03T01:20:54.044 INFO:tasks.workunit.client.1.smithi044.stdout:8/449: read f2 [474837,112962] 0 2024-06-03T01:20:54.045 INFO:tasks.workunit.client.1.smithi044.stdout:8/450: mkdir d5/d20/d2b/d58/d89 0 2024-06-03T01:20:54.045 INFO:tasks.workunit.client.1.smithi044.stdout:8/451: fdatasync d5/d20/d2b/f35 0 2024-06-03T01:20:54.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/485: dwrite d0/d3/d20/d40/f55 [0,4194304] 0 2024-06-03T01:20:54.050 INFO:tasks.workunit.client.1.smithi044.stdout:2/305: write db/f3e [2430269,103664] 0 2024-06-03T01:20:54.053 INFO:tasks.workunit.client.1.smithi044.stdout:2/306: mknod db/c64 0 2024-06-03T01:20:54.056 INFO:tasks.workunit.client.1.smithi044.stdout:2/307: unlink db/d15/c5e 0 2024-06-03T01:20:54.067 INFO:tasks.workunit.client.1.smithi044.stdout:5/411: dwrite d2/d11/d12/d2b/d44/f5f [4194304,4194304] 0 2024-06-03T01:20:54.069 INFO:tasks.workunit.client.1.smithi044.stdout:5/412: dread d2/d11/d12/f1b [0,4194304] 0 2024-06-03T01:20:54.078 INFO:tasks.workunit.client.1.smithi044.stdout:5/413: write d2/dc/d28/d4c/f7e [4637393,99879] 0 2024-06-03T01:20:54.078 INFO:tasks.workunit.client.1.smithi044.stdout:5/414: fdatasync d2/d11/d1e/d21/d24/f38 0 2024-06-03T01:20:54.079 INFO:tasks.workunit.client.1.smithi044.stdout:8/452: dwrite d5/d55/d74/d87/f7b [0,4194304] 0 2024-06-03T01:20:54.079 INFO:tasks.workunit.client.1.smithi044.stdout:8/453: write f2 [956394,7072] 0 2024-06-03T01:20:54.080 INFO:tasks.workunit.client.1.smithi044.stdout:8/454: chown d5/d16/f6e 120605724 1 2024-06-03T01:20:54.080 INFO:tasks.workunit.client.1.smithi044.stdout:9/347: dwrite d6/d26/f4a [0,4194304] 0 2024-06-03T01:20:54.080 INFO:tasks.workunit.client.1.smithi044.stdout:2/308: dwrite db/d1b/d27/f39 [0,4194304] 0 2024-06-03T01:20:54.081 INFO:tasks.workunit.client.1.smithi044.stdout:4/389: dwrite d17/d23/d28/d2b/f2e [0,4194304] 0 2024-06-03T01:20:54.090 INFO:tasks.workunit.client.1.smithi044.stdout:4/390: write d17/d1c/f49 [3979446,41588] 0 2024-06-03T01:20:54.091 INFO:tasks.workunit.client.1.smithi044.stdout:8/455: getdents d5/d55 0 2024-06-03T01:20:54.099 INFO:tasks.workunit.client.1.smithi044.stdout:8/456: write d5/d16/f24 [825350,72000] 0 2024-06-03T01:20:54.105 INFO:tasks.workunit.client.1.smithi044.stdout:0/522: sync 2024-06-03T01:20:54.105 INFO:tasks.workunit.client.1.smithi044.stdout:6/557: sync 2024-06-03T01:20:54.105 INFO:tasks.workunit.client.1.smithi044.stdout:6/558: dread - d2/d23/d30/d1e/d34/d39/da8/fbb zero size 2024-06-03T01:20:54.106 INFO:tasks.workunit.client.1.smithi044.stdout:0/523: mknod d3/d6b/d95/dae/cb9 0 2024-06-03T01:20:54.107 INFO:tasks.workunit.client.1.smithi044.stdout:6/559: mknod d2/d23/d30/d1e/d34/d39/d64/d70/cc8 0 2024-06-03T01:20:54.107 INFO:tasks.workunit.client.1.smithi044.stdout:5/415: dwrite d2/d11/d3e/f59 [4194304,4194304] 0 2024-06-03T01:20:54.109 INFO:tasks.workunit.client.1.smithi044.stdout:6/560: truncate d2/d23/d30/d1e/d34/d39/fad 429804 0 2024-06-03T01:20:54.109 INFO:tasks.workunit.client.1.smithi044.stdout:6/561: readlink d2/d23/d30/d1e/d34/d39/d64/lb6 0 2024-06-03T01:20:54.111 INFO:tasks.workunit.client.1.smithi044.stdout:6/562: link d2/d23/d30/d1e/d38/d77/fbd d2/d23/d30/d1e/d34/d39/d64/d70/fc9 0 2024-06-03T01:20:54.111 INFO:tasks.workunit.client.1.smithi044.stdout:0/524: dread d3/f6d [0,4194304] 0 2024-06-03T01:20:54.112 INFO:tasks.workunit.client.1.smithi044.stdout:6/563: mkdir d2/d23/d55/d5f/db2/dca 0 2024-06-03T01:20:54.112 INFO:tasks.workunit.client.1.smithi044.stdout:0/525: symlink d3/da6/lba 0 2024-06-03T01:20:54.115 INFO:tasks.workunit.client.1.smithi044.stdout:0/526: symlink d3/da6/d56/d8e/lbb 0 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/416: read d2/d11/d12/f2d [2972123,54525] 0 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/417: chown d2/d11/d1e/d80 5044 1 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/418: chown d2/d11/d12/d2b/l78 65905 1 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/419: fdatasync d2/d11/d1e/d21/d24/f38 0 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/420: creat d2/d11/d12/d2b/f84 x:0 0 0 2024-06-03T01:20:54.117 INFO:tasks.workunit.client.1.smithi044.stdout:0/527: creat d3/da6/d21/d94/da3/dad/fbc x:0 0 0 2024-06-03T01:20:54.119 INFO:tasks.workunit.client.1.smithi044.stdout:5/421: mknod d2/d11/c85 0 2024-06-03T01:20:54.120 INFO:tasks.workunit.client.1.smithi044.stdout:4/391: dwrite d17/d23/d28/d2b/f24 [0,4194304] 0 2024-06-03T01:20:54.122 INFO:tasks.workunit.client.1.smithi044.stdout:0/528: rename d3/d9/d55/d73/f9d to d3/d5c/fbd 0 2024-06-03T01:20:54.122 INFO:tasks.workunit.client.1.smithi044.stdout:0/529: chown d3/da6/d56/d62/fa1 7 1 2024-06-03T01:20:54.122 INFO:tasks.workunit.client.1.smithi044.stdout:9/348: dwrite d6/dd/d51/d16/d4d/d55/d68/f70 [0,4194304] 0 2024-06-03T01:20:54.123 INFO:tasks.workunit.client.1.smithi044.stdout:8/457: dwrite d5/d20/d2b/f48 [4194304,4194304] 0 2024-06-03T01:20:54.123 INFO:tasks.workunit.client.1.smithi044.stdout:2/309: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:54.124 INFO:tasks.workunit.client.1.smithi044.stdout:5/422: mknod d2/dc/c86 0 2024-06-03T01:20:54.125 INFO:tasks.workunit.client.1.smithi044.stdout:4/392: symlink d17/d23/d28/l8c 0 2024-06-03T01:20:54.130 INFO:tasks.workunit.client.1.smithi044.stdout:8/458: unlink d5/d10/c60 0 2024-06-03T01:20:54.130 INFO:tasks.workunit.client.1.smithi044.stdout:8/459: creat d5/d55/d56/f8a x:0 0 0 2024-06-03T01:20:54.130 INFO:tasks.workunit.client.1.smithi044.stdout:8/460: stat d5/d16/d81 0 2024-06-03T01:20:54.134 INFO:tasks.workunit.client.1.smithi044.stdout:5/423: link d2/d11/d1e/d2f/l56 d2/dc/d28/d66/l87 0 2024-06-03T01:20:54.134 INFO:tasks.workunit.client.1.smithi044.stdout:4/393: mkdir d17/d23/d8d 0 2024-06-03T01:20:54.134 INFO:tasks.workunit.client.1.smithi044.stdout:4/394: stat d17/d23/f67 0 2024-06-03T01:20:54.134 INFO:tasks.workunit.client.1.smithi044.stdout:0/530: dread d3/f8 [0,4194304] 0 2024-06-03T01:20:54.134 INFO:tasks.workunit.client.1.smithi044.stdout:0/531: write d3/d9/d55/d5a/f34 [609915,84325] 0 2024-06-03T01:20:54.135 INFO:tasks.workunit.client.1.smithi044.stdout:8/461: creat d5/d20/d47/d64/f8b x:0 0 0 2024-06-03T01:20:54.135 INFO:tasks.workunit.client.1.smithi044.stdout:8/462: fsync d5/f2d 0 2024-06-03T01:20:54.135 INFO:tasks.workunit.client.1.smithi044.stdout:8/463: dread - d5/d20/d33/f65 zero size 2024-06-03T01:20:54.136 INFO:tasks.workunit.client.1.smithi044.stdout:5/424: truncate d2/d11/d3e/f40 1613317 0 2024-06-03T01:20:54.137 INFO:tasks.workunit.client.1.smithi044.stdout:0/532: link d3/da6/d56/d8e/lbb d3/da6/d20/d25/lbe 0 2024-06-03T01:20:54.138 INFO:tasks.workunit.client.1.smithi044.stdout:0/533: write d3/d5c/fa2 [155075,127664] 0 2024-06-03T01:20:54.138 INFO:tasks.workunit.client.1.smithi044.stdout:8/464: mknod d5/d16/d1d/d6b/c8c 0 2024-06-03T01:20:54.139 INFO:tasks.workunit.client.1.smithi044.stdout:5/425: creat d2/dc/d28/d4c/f88 x:0 0 0 2024-06-03T01:20:54.140 INFO:tasks.workunit.client.1.smithi044.stdout:6/564: dwrite d2/d23/d55/d5f/db2/d3f/fa1 [0,4194304] 0 2024-06-03T01:20:54.140 INFO:tasks.workunit.client.1.smithi044.stdout:2/310: dwrite db/d1b/d27/f39 [0,4194304] 0 2024-06-03T01:20:54.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/465: unlink d5/d10/f73 0 2024-06-03T01:20:54.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/466: chown d5/d10/c1a 2047457 1 2024-06-03T01:20:54.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/467: read d5/d16/f24 [783687,39273] 0 2024-06-03T01:20:54.146 INFO:tasks.workunit.client.1.smithi044.stdout:2/311: symlink db/d1b/d5d/l65 0 2024-06-03T01:20:54.147 INFO:tasks.workunit.client.1.smithi044.stdout:2/312: creat db/d1b/d3f/f66 x:0 0 0 2024-06-03T01:20:54.147 INFO:tasks.workunit.client.1.smithi044.stdout:8/468: mknod d5/d55/d56/d5c/d68/c8d 0 2024-06-03T01:20:54.148 INFO:tasks.workunit.client.1.smithi044.stdout:6/565: getdents d2/d23/d30/d1e/d34/d39/d64 0 2024-06-03T01:20:54.148 INFO:tasks.workunit.client.1.smithi044.stdout:6/566: fsync d2/d23/d30/d1e/d34/d39/d64/dc7/f5d 0 2024-06-03T01:20:54.148 INFO:tasks.workunit.client.1.smithi044.stdout:5/426: write d2/d11/d12/d2b/d44/f5f [4094512,46159] 0 2024-06-03T01:20:54.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/313: dread db/f48 [0,4194304] 0 2024-06-03T01:20:54.151 INFO:tasks.workunit.client.1.smithi044.stdout:8/469: truncate d5/d20/f69 2075000 0 2024-06-03T01:20:54.153 INFO:tasks.workunit.client.1.smithi044.stdout:6/567: mkdir d2/d23/d44/dcb 0 2024-06-03T01:20:54.153 INFO:tasks.workunit.client.1.smithi044.stdout:6/568: readlink d2/d23/d30/d1e/d38/d6c/l1b 0 2024-06-03T01:20:54.154 INFO:tasks.workunit.client.1.smithi044.stdout:6/569: dread d2/d23/d30/f1a [0,4194304] 0 2024-06-03T01:20:54.155 INFO:tasks.workunit.client.1.smithi044.stdout:2/314: rename db/d1b/d3f/d47 to db/d1b/d2a/d67 0 2024-06-03T01:20:54.155 INFO:tasks.workunit.client.1.smithi044.stdout:2/315: readlink db/d1b/l1a 0 2024-06-03T01:20:54.156 INFO:tasks.workunit.client.1.smithi044.stdout:2/316: dread - db/d1b/d3f/f59 zero size 2024-06-03T01:20:54.157 INFO:tasks.workunit.client.1.smithi044.stdout:6/570: dread d2/f27 [0,4194304] 0 2024-06-03T01:20:54.157 INFO:tasks.workunit.client.1.smithi044.stdout:6/571: read - d2/d23/d30/d1e/d38/d6c/f89 zero size 2024-06-03T01:20:54.158 INFO:tasks.workunit.client.1.smithi044.stdout:6/572: mkdir d2/d23/d55/d5f/d99/dcc 0 2024-06-03T01:20:54.159 INFO:tasks.workunit.client.1.smithi044.stdout:4/395: dwrite d17/f31 [0,4194304] 0 2024-06-03T01:20:54.159 INFO:tasks.workunit.client.1.smithi044.stdout:0/534: dwrite d3/d9/d55/d7b/fb0 [0,4194304] 0 2024-06-03T01:20:54.160 INFO:tasks.workunit.client.1.smithi044.stdout:2/317: getdents db 0 2024-06-03T01:20:54.163 INFO:tasks.workunit.client.1.smithi044.stdout:6/573: mkdir d2/d23/d30/d1e/d34/d39/dcd 0 2024-06-03T01:20:54.170 INFO:tasks.workunit.client.1.smithi044.stdout:8/470: write d5/f25 [335624,42971] 0 2024-06-03T01:20:54.171 INFO:tasks.workunit.client.1.smithi044.stdout:8/471: rename d5/d16/d1d/f1f to d5/d20/d47/f8e 0 2024-06-03T01:20:54.171 INFO:tasks.workunit.client.1.smithi044.stdout:8/472: chown d5/d16/f3a 1 1 2024-06-03T01:20:54.171 INFO:tasks.workunit.client.1.smithi044.stdout:4/396: dread d17/d23/d28/f3e [0,4194304] 0 2024-06-03T01:20:54.171 INFO:tasks.workunit.client.1.smithi044.stdout:4/397: write d17/d23/d35/f3c [742996,50633] 0 2024-06-03T01:20:54.173 INFO:tasks.workunit.client.1.smithi044.stdout:4/398: mkdir d17/d23/d28/d2b/d8e 0 2024-06-03T01:20:54.173 INFO:tasks.workunit.client.1.smithi044.stdout:4/399: chown d17/d23/d6a 29955671 1 2024-06-03T01:20:54.176 INFO:tasks.workunit.client.1.smithi044.stdout:6/574: read d2/d23/d30/d1e/d38/d6c/f3b [3033649,54574] 0 2024-06-03T01:20:54.177 INFO:tasks.workunit.client.1.smithi044.stdout:5/427: dwrite d2/d11/d1e/d2f/f6e [0,4194304] 0 2024-06-03T01:20:54.182 INFO:tasks.workunit.client.1.smithi044.stdout:6/575: dread d2/d23/d44/f54 [0,4194304] 0 2024-06-03T01:20:54.183 INFO:tasks.workunit.client.0.smithi032.stdout:7/19: sync 2024-06-03T01:20:54.183 INFO:tasks.workunit.client.0.smithi032.stdout:7/20: creat f7 x:0 0 0 2024-06-03T01:20:54.183 INFO:tasks.workunit.client.0.smithi032.stdout:7/21: readlink l2 0 2024-06-03T01:20:54.183 INFO:tasks.workunit.client.0.smithi032.stdout:7/22: dread - f6 zero size 2024-06-03T01:20:54.186 INFO:tasks.workunit.client.0.smithi032.stdout:7/23: link f7 f8 0 2024-06-03T01:20:54.186 INFO:tasks.workunit.client.0.smithi032.stdout:7/24: stat f6 0 2024-06-03T01:20:54.187 INFO:tasks.workunit.client.0.smithi032.stdout:7/25: symlink l9 0 2024-06-03T01:20:54.188 INFO:tasks.workunit.client.0.smithi032.stdout:7/26: chown f7 17940373 1 2024-06-03T01:20:54.188 INFO:tasks.workunit.client.0.smithi032.stdout:7/27: write f6 [1002787,62086] 0 2024-06-03T01:20:54.188 INFO:tasks.workunit.client.0.smithi032.stdout:7/28: read - f7 zero size 2024-06-03T01:20:54.188 INFO:tasks.workunit.client.0.smithi032.stdout:7/29: write f8 [521662,44974] 0 2024-06-03T01:20:54.188 INFO:tasks.workunit.client.0.smithi032.stdout:7/30: creat fa x:0 0 0 2024-06-03T01:20:54.190 INFO:tasks.workunit.client.1.smithi044.stdout:4/400: dread d17/d23/f57 [0,4194304] 0 2024-06-03T01:20:54.190 INFO:tasks.workunit.client.1.smithi044.stdout:4/401: write d17/f7c [839892,24198] 0 2024-06-03T01:20:54.190 INFO:tasks.workunit.client.1.smithi044.stdout:4/402: creat d17/d23/d28/d2b/f8f x:0 0 0 2024-06-03T01:20:54.191 INFO:tasks.workunit.client.1.smithi044.stdout:4/403: write d17/d23/d59/f7b [533696,8299] 0 2024-06-03T01:20:54.191 INFO:tasks.workunit.client.1.smithi044.stdout:4/404: write d17/d23/f68 [4378484,39746] 0 2024-06-03T01:20:54.205 INFO:tasks.workunit.client.1.smithi044.stdout:6/576: dwrite d2/d23/f35 [0,4194304] 0 2024-06-03T01:20:54.205 INFO:tasks.workunit.client.1.smithi044.stdout:6/577: creat d2/d23/d44/d4d/da0/fce x:0 0 0 2024-06-03T01:20:54.205 INFO:tasks.workunit.client.1.smithi044.stdout:6/578: dread - d2/d23/d30/d1e/d38/d6c/f21 zero size 2024-06-03T01:20:54.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/579: symlink d2/d23/d30/d1e/d34/lcf 0 2024-06-03T01:20:54.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/428: dwrite d2/d11/d12/d2b/d4a/d57/f76 [0,4194304] 0 2024-06-03T01:20:54.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/429: fsync d2/dc/f62 0 2024-06-03T01:20:54.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/430: chown d2/d11/d1e/d2f/f42 2960 1 2024-06-03T01:20:54.207 INFO:tasks.workunit.client.0.smithi032.stdout:7/31: dwrite fa [0,4194304] 0 2024-06-03T01:20:54.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/535: dwrite d3/da6/d56/f60 [0,4194304] 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/318: dwrite db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/319: chown db 21334 1 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:0/536: chown d3/da6/d21/d94/da3 23737 1 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:0/537: write d3/da6/d21/f76 [1937099,76875] 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:0/538: stat d3/da6/d20/f5b 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:8/473: dwrite d5/d20/d33/f65 [0,4194304] 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:8/474: chown d5/d20/d47 7 1 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:6/580: dread d2/d23/d55/d5f/fa9 [0,4194304] 0 2024-06-03T01:20:54.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/431: mknod d2/d11/d12/d2b/d44/c89 0 2024-06-03T01:20:54.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/539: dread d3/d9/d36/f48 [0,4194304] 0 2024-06-03T01:20:54.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/540: write d3/f14 [4787003,25859] 0 2024-06-03T01:20:54.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/541: write d3/d5c/fa2 [1773613,45234] 0 2024-06-03T01:20:54.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/542: truncate d3/da6/d21/f76 2153905 0 2024-06-03T01:20:54.213 INFO:tasks.workunit.client.1.smithi044.stdout:0/543: dread d3/f8 [0,4194304] 0 2024-06-03T01:20:54.213 INFO:tasks.workunit.client.1.smithi044.stdout:2/320: creat db/d45/f68 x:0 0 0 2024-06-03T01:20:54.220 INFO:tasks.workunit.client.1.smithi044.stdout:4/405: getdents d17/d23/d28/d2b 0 2024-06-03T01:20:54.225 INFO:tasks.workunit.client.1.smithi044.stdout:8/475: symlink d5/d50/l8f 0 2024-06-03T01:20:54.226 INFO:tasks.workunit.client.1.smithi044.stdout:8/476: stat d5/d16/d1d/f38 0 2024-06-03T01:20:54.227 INFO:tasks.workunit.client.1.smithi044.stdout:6/581: mknod d2/d23/d30/d1e/d38/cd0 0 2024-06-03T01:20:54.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/432: symlink d2/d11/d1e/l8a 0 2024-06-03T01:20:54.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/433: chown d2/d11/d12/f2d 81536 1 2024-06-03T01:20:54.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/434: fsync d2/d11/d12/d2b/d44/f7d 0 2024-06-03T01:20:54.229 INFO:tasks.workunit.client.0.smithi032.stdout:7/32: dwrite f8 [0,4194304] 0 2024-06-03T01:20:54.230 INFO:tasks.workunit.client.1.smithi044.stdout:6/582: write d2/d23/d55/d5f/fa9 [24553,32587] 0 2024-06-03T01:20:54.230 INFO:tasks.workunit.client.1.smithi044.stdout:0/544: mkdir d3/da6/d20/d25/dbf 0 2024-06-03T01:20:54.230 INFO:tasks.workunit.client.1.smithi044.stdout:4/406: dread d17/d1c/d75/f5d [0,4194304] 0 2024-06-03T01:20:54.231 INFO:tasks.workunit.client.0.smithi032.stdout:7/33: rename l5 to lb 0 2024-06-03T01:20:54.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/321: symlink db/dd/l69 0 2024-06-03T01:20:54.233 INFO:tasks.workunit.client.1.smithi044.stdout:8/477: link d5/d16/d1d/d32/l4f d5/d50/d79/d88/l90 0 2024-06-03T01:20:54.233 INFO:tasks.workunit.client.1.smithi044.stdout:8/478: readlink d5/d16/d1d/d32/l4f 0 2024-06-03T01:20:54.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/435: mknod d2/dc/d28/d66/c8b 0 2024-06-03T01:20:54.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/436: creat d2/dc/f8c x:0 0 0 2024-06-03T01:20:54.236 INFO:tasks.workunit.client.1.smithi044.stdout:2/322: mknod db/d1b/d2a/d52/d54/c6a 0 2024-06-03T01:20:54.239 INFO:tasks.workunit.client.0.smithi032.stdout:7/34: getdents . 0 2024-06-03T01:20:54.239 INFO:tasks.workunit.client.1.smithi044.stdout:5/437: getdents d2/d11/d12/d2b/d4a 0 2024-06-03T01:20:54.239 INFO:tasks.workunit.client.1.smithi044.stdout:5/438: write d2/dc/f49 [1060732,34480] 0 2024-06-03T01:20:54.240 INFO:tasks.workunit.client.1.smithi044.stdout:2/323: symlink db/d15/d49/l6b 0 2024-06-03T01:20:54.241 INFO:tasks.workunit.client.0.smithi032.stdout:7/35: mkdir dc 0 2024-06-03T01:20:54.241 INFO:tasks.workunit.client.1.smithi044.stdout:4/407: dread d17/f31 [0,4194304] 0 2024-06-03T01:20:54.241 INFO:tasks.workunit.client.1.smithi044.stdout:4/408: write d17/d23/d28/f87 [13774,24753] 0 2024-06-03T01:20:54.241 INFO:tasks.workunit.client.1.smithi044.stdout:4/409: dread - d17/d23/d28/d2b/f8f zero size 2024-06-03T01:20:54.242 INFO:tasks.workunit.client.1.smithi044.stdout:2/324: mkdir db/d15/d49/d6c 0 2024-06-03T01:20:54.242 INFO:tasks.workunit.client.0.smithi032.stdout:7/36: mkdir dc/dd 0 2024-06-03T01:20:54.243 INFO:tasks.workunit.client.1.smithi044.stdout:4/410: symlink d17/d23/d28/d2b/d8e/l90 0 2024-06-03T01:20:54.245 INFO:tasks.workunit.client.1.smithi044.stdout:2/325: rename db/dd/l69 to db/d1b/d2a/d2f/d4c/l6d 0 2024-06-03T01:20:54.246 INFO:tasks.workunit.client.1.smithi044.stdout:2/326: fdatasync db/f12 0 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/411: symlink d17/d1c/d70/d7d/l91 0 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/412: read f5 [1748563,125009] 0 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/413: stat d17/d23/d6a 0 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/414: chown d17/d1c/d70/d7d/l83 102 1 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/415: truncate d17/d23/d59/f81 1238350 0 2024-06-03T01:20:54.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/416: write d17/d1c/d75/f43 [520523,38962] 0 2024-06-03T01:20:54.248 INFO:tasks.workunit.client.1.smithi044.stdout:2/327: mknod db/d1b/d2a/d52/d54/c6e 0 2024-06-03T01:20:54.248 INFO:tasks.workunit.client.1.smithi044.stdout:2/328: write db/f19 [1769880,122499] 0 2024-06-03T01:20:54.248 INFO:tasks.workunit.client.1.smithi044.stdout:9/349: sync 2024-06-03T01:20:54.248 INFO:tasks.workunit.client.1.smithi044.stdout:1/486: sync 2024-06-03T01:20:54.249 INFO:tasks.workunit.client.1.smithi044.stdout:9/350: dread - d6/dd/d51/d3b/f6a zero size 2024-06-03T01:20:54.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/487: read d0/d3/d20/d7/d2f/f33 [409686,33880] 0 2024-06-03T01:20:54.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/488: chown d0/d3/d20/d40/d45/d4b/c9d 0 1 2024-06-03T01:20:54.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/329: mknod db/d15/d49/c6f 0 2024-06-03T01:20:54.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/330: write db/f3e [3782479,4722] 0 2024-06-03T01:20:54.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/331: chown db/d15/c35 25 1 2024-06-03T01:20:54.250 INFO:tasks.workunit.client.1.smithi044.stdout:1/489: mknod d0/d3/d20/d40/d45/d4b/d5a/ca5 0 2024-06-03T01:20:54.251 INFO:tasks.workunit.client.1.smithi044.stdout:7/503: sync 2024-06-03T01:20:54.251 INFO:tasks.workunit.client.1.smithi044.stdout:3/357: sync 2024-06-03T01:20:54.252 INFO:tasks.workunit.client.1.smithi044.stdout:7/504: write d0/d3/d8/f6d [1244096,111488] 0 2024-06-03T01:20:54.256 INFO:tasks.workunit.client.1.smithi044.stdout:2/332: creat db/d15/f70 x:0 0 0 2024-06-03T01:20:54.256 INFO:tasks.workunit.client.1.smithi044.stdout:3/358: mkdir df/d13/d39/d3d/d43/d71 0 2024-06-03T01:20:54.257 INFO:tasks.workunit.client.1.smithi044.stdout:7/505: rename d0/d3/fa to d0/d3/d8/d95/d63/d89/fa5 0 2024-06-03T01:20:54.259 INFO:tasks.workunit.client.1.smithi044.stdout:3/359: truncate fe 125164 0 2024-06-03T01:20:54.259 INFO:tasks.workunit.client.1.smithi044.stdout:3/360: chown df/c3c 2 1 2024-06-03T01:20:54.260 INFO:tasks.workunit.client.1.smithi044.stdout:9/351: dread d6/d26/f4a [0,4194304] 0 2024-06-03T01:20:54.261 INFO:tasks.workunit.client.1.smithi044.stdout:7/506: link d0/d3/d8/d95/f28 d0/d3/d8/d95/fa6 0 2024-06-03T01:20:54.261 INFO:tasks.workunit.client.1.smithi044.stdout:7/507: fdatasync d0/d3/d8/d95/f53 0 2024-06-03T01:20:54.261 INFO:tasks.workunit.client.1.smithi044.stdout:7/508: chown d0/d3/d8/d32/c8c 2682510 1 2024-06-03T01:20:54.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/361: mkdir df/d13/d39/d3d/d43/d72 0 2024-06-03T01:20:54.263 INFO:tasks.workunit.client.1.smithi044.stdout:9/352: mkdir d6/dd/d51/d16/d27/d5e/d71 0 2024-06-03T01:20:54.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/362: creat df/d13/d1b/f73 x:0 0 0 2024-06-03T01:20:54.266 INFO:tasks.workunit.client.1.smithi044.stdout:1/490: write d0/d3/d14/d81/d47/f4c [4947848,114408] 0 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:2/333: dread db/d1b/d2a/f33 [0,4194304] 0 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:2/334: write db/d1b/d27/f5f [291330,50303] 0 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:2/335: chown db/d1b/d27/f5f 11 1 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:1/491: creat d0/d16/d64/d6a/d9e/fa6 x:0 0 0 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:1/492: chown d0/d3/d20/d7/d13/d2d/f4d 982562563 1 2024-06-03T01:20:54.269 INFO:tasks.workunit.client.1.smithi044.stdout:3/363: dread df/d13/f16 [0,4194304] 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:6/583: dwrite d2/d23/d55/d5f/db2/d3f/f9a [0,4194304] 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:6/584: write d2/f14 [3964391,115004] 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:6/585: dread - d2/d23/d30/d1e/d34/d39/d64/f9e zero size 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:6/586: fsync d2/d23/d30/d1e/d34/d39/d9b/f9c 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/479: dwrite d5/d55/d56/f75 [0,4194304] 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/480: fsync d5/d20/d47/f49 0 2024-06-03T01:20:54.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/481: fsync d5/f6a 0 2024-06-03T01:20:54.273 INFO:tasks.workunit.client.1.smithi044.stdout:5/439: dwrite d2/d11/d12/d2b/d4a/f5d [0,4194304] 0 2024-06-03T01:20:54.273 INFO:tasks.workunit.client.1.smithi044.stdout:4/417: dwrite d17/d1c/f49 [0,4194304] 0 2024-06-03T01:20:54.274 INFO:tasks.workunit.client.1.smithi044.stdout:2/336: unlink db/f3e 0 2024-06-03T01:20:54.275 INFO:tasks.workunit.client.1.smithi044.stdout:0/545: dwrite d3/d5c/fa2 [0,4194304] 0 2024-06-03T01:20:54.277 INFO:tasks.workunit.client.1.smithi044.stdout:8/482: dread d5/d16/f24 [0,4194304] 0 2024-06-03T01:20:54.277 INFO:tasks.workunit.client.1.smithi044.stdout:8/483: creat d5/d20/d47/f91 x:0 0 0 2024-06-03T01:20:54.279 INFO:tasks.workunit.client.1.smithi044.stdout:1/493: rename d0/d3/d20/d7/d13/d2d/f4d to d0/d3/d20/d7/fa7 0 2024-06-03T01:20:54.279 INFO:tasks.workunit.client.1.smithi044.stdout:1/494: write d0/d3/d20/d7/d4e/f78 [4953188,47018] 0 2024-06-03T01:20:54.280 INFO:tasks.workunit.client.1.smithi044.stdout:0/546: dread d3/f69 [0,4194304] 0 2024-06-03T01:20:54.280 INFO:tasks.workunit.client.1.smithi044.stdout:6/587: mkdir d2/d23/d30/d1e/d34/d39/dcd/dd1 0 2024-06-03T01:20:54.282 INFO:tasks.workunit.client.1.smithi044.stdout:7/509: dwrite d0/d3/f47 [0,4194304] 0 2024-06-03T01:20:54.285 INFO:tasks.workunit.client.0.smithi032.stdout:1/10: sync 2024-06-03T01:20:54.286 INFO:tasks.workunit.client.1.smithi044.stdout:2/337: mknod db/d45/d4e/c71 0 2024-06-03T01:20:54.287 INFO:tasks.workunit.client.0.smithi032.stdout:1/11: rename c0 to c2 0 2024-06-03T01:20:54.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/12: symlink l3 0 2024-06-03T01:20:54.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/13: dwrite - no filename 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/547: write d3/f6e [3304414,719] 0 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/548: write d3/da6/d21/f76 [2160474,25084] 0 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/549: dread - d3/d5c/f8f zero size 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/550: chown d3/d9/d36/d82/f9c 0 1 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/551: creat d3/da6/fc0 x:0 0 0 2024-06-03T01:20:54.290 INFO:tasks.workunit.client.1.smithi044.stdout:0/552: fdatasync d3/d9/f1d 0 2024-06-03T01:20:54.291 INFO:tasks.workunit.client.1.smithi044.stdout:0/553: chown d3/f81 8345686 1 2024-06-03T01:20:54.291 INFO:tasks.workunit.client.1.smithi044.stdout:0/554: chown d3/d5c/fa2 97795918 1 2024-06-03T01:20:54.291 INFO:tasks.workunit.client.1.smithi044.stdout:0/555: dread - d3/da6/fb1 zero size 2024-06-03T01:20:54.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/14: unlink l3 0 2024-06-03T01:20:54.296 INFO:tasks.workunit.client.1.smithi044.stdout:5/440: dwrite d2/d11/d12/f1b [0,4194304] 0 2024-06-03T01:20:54.296 INFO:tasks.workunit.client.1.smithi044.stdout:5/441: fdatasync d2/d11/d3e/f59 0 2024-06-03T01:20:54.296 INFO:tasks.workunit.client.1.smithi044.stdout:5/442: chown d2/dc/d28/d4c/f4e 0 1 2024-06-03T01:20:54.296 INFO:tasks.workunit.client.1.smithi044.stdout:5/443: write d2/d11/d12/d2b/d44/f7d [321060,107044] 0 2024-06-03T01:20:54.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/37: dwrite f6 [0,4194304] 0 2024-06-03T01:20:54.308 INFO:tasks.workunit.client.0.smithi032.stdout:7/38: dread f6 [0,4194304] 0 2024-06-03T01:20:54.308 INFO:tasks.workunit.client.0.smithi032.stdout:7/39: write f8 [4352854,68691] 0 2024-06-03T01:20:54.314 INFO:tasks.workunit.client.0.smithi032.stdout:6/46: sync 2024-06-03T01:20:54.314 INFO:tasks.workunit.client.0.smithi032.stdout:9/91: sync 2024-06-03T01:20:54.314 INFO:tasks.workunit.client.1.smithi044.stdout:4/418: rename d17/d23/d28/f2c to d17/d23/d28/d2b/f92 0 2024-06-03T01:20:54.316 INFO:tasks.workunit.client.0.smithi032.stdout:9/92: truncate d0/f5 2611581 0 2024-06-03T01:20:54.317 INFO:tasks.workunit.client.1.smithi044.stdout:6/588: symlink d2/d23/d55/d5f/ld2 0 2024-06-03T01:20:54.317 INFO:tasks.workunit.client.1.smithi044.stdout:6/589: chown d2/d23/d30/f7f 4 1 2024-06-03T01:20:54.318 INFO:tasks.workunit.client.0.smithi032.stdout:6/47: getdents d8 0 2024-06-03T01:20:54.318 INFO:tasks.workunit.client.1.smithi044.stdout:6/590: dread d2/d23/d44/f54 [0,4194304] 0 2024-06-03T01:20:54.318 INFO:tasks.workunit.client.1.smithi044.stdout:6/591: dread - d2/d23/d30/d1e/d38/d6c/f84 zero size 2024-06-03T01:20:54.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/444: dwrite d2/dc/f16 [0,4194304] 0 2024-06-03T01:20:54.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/40: write f7 [3040346,130369] 0 2024-06-03T01:20:54.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/41: getdents dc/dd 0 2024-06-03T01:20:54.320 INFO:tasks.workunit.client.0.smithi032.stdout:9/93: rename d0/f3 to d0/f1b 0 2024-06-03T01:20:54.320 INFO:tasks.workunit.client.0.smithi032.stdout:9/94: rename d0/d11 to d0/d11/d1c 22 2024-06-03T01:20:54.321 INFO:tasks.workunit.client.0.smithi032.stdout:6/48: symlink d8/le 0 2024-06-03T01:20:54.323 INFO:tasks.workunit.client.0.smithi032.stdout:7/42: creat dc/dd/fe x:0 0 0 2024-06-03T01:20:54.325 INFO:tasks.workunit.client.1.smithi044.stdout:7/510: mknod d0/d3/d8/d32/ca7 0 2024-06-03T01:20:54.325 INFO:tasks.workunit.client.0.smithi032.stdout:6/49: write d8/f9 [383052,105924] 0 2024-06-03T01:20:54.325 INFO:tasks.workunit.client.0.smithi032.stdout:6/50: truncate d8/f9 1401916 0 2024-06-03T01:20:54.326 INFO:tasks.workunit.client.1.smithi044.stdout:2/338: mkdir db/d15/d49/d6c/d72 0 2024-06-03T01:20:54.326 INFO:tasks.workunit.client.0.smithi032.stdout:7/43: symlink dc/dd/lf 0 2024-06-03T01:20:54.329 INFO:tasks.workunit.client.1.smithi044.stdout:0/556: mkdir d3/da6/dc1 0 2024-06-03T01:20:54.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/51: link d8/fd d8/ff 0 2024-06-03T01:20:54.331 INFO:tasks.workunit.client.1.smithi044.stdout:4/419: creat d17/d23/d28/d2b/f93 x:0 0 0 2024-06-03T01:20:54.333 INFO:tasks.workunit.client.0.smithi032.stdout:9/95: write d0/d11/f14 [1406720,91013] 0 2024-06-03T01:20:54.334 INFO:tasks.workunit.client.1.smithi044.stdout:1/495: mknod d0/d3/d20/d7/d4e/ca8 0 2024-06-03T01:20:54.335 INFO:tasks.workunit.client.0.smithi032.stdout:9/96: mkdir d0/d11/d1d 0 2024-06-03T01:20:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:54 smithi032 ceph-mon[29360]: pgmap v256: 65 pgs: 65 active+clean; 705 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 53 MiB/s rd, 74 MiB/s wr, 888 op/s 2024-06-03T01:20:54.336 INFO:tasks.workunit.client.1.smithi044.stdout:6/592: mknod d2/d23/d30/d1e/d34/d39/d64/d70/cd3 0 2024-06-03T01:20:54.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/420: dread d17/d23/d28/f87 [0,4194304] 0 2024-06-03T01:20:54.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/421: creat d17/f94 x:0 0 0 2024-06-03T01:20:54.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/422: readlink d17/l60 0 2024-06-03T01:20:54.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/423: creat d17/d23/d58/f95 x:0 0 0 2024-06-03T01:20:54.337 INFO:tasks.workunit.client.0.smithi032.stdout:7/44: dread fa [0,4194304] 0 2024-06-03T01:20:54.340 INFO:tasks.workunit.client.0.smithi032.stdout:7/45: mknod dc/c10 0 2024-06-03T01:20:54.340 INFO:tasks.workunit.client.0.smithi032.stdout:7/46: truncate fa 4576559 0 2024-06-03T01:20:54.342 INFO:tasks.workunit.client.1.smithi044.stdout:7/511: mkdir d0/d3/da8 0 2024-06-03T01:20:54.343 INFO:tasks.workunit.client.1.smithi044.stdout:5/445: unlink d2/dc/f16 0 2024-06-03T01:20:54.344 INFO:tasks.workunit.client.0.smithi032.stdout:9/97: dread d0/f1b [0,4194304] 0 2024-06-03T01:20:54.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/557: mkdir d3/da6/dc1/dc2 0 2024-06-03T01:20:54.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/558: fsync d3/da6/fb1 0 2024-06-03T01:20:54.345 INFO:tasks.workunit.client.0.smithi032.stdout:9/98: creat d0/d11/d1d/f1e x:0 0 0 2024-06-03T01:20:54.345 INFO:tasks.workunit.client.0.smithi032.stdout:9/99: rename d0/d11/d1d to d0/d11/d1d/d1f 22 2024-06-03T01:20:54.346 INFO:tasks.workunit.client.0.smithi032.stdout:9/100: dread - d0/f19 zero size 2024-06-03T01:20:54.346 INFO:tasks.workunit.client.0.smithi032.stdout:9/101: dread - d0/d11/d1d/f1e zero size 2024-06-03T01:20:54.350 INFO:tasks.workunit.client.1.smithi044.stdout:4/424: link d17/d23/d35/f41 d17/d23/d58/f96 0 2024-06-03T01:20:54.350 INFO:tasks.workunit.client.1.smithi044.stdout:4/425: stat d17/d23/d59/c6e 0 2024-06-03T01:20:54.358 INFO:tasks.workunit.client.1.smithi044.stdout:0/559: creat d3/da6/d20/d25/fc3 x:0 0 0 2024-06-03T01:20:54.359 INFO:tasks.workunit.client.1.smithi044.stdout:0/560: chown d3/da6/d20/f78 10 1 2024-06-03T01:20:54.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/339: dwrite db/d15/f70 [0,4194304] 0 2024-06-03T01:20:54.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/340: chown db/f1f 8442 1 2024-06-03T01:20:54.363 INFO:tasks.workunit.client.1.smithi044.stdout:5/446: dread d2/d11/d12/f6c [0,4194304] 0 2024-06-03T01:20:54.363 INFO:tasks.workunit.client.0.smithi032.stdout:6/52: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/47: dread f8 [0,4194304] 0 2024-06-03T01:20:54.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/496: dwrite d0/d3/d20/d7/d4e/f78 [0,4194304] 0 2024-06-03T01:20:54.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/497: chown d0/d3/d14 5 1 2024-06-03T01:20:54.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/498: creat d0/d16/d64/d6a/d9e/fa9 x:0 0 0 2024-06-03T01:20:54.367 INFO:tasks.workunit.client.0.smithi032.stdout:6/53: link l7 d8/l10 0 2024-06-03T01:20:54.368 INFO:tasks.workunit.client.1.smithi044.stdout:1/499: mknod d0/d16/d64/caa 0 2024-06-03T01:20:54.368 INFO:tasks.workunit.client.1.smithi044.stdout:1/500: creat d0/d3/d20/d7/d13/fab x:0 0 0 2024-06-03T01:20:54.368 INFO:tasks.workunit.client.1.smithi044.stdout:5/447: getdents d2/d11/d1e/d2f 0 2024-06-03T01:20:54.371 INFO:tasks.workunit.client.1.smithi044.stdout:1/501: symlink d0/d3/d20/d40/d45/lac 0 2024-06-03T01:20:54.371 INFO:tasks.workunit.client.1.smithi044.stdout:1/502: write d0/d16/f93 [1033458,92187] 0 2024-06-03T01:20:54.372 INFO:tasks.workunit.client.1.smithi044.stdout:5/448: link d2/d11/c47 d2/dc/c8d 0 2024-06-03T01:20:54.373 INFO:tasks.workunit.client.1.smithi044.stdout:2/341: read db/dd/fe [3269869,65993] 0 2024-06-03T01:20:54.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/54: mkdir d8/d11 0 2024-06-03T01:20:54.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/55: fsync f3 0 2024-06-03T01:20:54.374 INFO:tasks.workunit.client.1.smithi044.stdout:1/503: mknod d0/d3/d20/d40/d45/d4b/d80/cad 0 2024-06-03T01:20:54.374 INFO:tasks.workunit.client.1.smithi044.stdout:7/512: dwrite d0/d3/d8/d32/d33/f67 [0,4194304] 0 2024-06-03T01:20:54.374 INFO:tasks.workunit.client.1.smithi044.stdout:7/513: read - d0/d3/d8/d95/d63/d89/da3/f5e zero size 2024-06-03T01:20:54.374 INFO:tasks.workunit.client.1.smithi044.stdout:7/514: write d0/f1d [1175473,76850] 0 2024-06-03T01:20:54.375 INFO:tasks.workunit.client.1.smithi044.stdout:4/426: dwrite d17/f94 [0,4194304] 0 2024-06-03T01:20:54.377 INFO:tasks.workunit.client.0.smithi032.stdout:7/48: truncate f7 1223031 0 2024-06-03T01:20:54.380 INFO:tasks.workunit.client.1.smithi044.stdout:1/504: mkdir d0/d3/d14/dae 0 2024-06-03T01:20:54.381 INFO:tasks.workunit.client.0.smithi032.stdout:5/15: sync 2024-06-03T01:20:54.381 INFO:tasks.workunit.client.0.smithi032.stdout:2/62: sync 2024-06-03T01:20:54.382 INFO:tasks.workunit.client.0.smithi032.stdout:8/19: sync 2024-06-03T01:20:54.382 INFO:tasks.workunit.client.0.smithi032.stdout:0/13: sync 2024-06-03T01:20:54.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/63: stat d5/l12 0 2024-06-03T01:20:54.382 INFO:tasks.workunit.client.0.smithi032.stdout:4/18: sync 2024-06-03T01:20:54.382 INFO:tasks.workunit.client.0.smithi032.stdout:3/49: sync 2024-06-03T01:20:54.383 INFO:tasks.workunit.client.1.smithi044.stdout:7/515: mkdir d0/d3/d8/d95/d63/d89/d8b/da9 0 2024-06-03T01:20:54.384 INFO:tasks.workunit.client.0.smithi032.stdout:5/16: mkdir d3 0 2024-06-03T01:20:54.384 INFO:tasks.workunit.client.0.smithi032.stdout:5/17: fsync f0 0 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:9/102: dwrite d0/f6 [0,4194304] 0 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:2/64: truncate d5/fd 1168663 0 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/14: mknod d1/c2 0 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/15: chown d1 0 1 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/16: stat d1 0 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/17: dread - no filename 2024-06-03T01:20:54.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/18: dread - no filename 2024-06-03T01:20:54.387 INFO:tasks.workunit.client.1.smithi044.stdout:2/342: read db/d1b/d27/f50 [3951813,103822] 0 2024-06-03T01:20:54.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/49: link f8 dc/dd/f11 0 2024-06-03T01:20:54.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/50: write f8 [247516,97768] 0 2024-06-03T01:20:54.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/51: chown dc/dd/f11 410 1 2024-06-03T01:20:54.388 INFO:tasks.workunit.client.0.smithi032.stdout:3/50: mknod de/cf 0 2024-06-03T01:20:54.388 INFO:tasks.workunit.client.1.smithi044.stdout:5/449: rmdir d2/d11/d1e/d21/d24 39 2024-06-03T01:20:54.389 INFO:tasks.workunit.client.1.smithi044.stdout:4/427: dread d17/d1c/f3a [0,4194304] 0 2024-06-03T01:20:54.390 INFO:tasks.workunit.client.1.smithi044.stdout:4/428: truncate d17/d23/d28/d2b/f93 145621 0 2024-06-03T01:20:54.390 INFO:tasks.workunit.client.1.smithi044.stdout:4/429: write d17/d23/d58/f85 [489998,119844] 0 2024-06-03T01:20:54.391 INFO:tasks.workunit.client.0.smithi032.stdout:5/18: rmdir d3 0 2024-06-03T01:20:54.392 INFO:tasks.workunit.client.0.smithi032.stdout:9/103: stat d0/d11/l17 0 2024-06-03T01:20:54.392 INFO:tasks.workunit.client.0.smithi032.stdout:9/104: stat d0/f15 0 2024-06-03T01:20:54.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/505: rename d0/d3/d14/f5e to d0/d3/d20/d40/d45/d4b/d5c/faf 0 2024-06-03T01:20:54.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/506: chown d0/d3/d20/d40/d45/l9b 4191035 1 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/19: symlink d1/l3 0 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/20: write - no filename 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/21: write - no filename 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/22: stat d1/c2 0 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/23: readlink d1/l3 0 2024-06-03T01:20:54.394 INFO:tasks.workunit.client.0.smithi032.stdout:0/24: write - no filename 2024-06-03T01:20:54.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/51: creat de/f10 x:0 0 0 2024-06-03T01:20:54.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/52: creat de/f11 x:0 0 0 2024-06-03T01:20:54.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/53: dread - de/f10 zero size 2024-06-03T01:20:54.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/19: creat f4 x:0 0 0 2024-06-03T01:20:54.398 INFO:tasks.workunit.client.0.smithi032.stdout:9/105: truncate d0/fd 627096 0 2024-06-03T01:20:54.399 INFO:tasks.workunit.client.0.smithi032.stdout:0/25: mknod d1/c4 0 2024-06-03T01:20:54.399 INFO:tasks.workunit.client.0.smithi032.stdout:0/26: write - no filename 2024-06-03T01:20:54.399 INFO:tasks.workunit.client.1.smithi044.stdout:7/516: rename d0/d3/d8/f58 to d0/d3/d8/d95/d63/d7f/d9c/d6e/faa 0 2024-06-03T01:20:54.402 INFO:tasks.workunit.client.1.smithi044.stdout:1/507: creat d0/d3/d20/d40/d45/d4b/d5a/fb0 x:0 0 0 2024-06-03T01:20:54.402 INFO:tasks.workunit.client.0.smithi032.stdout:9/106: truncate d0/f15 3831082 0 2024-06-03T01:20:54.402 INFO:tasks.workunit.client.0.smithi032.stdout:9/107: write d0/fd [651442,123668] 0 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.1.smithi044.stdout:2/343: dread db/d1b/d27/f39 [0,4194304] 0 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.0.smithi032.stdout:0/27: link d1/c2 d1/c5 0 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.0.smithi032.stdout:0/28: dwrite - no filename 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.0.smithi032.stdout:6/56: dwrite d8/fb [0,4194304] 0 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.0.smithi032.stdout:6/57: chown c5 16 1 2024-06-03T01:20:54.404 INFO:tasks.workunit.client.0.smithi032.stdout:6/58: chown d8/l10 63387401 1 2024-06-03T01:20:54.405 INFO:tasks.workunit.client.1.smithi044.stdout:5/450: rmdir d2/d11/d12/d2b 39 2024-06-03T01:20:54.406 INFO:tasks.workunit.client.0.smithi032.stdout:9/108: symlink d0/d11/d1d/l20 0 2024-06-03T01:20:54.408 INFO:tasks.workunit.client.1.smithi044.stdout:4/430: rmdir d17/d23/d28 39 2024-06-03T01:20:54.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/431: creat d17/d23/d58/f97 x:0 0 0 2024-06-03T01:20:54.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/432: write d17/d23/d58/f85 [801398,18475] 0 2024-06-03T01:20:54.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/433: chown d17/d23/d28/f33 7221731 1 2024-06-03T01:20:54.411 INFO:tasks.workunit.client.1.smithi044.stdout:1/508: creat d0/d3/d20/d7/fb1 x:0 0 0 2024-06-03T01:20:54.411 INFO:tasks.workunit.client.1.smithi044.stdout:5/451: read d2/d11/d12/f6c [2461502,17954] 0 2024-06-03T01:20:54.413 INFO:tasks.workunit.client.0.smithi032.stdout:6/59: creat d8/d11/f12 x:0 0 0 2024-06-03T01:20:54.414 INFO:tasks.workunit.client.1.smithi044.stdout:2/344: unlink db/f4f 0 2024-06-03T01:20:54.414 INFO:tasks.workunit.client.1.smithi044.stdout:2/345: write db/dd/f60 [1006405,46486] 0 2024-06-03T01:20:54.416 INFO:tasks.workunit.client.0.smithi032.stdout:9/109: symlink d0/d11/d1d/l21 0 2024-06-03T01:20:54.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/509: symlink d0/d16/d64/d6a/d9e/lb2 0 2024-06-03T01:20:54.420 INFO:tasks.workunit.client.1.smithi044.stdout:1/510: fsync d0/d3/d20/d7/d13/f84 0 2024-06-03T01:20:54.424 INFO:tasks.workunit.client.1.smithi044.stdout:5/452: unlink d2/d11/d12/d2b/d44/c54 0 2024-06-03T01:20:54.424 INFO:tasks.workunit.client.1.smithi044.stdout:4/434: write d17/d23/d35/f4f [7528014,120959] 0 2024-06-03T01:20:54.426 INFO:tasks.workunit.client.0.smithi032.stdout:8/20: dwrite f2 [0,4194304] 0 2024-06-03T01:20:54.426 INFO:tasks.workunit.client.1.smithi044.stdout:2/346: creat db/f73 x:0 0 0 2024-06-03T01:20:54.426 INFO:tasks.workunit.client.1.smithi044.stdout:2/347: fsync f2 0 2024-06-03T01:20:54.427 INFO:tasks.workunit.client.1.smithi044.stdout:1/511: getdents d0/d3/d20/d40/d45/d6b 0 2024-06-03T01:20:54.428 INFO:tasks.workunit.client.1.smithi044.stdout:1/512: write d0/d3/d20/d7/d13/fa2 [1010497,23485] 0 2024-06-03T01:20:54.428 INFO:tasks.workunit.client.1.smithi044.stdout:1/513: write d0/d16/d64/d6a/d9e/fa6 [777878,120835] 0 2024-06-03T01:20:54.428 INFO:tasks.workunit.client.1.smithi044.stdout:4/435: dread d17/d23/f57 [0,4194304] 0 2024-06-03T01:20:54.429 INFO:tasks.workunit.client.0.smithi032.stdout:8/21: getdents . 0 2024-06-03T01:20:54.429 INFO:tasks.workunit.client.1.smithi044.stdout:5/453: creat d2/dc/d28/d66/f8e x:0 0 0 2024-06-03T01:20:54.432 INFO:tasks.workunit.client.1.smithi044.stdout:2/348: truncate db/d1b/d5d/f62 7056786 0 2024-06-03T01:20:54.434 INFO:tasks.workunit.client.1.smithi044.stdout:4/436: mkdir d17/d23/d58/d98 0 2024-06-03T01:20:54.436 INFO:tasks.workunit.client.1.smithi044.stdout:2/349: mknod db/d1b/d2a/c74 0 2024-06-03T01:20:54.437 INFO:tasks.workunit.client.1.smithi044.stdout:1/514: rmdir d0/d3/d20/d40/d45/d6f 39 2024-06-03T01:20:54.441 INFO:tasks.workunit.client.1.smithi044.stdout:1/515: creat d0/d3/d20/d40/d45/d4b/d5c/d94/fb3 x:0 0 0 2024-06-03T01:20:54.441 INFO:tasks.workunit.client.1.smithi044.stdout:1/516: read d0/d3/f19 [1199828,81090] 0 2024-06-03T01:20:54.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/437: getdents d17/d23/d59 0 2024-06-03T01:20:54.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/438: chown d17/d23/d28/f33 10201471 1 2024-06-03T01:20:54.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/439: read d17/d23/f20 [185206,13502] 0 2024-06-03T01:20:54.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/440: chown d17/d1c/f3a 23459151 1 2024-06-03T01:20:54.444 INFO:tasks.workunit.client.1.smithi044.stdout:4/441: read d17/d23/d35/f3c [402611,106106] 0 2024-06-03T01:20:54.444 INFO:tasks.workunit.client.1.smithi044.stdout:4/442: readlink d17/d1c/l4e 0 2024-06-03T01:20:54.445 INFO:tasks.workunit.client.1.smithi044.stdout:1/517: write d0/d3/d20/f6d [1115097,71805] 0 2024-06-03T01:20:54.448 INFO:tasks.workunit.client.1.smithi044.stdout:1/518: rename d0/d3/d20/d40/d45/d4b/d5c/f8a to d0/d3/d20/d40/d45/d4b/d80/fb4 0 2024-06-03T01:20:54.449 INFO:tasks.workunit.client.1.smithi044.stdout:1/519: symlink d0/d3/d20/d40/d45/d6b/lb5 0 2024-06-03T01:20:54.449 INFO:tasks.workunit.client.1.smithi044.stdout:1/520: stat d0/d3/d20/d7/f17 0 2024-06-03T01:20:54.456 INFO:tasks.workunit.client.0.smithi032.stdout:4/19: dwrite f0 [0,4194304] 0 2024-06-03T01:20:54.457 INFO:tasks.workunit.client.0.smithi032.stdout:2/65: dwrite f3 [4194304,4194304] 0 2024-06-03T01:20:54.461 INFO:tasks.workunit.client.0.smithi032.stdout:0/29: getdents d1 0 2024-06-03T01:20:54.461 INFO:tasks.workunit.client.0.smithi032.stdout:0/30: fsync - no filename 2024-06-03T01:20:54.462 INFO:tasks.workunit.client.0.smithi032.stdout:0/31: creat d1/f6 x:0 0 0 2024-06-03T01:20:54.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/443: write d17/d23/d28/d2b/f21 [2788654,125893] 0 2024-06-03T01:20:54.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/444: creat d17/d23/f99 x:0 0 0 2024-06-03T01:20:54.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/445: chown d17/d1c/d75/f48 6 1 2024-06-03T01:20:54.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/446: readlink d17/d23/d28/l8c 0 2024-06-03T01:20:54.464 INFO:tasks.workunit.client.0.smithi032.stdout:0/32: symlink d1/l7 0 2024-06-03T01:20:54.465 INFO:tasks.workunit.client.0.smithi032.stdout:0/33: dread - d1/f6 zero size 2024-06-03T01:20:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:54 smithi044 ceph-mon[36956]: pgmap v256: 65 pgs: 65 active+clean; 705 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 53 MiB/s rd, 74 MiB/s wr, 888 op/s 2024-06-03T01:20:54.472 INFO:tasks.workunit.client.0.smithi032.stdout:2/66: write f3 [4471627,5406] 0 2024-06-03T01:20:54.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/20: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.480 INFO:tasks.workunit.client.0.smithi032.stdout:3/54: dwrite fd [0,4194304] 0 2024-06-03T01:20:54.481 INFO:tasks.workunit.client.0.smithi032.stdout:9/110: dwrite d0/fe [4194304,4194304] 0 2024-06-03T01:20:54.481 INFO:tasks.workunit.client.0.smithi032.stdout:9/111: fsync d0/fe 0 2024-06-03T01:20:54.481 INFO:tasks.workunit.client.1.smithi044.stdout:7/517: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [0,4194304] 0 2024-06-03T01:20:54.481 INFO:tasks.workunit.client.1.smithi044.stdout:7/518: chown d0/d3/d8/d95/d63/d7f/d9c/c31 329807 1 2024-06-03T01:20:54.485 INFO:tasks.workunit.client.1.smithi044.stdout:2/350: dwrite db/f26 [4194304,4194304] 0 2024-06-03T01:20:54.486 INFO:tasks.workunit.client.1.smithi044.stdout:5/454: dwrite d2/d11/d12/d2b/f72 [0,4194304] 0 2024-06-03T01:20:54.486 INFO:tasks.workunit.client.1.smithi044.stdout:5/455: dread - d2/d11/d1e/d2f/f64 zero size 2024-06-03T01:20:54.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/60: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.486 INFO:tasks.workunit.client.0.smithi032.stdout:5/21: symlink l5 0 2024-06-03T01:20:54.486 INFO:tasks.workunit.client.0.smithi032.stdout:4/20: dwrite f0 [4194304,4194304] 0 2024-06-03T01:20:54.487 INFO:tasks.workunit.client.1.smithi044.stdout:5/456: write d2/d11/d12/f25 [1361826,16948] 0 2024-06-03T01:20:54.487 INFO:tasks.workunit.client.1.smithi044.stdout:4/447: dwrite ff [0,4194304] 0 2024-06-03T01:20:54.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/521: dread d0/d3/d20/d7/d13/fa2 [0,4194304] 0 2024-06-03T01:20:54.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/522: dread - d0/d3/d20/f9c zero size 2024-06-03T01:20:54.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/523: chown d0 77 1 2024-06-03T01:20:54.490 INFO:tasks.workunit.client.0.smithi032.stdout:5/22: mknod c6 0 2024-06-03T01:20:54.496 INFO:tasks.workunit.client.1.smithi044.stdout:2/351: rename db/d15/l2e to db/d1b/d2a/d52/d54/l75 0 2024-06-03T01:20:54.496 INFO:tasks.workunit.client.1.smithi044.stdout:2/352: readlink db/d1b/d5d/l65 0 2024-06-03T01:20:54.496 INFO:tasks.workunit.client.1.smithi044.stdout:2/353: fdatasync db/d1b/d27/f32 0 2024-06-03T01:20:54.496 INFO:tasks.workunit.client.0.smithi032.stdout:0/34: fsync d1/f6 0 2024-06-03T01:20:54.497 INFO:tasks.workunit.client.1.smithi044.stdout:7/519: dread d0/d3/d8/f37 [0,4194304] 0 2024-06-03T01:20:54.497 INFO:tasks.workunit.client.1.smithi044.stdout:7/520: fdatasync d0/d3/d8/d95/d63/d7f/d9c/f83 0 2024-06-03T01:20:54.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/448: creat d17/d23/d6a/f9a x:0 0 0 2024-06-03T01:20:54.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/449: creat d17/d23/f9b x:0 0 0 2024-06-03T01:20:54.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/450: dread - d17/d23/d59/f82 zero size 2024-06-03T01:20:54.500 INFO:tasks.workunit.client.0.smithi032.stdout:4/21: mknod d4/c8 0 2024-06-03T01:20:54.502 INFO:tasks.workunit.client.0.smithi032.stdout:5/23: unlink f0 0 2024-06-03T01:20:54.503 INFO:tasks.workunit.client.1.smithi044.stdout:1/524: write d0/d3/d20/d7/f36 [2666751,112648] 0 2024-06-03T01:20:54.503 INFO:tasks.workunit.client.1.smithi044.stdout:1/525: readlink d0/d3/d20/d40/d45/d4b/d5a/l97 0 2024-06-03T01:20:54.505 INFO:tasks.workunit.client.0.smithi032.stdout:0/35: creat d1/f8 x:0 0 0 2024-06-03T01:20:54.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/22: mkdir d4/d9 0 2024-06-03T01:20:54.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/23: dread - d4/f7 zero size 2024-06-03T01:20:54.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/24: truncate d4/f6 397235 0 2024-06-03T01:20:54.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/25: fsync d4/f7 0 2024-06-03T01:20:54.510 INFO:tasks.workunit.client.0.smithi032.stdout:0/36: creat d1/f9 x:0 0 0 2024-06-03T01:20:54.510 INFO:tasks.workunit.client.0.smithi032.stdout:4/26: mkdir d4/d9/da 0 2024-06-03T01:20:54.511 INFO:tasks.workunit.client.1.smithi044.stdout:2/354: mknod db/d45/d4e/c76 0 2024-06-03T01:20:54.511 INFO:tasks.workunit.client.1.smithi044.stdout:4/451: dread d17/d23/d58/f96 [0,4194304] 0 2024-06-03T01:20:54.511 INFO:tasks.workunit.client.0.smithi032.stdout:4/27: dread d4/f6 [0,4194304] 0 2024-06-03T01:20:54.513 INFO:tasks.workunit.client.0.smithi032.stdout:0/37: unlink d1/c4 0 2024-06-03T01:20:54.513 INFO:tasks.workunit.client.1.smithi044.stdout:7/521: rename d0/d3/d8/d32/fa2 to d0/d3/d8/d95/d63/d89/d8b/fab 0 2024-06-03T01:20:54.515 INFO:tasks.workunit.client.1.smithi044.stdout:2/355: creat db/d1b/d2a/d2f/d4c/f77 x:0 0 0 2024-06-03T01:20:54.515 INFO:tasks.workunit.client.1.smithi044.stdout:2/356: chown db/dd/l16 59942 1 2024-06-03T01:20:54.517 INFO:tasks.workunit.client.1.smithi044.stdout:1/526: rename d0/d3/d20/d7/d4e/f8e to d0/d3/d20/d7/d13/fb6 0 2024-06-03T01:20:54.517 INFO:tasks.workunit.client.1.smithi044.stdout:7/522: dread d0/d16/f54 [0,4194304] 0 2024-06-03T01:20:54.517 INFO:tasks.workunit.client.1.smithi044.stdout:2/357: symlink db/d1b/d5d/l78 0 2024-06-03T01:20:54.518 INFO:tasks.workunit.client.1.smithi044.stdout:1/527: mkdir d0/d3/d20/d7/d2f/db7 0 2024-06-03T01:20:54.518 INFO:tasks.workunit.client.1.smithi044.stdout:1/528: fdatasync d0/d3/d20/d40/d45/d4b/d5c/f9a 0 2024-06-03T01:20:54.520 INFO:tasks.workunit.client.1.smithi044.stdout:2/358: unlink db/d1b/d2a/f33 0 2024-06-03T01:20:54.520 INFO:tasks.workunit.client.1.smithi044.stdout:2/359: dread - db/d1b/d2a/d52/d54/f56 zero size 2024-06-03T01:20:54.520 INFO:tasks.workunit.client.1.smithi044.stdout:1/529: symlink d0/d3/d20/d40/d45/d6f/lb8 0 2024-06-03T01:20:54.521 INFO:tasks.workunit.client.1.smithi044.stdout:7/523: symlink d0/d3/d8/d95/d63/d7f/d9c/lac 0 2024-06-03T01:20:54.522 INFO:tasks.workunit.client.1.smithi044.stdout:1/530: symlink d0/d3/d20/d40/d45/d4b/d80/lb9 0 2024-06-03T01:20:54.522 INFO:tasks.workunit.client.1.smithi044.stdout:1/531: write d0/d3/d20/d40/d45/d6b/f83 [706021,93439] 0 2024-06-03T01:20:54.523 INFO:tasks.workunit.client.1.smithi044.stdout:1/532: creat d0/fba x:0 0 0 2024-06-03T01:20:54.523 INFO:tasks.workunit.client.1.smithi044.stdout:1/533: fdatasync d0/d3/d20/d7/d13/f24 0 2024-06-03T01:20:54.524 INFO:tasks.workunit.client.1.smithi044.stdout:2/360: dread db/d1b/d27/f39 [0,4194304] 0 2024-06-03T01:20:54.525 INFO:tasks.workunit.client.1.smithi044.stdout:2/361: mkdir db/d1b/d27/d79 0 2024-06-03T01:20:54.525 INFO:tasks.workunit.client.1.smithi044.stdout:2/362: chown db/d15/c18 90439 1 2024-06-03T01:20:54.525 INFO:tasks.workunit.client.1.smithi044.stdout:2/363: write db/d1b/d2a/d2f/d4c/f77 [531818,31118] 0 2024-06-03T01:20:54.525 INFO:tasks.workunit.client.1.smithi044.stdout:2/364: write db/d1b/d27/f32 [1429931,13225] 0 2024-06-03T01:20:54.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/365: write db/d1b/d27/f32 [514358,70695] 0 2024-06-03T01:20:54.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/366: dread - db/d1b/d2a/d52/d54/f56 zero size 2024-06-03T01:20:54.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/367: chown db/dd/c29 11397 1 2024-06-03T01:20:54.538 INFO:tasks.workunit.client.0.smithi032.stdout:5/24: dread f4 [0,4194304] 0 2024-06-03T01:20:54.538 INFO:tasks.workunit.client.0.smithi032.stdout:9/112: dwrite d0/f6 [4194304,4194304] 0 2024-06-03T01:20:54.538 INFO:tasks.workunit.client.1.smithi044.stdout:4/452: dread d17/d23/f57 [0,4194304] 0 2024-06-03T01:20:54.538 INFO:tasks.workunit.client.1.smithi044.stdout:4/453: write d17/d23/d28/d2b/f4a [428711,96090] 0 2024-06-03T01:20:54.539 INFO:tasks.workunit.client.0.smithi032.stdout:9/113: link d0/fe d0/d11/f22 0 2024-06-03T01:20:54.539 INFO:tasks.workunit.client.0.smithi032.stdout:9/114: mkdir d0/d23 0 2024-06-03T01:20:54.540 INFO:tasks.workunit.client.0.smithi032.stdout:9/115: chown d0/c12 0 1 2024-06-03T01:20:54.541 INFO:tasks.workunit.client.0.smithi032.stdout:9/116: mkdir d0/d23/d24 0 2024-06-03T01:20:54.542 INFO:tasks.workunit.client.0.smithi032.stdout:9/117: dread - d0/d11/d1d/f1e zero size 2024-06-03T01:20:54.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/454: dread d17/d23/f4d [0,4194304] 0 2024-06-03T01:20:54.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/455: write d17/d1c/d75/f76 [275783,43036] 0 2024-06-03T01:20:54.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/456: chown d17/d1c/d70/d7d 365244849 1 2024-06-03T01:20:54.548 INFO:tasks.workunit.client.0.smithi032.stdout:3/55: dwrite de/f11 [0,4194304] 0 2024-06-03T01:20:54.549 INFO:tasks.workunit.client.1.smithi044.stdout:4/457: rmdir d17/d23/d58/d98 0 2024-06-03T01:20:54.551 INFO:tasks.workunit.client.1.smithi044.stdout:5/457: dwrite d2/d11/d1e/d21/f75 [0,4194304] 0 2024-06-03T01:20:54.551 INFO:tasks.workunit.client.1.smithi044.stdout:5/458: readlink d2/d11/d12/l1f 0 2024-06-03T01:20:54.551 INFO:tasks.workunit.client.0.smithi032.stdout:8/22: dwrite f2 [4194304,4194304] 0 2024-06-03T01:20:54.552 INFO:tasks.workunit.client.1.smithi044.stdout:4/458: unlink d17/d23/d28/d2b/f8f 0 2024-06-03T01:20:54.552 INFO:tasks.workunit.client.0.smithi032.stdout:2/67: dwrite d5/da/f10 [0,4194304] 0 2024-06-03T01:20:54.554 INFO:tasks.workunit.client.0.smithi032.stdout:8/23: creat f8 x:0 0 0 2024-06-03T01:20:54.554 INFO:tasks.workunit.client.0.smithi032.stdout:8/24: creat f9 x:0 0 0 2024-06-03T01:20:54.557 INFO:tasks.workunit.client.0.smithi032.stdout:6/61: dwrite d8/f9 [0,4194304] 0 2024-06-03T01:20:54.559 INFO:tasks.workunit.client.0.smithi032.stdout:3/56: read de/f11 [1290412,111454] 0 2024-06-03T01:20:54.559 INFO:tasks.workunit.client.0.smithi032.stdout:3/57: readlink l6 0 2024-06-03T01:20:54.563 INFO:tasks.workunit.client.0.smithi032.stdout:6/62: symlink d8/d11/l13 0 2024-06-03T01:20:54.564 INFO:tasks.workunit.client.0.smithi032.stdout:2/68: getdents d5 0 2024-06-03T01:20:54.565 INFO:tasks.workunit.client.1.smithi044.stdout:7/524: dwrite d0/d3/d8/d95/d63/d89/da3/f9e [0,4194304] 0 2024-06-03T01:20:54.565 INFO:tasks.workunit.client.1.smithi044.stdout:4/459: write d17/d23/f4d [6855363,559] 0 2024-06-03T01:20:54.565 INFO:tasks.workunit.client.1.smithi044.stdout:1/534: dwrite d0/d16/d64/d6a/f75 [0,4194304] 0 2024-06-03T01:20:54.566 INFO:tasks.workunit.client.1.smithi044.stdout:5/459: mknod d2/d11/d3e/c8f 0 2024-06-03T01:20:54.567 INFO:tasks.workunit.client.0.smithi032.stdout:3/58: unlink fc 0 2024-06-03T01:20:54.567 INFO:tasks.workunit.client.0.smithi032.stdout:3/59: chown de 5882 1 2024-06-03T01:20:54.570 INFO:tasks.workunit.client.0.smithi032.stdout:2/69: creat d5/da/f18 x:0 0 0 2024-06-03T01:20:54.570 INFO:tasks.workunit.client.1.smithi044.stdout:2/368: dread db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:20:54.573 INFO:tasks.workunit.client.1.smithi044.stdout:1/535: creat d0/d3/d20/d40/d45/d6f/fbb x:0 0 0 2024-06-03T01:20:54.573 INFO:tasks.workunit.client.0.smithi032.stdout:3/60: mknod de/c12 0 2024-06-03T01:20:54.575 INFO:tasks.workunit.client.0.smithi032.stdout:2/70: rename d5/f17 to d5/f19 0 2024-06-03T01:20:54.576 INFO:tasks.workunit.client.0.smithi032.stdout:3/61: mkdir de/d13 0 2024-06-03T01:20:54.576 INFO:tasks.workunit.client.0.smithi032.stdout:9/118: dwrite d0/f1b [0,4194304] 0 2024-06-03T01:20:54.576 INFO:tasks.workunit.client.0.smithi032.stdout:9/119: fsync d0/fd 0 2024-06-03T01:20:54.577 INFO:tasks.workunit.client.1.smithi044.stdout:7/525: write d0/fd [2756565,92001] 0 2024-06-03T01:20:54.577 INFO:tasks.workunit.client.1.smithi044.stdout:1/536: rmdir d0/d16/d70 0 2024-06-03T01:20:54.577 INFO:tasks.workunit.client.1.smithi044.stdout:1/537: fsync d0/d3/d14/f50 0 2024-06-03T01:20:54.578 INFO:tasks.workunit.client.0.smithi032.stdout:5/25: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.578 INFO:tasks.workunit.client.0.smithi032.stdout:2/71: mkdir d5/da/d1a 0 2024-06-03T01:20:54.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/72: chown d5/da/d1a 148 1 2024-06-03T01:20:54.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/73: write d5/f19 [683632,93573] 0 2024-06-03T01:20:54.579 INFO:tasks.workunit.client.0.smithi032.stdout:3/62: rmdir de/d13 0 2024-06-03T01:20:54.582 INFO:tasks.workunit.client.1.smithi044.stdout:4/460: dread d17/d23/f68 [0,4194304] 0 2024-06-03T01:20:54.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/369: rename db/d45 to db/d15/d49/d6c/d72/d7a 0 2024-06-03T01:20:54.583 INFO:tasks.workunit.client.1.smithi044.stdout:7/526: rename d0 to d0/d3/d8/d95/d63/dad 22 2024-06-03T01:20:54.583 INFO:tasks.workunit.client.1.smithi044.stdout:7/527: readlink d0/d16/l77 0 2024-06-03T01:20:54.583 INFO:tasks.workunit.client.0.smithi032.stdout:3/63: rename de/cf to de/c14 0 2024-06-03T01:20:54.585 INFO:tasks.workunit.client.0.smithi032.stdout:3/64: truncate fd 1274170 0 2024-06-03T01:20:54.585 INFO:tasks.workunit.client.1.smithi044.stdout:4/461: mkdir d17/d1c/d9c 0 2024-06-03T01:20:54.586 INFO:tasks.workunit.client.1.smithi044.stdout:4/462: write d17/d63/f65 [859993,9214] 0 2024-06-03T01:20:54.586 INFO:tasks.workunit.client.1.smithi044.stdout:4/463: stat d17/d23/f4d 0 2024-06-03T01:20:54.586 INFO:tasks.workunit.client.1.smithi044.stdout:4/464: readlink d17/d1c/d70/d7d/l83 0 2024-06-03T01:20:54.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/370: mkdir db/d15/d49/d7b 0 2024-06-03T01:20:54.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/371: truncate db/d1b/d27/f39 5178978 0 2024-06-03T01:20:54.588 INFO:tasks.workunit.client.1.smithi044.stdout:7/528: symlink d0/d3/d8/d95/d63/d89/lae 0 2024-06-03T01:20:54.588 INFO:tasks.workunit.client.1.smithi044.stdout:7/529: write d0/d3/d8/d95/d63/d7f/d9c/f50 [1598218,98712] 0 2024-06-03T01:20:54.589 INFO:tasks.workunit.client.0.smithi032.stdout:8/25: dwrite f8 [0,4194304] 0 2024-06-03T01:20:54.589 INFO:tasks.workunit.client.0.smithi032.stdout:4/28: rename d4/d9 to d4/db 0 2024-06-03T01:20:54.589 INFO:tasks.workunit.client.0.smithi032.stdout:4/29: fdatasync f0 0 2024-06-03T01:20:54.589 INFO:tasks.workunit.client.0.smithi032.stdout:4/30: chown d4/f6 1 1 2024-06-03T01:20:54.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/372: creat db/d1b/d2a/d67/f7c x:0 0 0 2024-06-03T01:20:54.591 INFO:tasks.workunit.client.0.smithi032.stdout:6/63: dwrite f3 [0,4194304] 0 2024-06-03T01:20:54.592 INFO:tasks.workunit.client.0.smithi032.stdout:0/38: rmdir d1 39 2024-06-03T01:20:54.593 INFO:tasks.workunit.client.1.smithi044.stdout:7/530: rename d0/d3/d8/d95/f28 to d0/d3/d8/d95/d63/d7f/faf 0 2024-06-03T01:20:54.595 INFO:tasks.workunit.client.1.smithi044.stdout:7/531: rmdir d0/d3/d8/d95/d63/d89/d8b 39 2024-06-03T01:20:54.596 INFO:tasks.workunit.client.0.smithi032.stdout:5/26: write f4 [2306604,14641] 0 2024-06-03T01:20:54.598 INFO:tasks.workunit.client.0.smithi032.stdout:4/31: rename f0 to d4/db/fc 0 2024-06-03T01:20:54.607 INFO:tasks.workunit.client.1.smithi044.stdout:5/460: rmdir d2/d11/d3e 39 2024-06-03T01:20:54.608 INFO:tasks.workunit.client.1.smithi044.stdout:7/532: read d0/d16/f4d [2516621,34462] 0 2024-06-03T01:20:54.608 INFO:tasks.workunit.client.1.smithi044.stdout:7/533: write d0/f74 [4693309,122461] 0 2024-06-03T01:20:54.609 INFO:tasks.workunit.client.1.smithi044.stdout:5/461: symlink d2/d11/d12/d2b/d4a/d57/l90 0 2024-06-03T01:20:54.609 INFO:tasks.workunit.client.1.smithi044.stdout:5/462: chown d2/dc/d28/f2c 3 1 2024-06-03T01:20:54.610 INFO:tasks.workunit.client.1.smithi044.stdout:7/534: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/lb0 0 2024-06-03T01:20:54.611 INFO:tasks.workunit.client.1.smithi044.stdout:5/463: mknod d2/d11/c91 0 2024-06-03T01:20:54.613 INFO:tasks.workunit.client.1.smithi044.stdout:7/535: creat d0/d3/d8/d32/d33/fb1 x:0 0 0 2024-06-03T01:20:54.613 INFO:tasks.workunit.client.1.smithi044.stdout:7/536: rmdir d0/d3/d8 39 2024-06-03T01:20:54.615 INFO:tasks.workunit.client.1.smithi044.stdout:7/537: mknod d0/d3/d8/d32/cb2 0 2024-06-03T01:20:54.616 INFO:tasks.workunit.client.1.smithi044.stdout:7/538: truncate d0/d3/d8/d95/d63/d7f/faf 57070 0 2024-06-03T01:20:54.617 INFO:tasks.workunit.client.1.smithi044.stdout:7/539: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/lb3 0 2024-06-03T01:20:54.621 INFO:tasks.workunit.client.1.smithi044.stdout:7/540: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 x:0 0 0 2024-06-03T01:20:54.621 INFO:tasks.workunit.client.1.smithi044.stdout:7/541: write d0/d3/d8/d95/d63/d7f/f91 [5045938,73923] 0 2024-06-03T01:20:54.621 INFO:tasks.workunit.client.1.smithi044.stdout:7/542: write d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 [853084,35702] 0 2024-06-03T01:20:54.621 INFO:tasks.workunit.client.1.smithi044.stdout:7/543: fsync d0/d3/f47 0 2024-06-03T01:20:54.622 INFO:tasks.workunit.client.0.smithi032.stdout:6/64: dread f4 [0,4194304] 0 2024-06-03T01:20:54.622 INFO:tasks.workunit.client.0.smithi032.stdout:6/65: read - d8/d11/f12 zero size 2024-06-03T01:20:54.626 INFO:tasks.workunit.client.1.smithi044.stdout:5/464: dwrite d2/d11/d12/f67 [0,4194304] 0 2024-06-03T01:20:54.627 INFO:tasks.workunit.client.1.smithi044.stdout:5/465: truncate d2/d11/d12/d2b/f84 87047 0 2024-06-03T01:20:54.627 INFO:tasks.workunit.client.1.smithi044.stdout:5/466: fsync d2/dc/d28/d4c/f88 0 2024-06-03T01:20:54.629 INFO:tasks.workunit.client.0.smithi032.stdout:4/32: dread d4/db/fc [0,4194304] 0 2024-06-03T01:20:54.631 INFO:tasks.workunit.client.1.smithi044.stdout:5/467: mknod d2/d11/d1e/d21/c92 0 2024-06-03T01:20:54.631 INFO:tasks.workunit.client.1.smithi044.stdout:5/468: read d2/d11/d3e/f40 [74670,126651] 0 2024-06-03T01:20:54.631 INFO:tasks.workunit.client.1.smithi044.stdout:5/469: write d2/d11/d12/d2b/f72 [4502041,95107] 0 2024-06-03T01:20:54.631 INFO:tasks.workunit.client.1.smithi044.stdout:5/470: getdents d2/d11/d1e/d80 0 2024-06-03T01:20:54.632 INFO:tasks.workunit.client.1.smithi044.stdout:5/471: mkdir d2/d11/d1e/d21/d24/d93 0 2024-06-03T01:20:54.634 INFO:tasks.workunit.client.1.smithi044.stdout:5/472: link d2/d11/d12/l1f d2/d11/d3e/l94 0 2024-06-03T01:20:54.637 INFO:tasks.workunit.client.0.smithi032.stdout:6/66: dread d8/ff [0,4194304] 0 2024-06-03T01:20:54.637 INFO:tasks.workunit.client.0.smithi032.stdout:6/67: readlink d8/d11/l13 0 2024-06-03T01:20:54.637 INFO:tasks.workunit.client.0.smithi032.stdout:6/68: write d8/d11/f12 [596693,110516] 0 2024-06-03T01:20:54.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/69: rename c1 to d8/d11/c14 0 2024-06-03T01:20:54.640 INFO:tasks.workunit.client.1.smithi044.stdout:2/373: dwrite db/d1b/d2a/d67/f7c [0,4194304] 0 2024-06-03T01:20:54.640 INFO:tasks.workunit.client.0.smithi032.stdout:3/65: dwrite f8 [0,4194304] 0 2024-06-03T01:20:54.643 INFO:tasks.workunit.client.0.smithi032.stdout:6/70: dread d8/ff [0,4194304] 0 2024-06-03T01:20:54.643 INFO:tasks.workunit.client.0.smithi032.stdout:6/71: fsync d8/fd 0 2024-06-03T01:20:54.643 INFO:tasks.workunit.client.0.smithi032.stdout:6/72: fsync d8/f9 0 2024-06-03T01:20:54.644 INFO:tasks.workunit.client.1.smithi044.stdout:5/473: write d2/d11/d12/d2b/d44/f5f [5529275,104232] 0 2024-06-03T01:20:54.645 INFO:tasks.workunit.client.1.smithi044.stdout:5/474: creat d2/d11/d1e/f95 x:0 0 0 2024-06-03T01:20:54.646 INFO:tasks.workunit.client.1.smithi044.stdout:1/538: dwrite d0/d3/d20/d7/d13/f24 [0,4194304] 0 2024-06-03T01:20:54.648 INFO:tasks.workunit.client.1.smithi044.stdout:7/544: dwrite d0/d3/d8/d95/d63/f6b [0,4194304] 0 2024-06-03T01:20:54.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/465: dwrite d17/d1c/d75/f5d [4194304,4194304] 0 2024-06-03T01:20:54.653 INFO:tasks.workunit.client.0.smithi032.stdout:8/26: truncate f8 1720941 0 2024-06-03T01:20:54.653 INFO:tasks.workunit.client.0.smithi032.stdout:8/27: readlink l4 0 2024-06-03T01:20:54.654 INFO:tasks.workunit.client.1.smithi044.stdout:2/374: dread db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:20:54.654 INFO:tasks.workunit.client.0.smithi032.stdout:4/33: getdents d4/db 0 2024-06-03T01:20:54.655 INFO:tasks.workunit.client.1.smithi044.stdout:4/466: truncate d17/f94 4116456 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.0.smithi032.stdout:6/73: write d8/ff [1138222,129367] 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.0.smithi032.stdout:4/34: creat d4/db/da/fd x:0 0 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.0.smithi032.stdout:9/120: dwrite d0/d11/f22 [0,4194304] 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.0.smithi032.stdout:9/121: fdatasync d0/d11/f14 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/74: dwrite d5/ff [0,4194304] 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.1.smithi044.stdout:2/375: rename db/dd/c29 to db/d15/d49/d6c/d72/d7a/d4e/c7d 0 2024-06-03T01:20:54.658 INFO:tasks.workunit.client.1.smithi044.stdout:2/376: write db/d1b/d3f/f61 [241947,51153] 0 2024-06-03T01:20:54.659 INFO:tasks.workunit.client.0.smithi032.stdout:4/35: write d4/f6 [56197,81751] 0 2024-06-03T01:20:54.659 INFO:tasks.workunit.client.0.smithi032.stdout:4/36: chown d4/db/fc 1 1 2024-06-03T01:20:54.659 INFO:tasks.workunit.client.0.smithi032.stdout:8/28: getdents . 0 2024-06-03T01:20:54.661 INFO:tasks.workunit.client.1.smithi044.stdout:4/467: creat d17/d23/d28/f9d x:0 0 0 2024-06-03T01:20:54.663 INFO:tasks.workunit.client.0.smithi032.stdout:5/27: dread f4 [0,4194304] 0 2024-06-03T01:20:54.663 INFO:tasks.workunit.client.0.smithi032.stdout:5/28: creat f7 x:0 0 0 2024-06-03T01:20:54.667 INFO:tasks.workunit.client.1.smithi044.stdout:2/377: mkdir db/d15/d49/d6c/d72/d7a/d4e/d7e 0 2024-06-03T01:20:54.667 INFO:tasks.workunit.client.1.smithi044.stdout:2/378: truncate db/d1b/d3f/f41 5630812 0 2024-06-03T01:20:54.669 INFO:tasks.workunit.client.0.smithi032.stdout:3/66: dwrite de/f10 [0,4194304] 0 2024-06-03T01:20:54.669 INFO:tasks.workunit.client.1.smithi044.stdout:2/379: symlink db/d15/d49/d6c/d72/d7a/d4e/l7f 0 2024-06-03T01:20:54.670 INFO:tasks.workunit.client.0.smithi032.stdout:2/75: write d5/da/f10 [4001314,82984] 0 2024-06-03T01:20:54.672 INFO:tasks.workunit.client.1.smithi044.stdout:2/380: mknod db/d1b/d2a/c80 0 2024-06-03T01:20:54.672 INFO:tasks.workunit.client.1.smithi044.stdout:4/468: read d17/d1c/d75/f4b [1089226,82287] 0 2024-06-03T01:20:54.674 INFO:tasks.workunit.client.1.smithi044.stdout:2/381: rename f2 to db/d15/d49/d6c/d72/d7a/f81 0 2024-06-03T01:20:54.675 INFO:tasks.workunit.client.0.smithi032.stdout:8/29: truncate f2 4998989 0 2024-06-03T01:20:54.676 INFO:tasks.workunit.client.0.smithi032.stdout:5/29: symlink l8 0 2024-06-03T01:20:54.677 INFO:tasks.workunit.client.0.smithi032.stdout:4/37: dread d4/db/fc [0,4194304] 0 2024-06-03T01:20:54.677 INFO:tasks.workunit.client.0.smithi032.stdout:4/38: readlink d4/l5 0 2024-06-03T01:20:54.677 INFO:tasks.workunit.client.1.smithi044.stdout:4/469: creat d17/d23/d8d/f9e x:0 0 0 2024-06-03T01:20:54.678 INFO:tasks.workunit.client.1.smithi044.stdout:7/545: dwrite d0/d3/d8/d32/d33/fb1 [0,4194304] 0 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:1/539: dwrite d0/d3/d20/d40/d45/d4b/d80/fb4 [0,4194304] 0 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:1/540: creat d0/d3/d20/d7/d13/fbc x:0 0 0 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:1/541: chown d0/d3/d20/d7/d13/d2d/f79 53 1 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:1/542: creat d0/d3/d20/d40/d45/d4b/fbd x:0 0 0 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:1/543: chown d0/d3/d20/d7/fb1 12401 1 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.1.smithi044.stdout:5/475: dwrite d2/dc/d28/d66/f8e [0,4194304] 0 2024-06-03T01:20:54.680 INFO:tasks.workunit.client.0.smithi032.stdout:2/76: creat d5/f1b x:0 0 0 2024-06-03T01:20:54.681 INFO:tasks.workunit.client.0.smithi032.stdout:1/15: sync 2024-06-03T01:20:54.681 INFO:tasks.workunit.client.0.smithi032.stdout:7/52: sync 2024-06-03T01:20:54.682 INFO:tasks.workunit.client.1.smithi044.stdout:7/546: mknod d0/d3/d30/cb5 0 2024-06-03T01:20:54.683 INFO:tasks.workunit.client.0.smithi032.stdout:4/39: dread d4/f6 [0,4194304] 0 2024-06-03T01:20:54.683 INFO:tasks.workunit.client.0.smithi032.stdout:6/74: mknod d8/c15 0 2024-06-03T01:20:54.684 INFO:tasks.workunit.client.1.smithi044.stdout:2/382: getdents db/d15 0 2024-06-03T01:20:54.684 INFO:tasks.workunit.client.1.smithi044.stdout:4/470: write d17/d23/f57 [554965,120149] 0 2024-06-03T01:20:54.685 INFO:tasks.workunit.client.0.smithi032.stdout:8/30: truncate f1 1084623 0 2024-06-03T01:20:54.685 INFO:tasks.workunit.client.0.smithi032.stdout:2/77: fsync d5/da/f10 0 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:2/78: fdatasync d5/da/f18 0 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:2/79: creat d5/f1c x:0 0 0 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:2/80: stat d5/ff 0 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:2/81: dread - d5/f1b zero size 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:9/122: dwrite d0/f1 [0,4194304] 0 2024-06-03T01:20:54.686 INFO:tasks.workunit.client.0.smithi032.stdout:9/123: chown d0/d11/c18 8209061 1 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:9/124: readlink d0/d11/d1d/l20 0 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:9/125: truncate d0/d11/d1d/f1e 286221 0 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:5/30: unlink l8 0 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:5/31: write f4 [4308217,90626] 0 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:5/32: dread - f7 zero size 2024-06-03T01:20:54.687 INFO:tasks.workunit.client.0.smithi032.stdout:5/33: write f7 [577539,87597] 0 2024-06-03T01:20:54.690 INFO:tasks.workunit.client.0.smithi032.stdout:1/16: symlink l4 0 2024-06-03T01:20:54.690 INFO:tasks.workunit.client.0.smithi032.stdout:1/17: write - no filename 2024-06-03T01:20:54.692 INFO:tasks.workunit.client.0.smithi032.stdout:9/126: dread d0/fd [0,4194304] 0 2024-06-03T01:20:54.692 INFO:tasks.workunit.client.0.smithi032.stdout:9/127: fdatasync d0/fe 0 2024-06-03T01:20:54.692 INFO:tasks.workunit.client.0.smithi032.stdout:4/40: link d4/l5 d4/db/le 0 2024-06-03T01:20:54.694 INFO:tasks.workunit.client.0.smithi032.stdout:6/75: mknod d8/d11/c16 0 2024-06-03T01:20:54.695 INFO:tasks.workunit.client.1.smithi044.stdout:7/547: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/lb6 0 2024-06-03T01:20:54.695 INFO:tasks.workunit.client.1.smithi044.stdout:7/548: fsync d0/d3/d8/d32/f6c 0 2024-06-03T01:20:54.697 INFO:tasks.workunit.client.1.smithi044.stdout:2/383: mkdir db/d15/d49/d82 0 2024-06-03T01:20:54.698 INFO:tasks.workunit.client.0.smithi032.stdout:2/82: mknod d5/da/d1a/c1d 0 2024-06-03T01:20:54.698 INFO:tasks.workunit.client.1.smithi044.stdout:4/471: symlink d17/d23/d8d/l9f 0 2024-06-03T01:20:54.700 INFO:tasks.workunit.client.1.smithi044.stdout:1/544: rename d0/d16/d64/d6a to d0/d66/dbe 0 2024-06-03T01:20:54.701 INFO:tasks.workunit.client.0.smithi032.stdout:5/34: symlink l9 0 2024-06-03T01:20:54.703 INFO:tasks.workunit.client.0.smithi032.stdout:9/128: mkdir d0/d23/d25 0 2024-06-03T01:20:54.704 INFO:tasks.workunit.client.1.smithi044.stdout:7/549: symlink d0/lb7 0 2024-06-03T01:20:54.704 INFO:tasks.workunit.client.1.smithi044.stdout:2/384: mknod db/d1b/d2a/d2f/d4c/c83 0 2024-06-03T01:20:54.704 INFO:tasks.workunit.client.1.smithi044.stdout:2/385: chown db/d15/d49/d6c 10 1 2024-06-03T01:20:54.705 INFO:tasks.workunit.client.0.smithi032.stdout:4/41: creat d4/db/ff x:0 0 0 2024-06-03T01:20:54.705 INFO:tasks.workunit.client.0.smithi032.stdout:4/42: write d4/db/ff [655506,76510] 0 2024-06-03T01:20:54.706 INFO:tasks.workunit.client.1.smithi044.stdout:4/472: creat d17/d1c/d70/d7d/fa0 x:0 0 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/473: truncate d17/d23/d28/d2b/f34 4512256 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/474: creat d17/d23/d58/fa1 x:0 0 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/475: write f5 [1724191,60902] 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/476: write d17/d23/d8d/f9e [838392,6133] 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/477: readlink d17/d1c/d70/l80 0 2024-06-03T01:20:54.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/478: chown d17/d1c/d75/f5d 15 1 2024-06-03T01:20:54.708 INFO:tasks.workunit.client.1.smithi044.stdout:5/476: rename d2/d11/d12/d2b/d44/c89 to d2/d11/d12/d2b/c96 0 2024-06-03T01:20:54.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/35: mkdir da 0 2024-06-03T01:20:54.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/36: getdents da 0 2024-06-03T01:20:54.710 INFO:tasks.workunit.client.0.smithi032.stdout:9/129: mkdir d0/d23/d24/d26 0 2024-06-03T01:20:54.712 INFO:tasks.workunit.client.1.smithi044.stdout:4/479: creat d17/d1c/d7e/fa2 x:0 0 0 2024-06-03T01:20:54.712 INFO:tasks.workunit.client.0.smithi032.stdout:2/83: rename d5/f1b to d5/da/f1e 0 2024-06-03T01:20:54.712 INFO:tasks.workunit.client.0.smithi032.stdout:2/84: dread - d5/da/f18 zero size 2024-06-03T01:20:54.712 INFO:tasks.workunit.client.0.smithi032.stdout:2/85: chown d5/fe 117 1 2024-06-03T01:20:54.712 INFO:tasks.workunit.client.0.smithi032.stdout:2/86: readlink d5/l15 0 2024-06-03T01:20:54.713 INFO:tasks.workunit.client.1.smithi044.stdout:5/477: creat d2/f97 x:0 0 0 2024-06-03T01:20:54.714 INFO:tasks.workunit.client.1.smithi044.stdout:5/478: mknod d2/dc/d28/d66/c98 0 2024-06-03T01:20:54.716 INFO:tasks.workunit.client.0.smithi032.stdout:5/37: rename c1 to da/cb 0 2024-06-03T01:20:54.717 INFO:tasks.workunit.client.1.smithi044.stdout:0/561: sync 2024-06-03T01:20:54.717 INFO:tasks.workunit.client.1.smithi044.stdout:9/353: sync 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/484: sync 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/485: fsync d5/d20/d47/f71 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:3/364: sync 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/486: truncate d5/d55/d74/d87/f7b 4932373 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:6/593: sync 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/487: read d5/d16/f24 [848340,19567] 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/488: write d5/d16/f24 [1064309,111593] 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/489: truncate d5/d55/d56/f75 4617576 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/490: write d5/d55/d74/d87/f70 [339481,19959] 0 2024-06-03T01:20:54.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/491: chown d5/d55/d74/d87/f7b 37385867 1 2024-06-03T01:20:54.719 INFO:tasks.workunit.client.0.smithi032.stdout:9/130: truncate d0/f5 1341860 0 2024-06-03T01:20:54.719 INFO:tasks.workunit.client.1.smithi044.stdout:9/354: mknod d6/dd/d51/d16/c72 0 2024-06-03T01:20:54.721 INFO:tasks.workunit.client.1.smithi044.stdout:3/365: creat df/d1e/d3a/f74 x:0 0 0 2024-06-03T01:20:54.722 INFO:tasks.workunit.client.1.smithi044.stdout:6/594: mkdir d2/d23/d55/d5f/d99/dcc/dd4 0 2024-06-03T01:20:54.723 INFO:tasks.workunit.client.1.smithi044.stdout:9/355: symlink d6/d38/l73 0 2024-06-03T01:20:54.726 INFO:tasks.workunit.client.1.smithi044.stdout:3/366: rename df/d1e/d3a/f74 to df/d13/d1b/d4b/d69/f75 0 2024-06-03T01:20:54.728 INFO:tasks.workunit.client.1.smithi044.stdout:6/595: mkdir d2/d23/d44/d4d/da0/da4/dd5 0 2024-06-03T01:20:54.728 INFO:tasks.workunit.client.1.smithi044.stdout:6/596: fsync d2/d23/f33 0 2024-06-03T01:20:54.730 INFO:tasks.workunit.client.0.smithi032.stdout:5/38: dread f7 [0,4194304] 0 2024-06-03T01:20:54.734 INFO:tasks.workunit.client.1.smithi044.stdout:6/597: rename d2/d23/d30/d1e/d38/d6c/d5c/c51 to d2/d23/d30/d1e/d38/d6c/dc2/cd6 0 2024-06-03T01:20:54.735 INFO:tasks.workunit.client.1.smithi044.stdout:6/598: chown d2/d23/d30/d1e/d34/d39/d64/lb6 42 1 2024-06-03T01:20:54.737 INFO:tasks.workunit.client.0.smithi032.stdout:5/39: symlink da/lc 0 2024-06-03T01:20:54.739 INFO:tasks.workunit.client.1.smithi044.stdout:9/356: write f4 [7583521,92330] 0 2024-06-03T01:20:54.739 INFO:tasks.workunit.client.1.smithi044.stdout:9/357: read - d6/dd/d51/d3b/f5d zero size 2024-06-03T01:20:54.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/31: truncate f1 772230 0 2024-06-03T01:20:54.742 INFO:tasks.workunit.client.1.smithi044.stdout:6/599: creat d2/d23/d55/d5f/d99/dcc/dd4/fd7 x:0 0 0 2024-06-03T01:20:54.743 INFO:tasks.workunit.client.1.smithi044.stdout:6/600: write d2/d23/d30/f1f [1432674,14918] 0 2024-06-03T01:20:54.743 INFO:tasks.workunit.client.1.smithi044.stdout:5/479: dread d2/d11/d1e/d21/f75 [0,4194304] 0 2024-06-03T01:20:54.745 INFO:tasks.workunit.client.0.smithi032.stdout:8/32: creat fa x:0 0 0 2024-06-03T01:20:54.746 INFO:tasks.workunit.client.1.smithi044.stdout:6/601: creat d2/d23/d55/d5f/d99/dcc/fd8 x:0 0 0 2024-06-03T01:20:54.746 INFO:tasks.workunit.client.1.smithi044.stdout:0/562: dread d3/d9/d55/d7b/fb0 [0,4194304] 0 2024-06-03T01:20:54.747 INFO:tasks.workunit.client.0.smithi032.stdout:7/53: dwrite dc/dd/f11 [0,4194304] 0 2024-06-03T01:20:54.747 INFO:tasks.workunit.client.0.smithi032.stdout:7/54: fsync f6 0 2024-06-03T01:20:54.747 INFO:tasks.workunit.client.0.smithi032.stdout:6/76: dwrite f3 [4194304,4194304] 0 2024-06-03T01:20:54.748 INFO:tasks.workunit.client.1.smithi044.stdout:5/480: creat d2/d11/d3e/f99 x:0 0 0 2024-06-03T01:20:54.749 INFO:tasks.workunit.client.0.smithi032.stdout:7/55: unlink dc/c10 0 2024-06-03T01:20:54.752 INFO:tasks.workunit.client.0.smithi032.stdout:6/77: link f4 d8/d11/f17 0 2024-06-03T01:20:54.752 INFO:tasks.workunit.client.1.smithi044.stdout:9/358: dread d6/dd/d51/f13 [0,4194304] 0 2024-06-03T01:20:54.753 INFO:tasks.workunit.client.1.smithi044.stdout:0/563: creat d3/da6/d21/fc4 x:0 0 0 2024-06-03T01:20:54.753 INFO:tasks.workunit.client.0.smithi032.stdout:7/56: mkdir dc/d12 0 2024-06-03T01:20:54.754 INFO:tasks.workunit.client.0.smithi032.stdout:7/57: fsync f7 0 2024-06-03T01:20:54.755 INFO:tasks.workunit.client.1.smithi044.stdout:5/481: mkdir d2/d11/d1e/d80/d9a 0 2024-06-03T01:20:54.756 INFO:tasks.workunit.client.0.smithi032.stdout:5/40: dread f4 [0,4194304] 0 2024-06-03T01:20:54.757 INFO:tasks.workunit.client.1.smithi044.stdout:9/359: mkdir d6/dd/d51/d16/d4d/d55/d68/d74 0 2024-06-03T01:20:54.757 INFO:tasks.workunit.client.0.smithi032.stdout:6/78: mkdir d8/d18 0 2024-06-03T01:20:54.759 INFO:tasks.workunit.client.0.smithi032.stdout:6/79: mkdir d8/d18/d19 0 2024-06-03T01:20:54.759 INFO:tasks.workunit.client.1.smithi044.stdout:0/564: dread d3/f6d [0,4194304] 0 2024-06-03T01:20:54.761 INFO:tasks.workunit.client.1.smithi044.stdout:5/482: rename d2/dc/l81 to d2/d11/d12/d2b/d44/l9b 0 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/565: symlink d3/d9/d55/d5a/d29/d58/lc5 0 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/566: chown d3/da6/d20/d25/f7d 55624 1 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/567: chown d3/da6/d21/d94/da3/dad 234467591 1 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/568: dread - d3/da6/d56/d62/fa1 zero size 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/569: dread - d3/da6/d20/f78 zero size 2024-06-03T01:20:54.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/570: write d3/da6/fc0 [330717,89054] 0 2024-06-03T01:20:54.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/550: dwrite d0/d16/f8e [0,4194304] 0 2024-06-03T01:20:54.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/551: truncate d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 1158203 0 2024-06-03T01:20:54.776 INFO:tasks.workunit.client.0.smithi032.stdout:9/131: dwrite d0/f1b [0,4194304] 0 2024-06-03T01:20:54.777 INFO:tasks.workunit.client.1.smithi044.stdout:7/552: creat d0/d16/fb8 x:0 0 0 2024-06-03T01:20:54.779 INFO:tasks.workunit.client.1.smithi044.stdout:7/553: write d0/d3/d8/d32/f4a [771864,104520] 0 2024-06-03T01:20:54.779 INFO:tasks.workunit.client.1.smithi044.stdout:7/554: fsync d0/d3/d8/d95/d63/d7f/d9c/f50 0 2024-06-03T01:20:54.780 INFO:tasks.workunit.client.0.smithi032.stdout:9/132: creat d0/d23/d24/f27 x:0 0 0 2024-06-03T01:20:54.780 INFO:tasks.workunit.client.0.smithi032.stdout:9/133: dread - d0/d23/d24/f27 zero size 2024-06-03T01:20:54.781 INFO:tasks.workunit.client.0.smithi032.stdout:9/134: chown d0/c4 254202 1 2024-06-03T01:20:54.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/555: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fb9 x:0 0 0 2024-06-03T01:20:54.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/556: creat d0/d16/fba x:0 0 0 2024-06-03T01:20:54.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/557: write d0/d3/d8/d95/d63/f6b [4328323,49552] 0 2024-06-03T01:20:54.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/558: write d0/d3/d8/f10 [3866005,61067] 0 2024-06-03T01:20:54.783 INFO:tasks.workunit.client.0.smithi032.stdout:9/135: unlink d0/d11/l17 0 2024-06-03T01:20:54.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/43: dwrite d4/db/da/fd [0,4194304] 0 2024-06-03T01:20:54.784 INFO:tasks.workunit.client.1.smithi044.stdout:6/602: dwrite d2/d23/d30/f45 [0,4194304] 0 2024-06-03T01:20:54.784 INFO:tasks.workunit.client.1.smithi044.stdout:3/367: dwrite df/d13/d1b/f34 [0,4194304] 0 2024-06-03T01:20:54.785 INFO:tasks.workunit.client.0.smithi032.stdout:2/87: dwrite d5/da/f18 [0,4194304] 0 2024-06-03T01:20:54.785 INFO:tasks.workunit.client.1.smithi044.stdout:7/559: mknod d0/d16/cbb 0 2024-06-03T01:20:54.785 INFO:tasks.workunit.client.1.smithi044.stdout:7/560: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fb9 zero size 2024-06-03T01:20:54.786 INFO:tasks.workunit.client.1.smithi044.stdout:7/561: write d0/f1d [557305,57768] 0 2024-06-03T01:20:54.788 INFO:tasks.workunit.client.0.smithi032.stdout:4/44: unlink d4/c8 0 2024-06-03T01:20:54.788 INFO:tasks.workunit.client.0.smithi032.stdout:4/45: readlink d4/l5 0 2024-06-03T01:20:54.789 INFO:tasks.workunit.client.1.smithi044.stdout:6/603: unlink d2/d23/d55/d5f/db2/c25 0 2024-06-03T01:20:54.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/88: symlink d5/da/l1f 0 2024-06-03T01:20:54.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/89: creat d5/da/f20 x:0 0 0 2024-06-03T01:20:54.793 INFO:tasks.workunit.client.0.smithi032.stdout:2/90: dread - d5/f1c zero size 2024-06-03T01:20:54.793 INFO:tasks.workunit.client.0.smithi032.stdout:8/33: getdents . 0 2024-06-03T01:20:54.793 INFO:tasks.workunit.client.1.smithi044.stdout:3/368: link df/d1e/d2a/f30 df/d13/d1b/d4b/d69/f76 0 2024-06-03T01:20:54.794 INFO:tasks.workunit.client.0.smithi032.stdout:9/136: write d0/f1 [2940627,58951] 0 2024-06-03T01:20:54.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/46: mknod d4/db/da/c10 0 2024-06-03T01:20:54.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/47: creat d4/db/f11 x:0 0 0 2024-06-03T01:20:54.796 INFO:tasks.workunit.client.1.smithi044.stdout:7/562: getdents d0/d3/d8/d95/d63/d89/d8b 0 2024-06-03T01:20:54.796 INFO:tasks.workunit.client.1.smithi044.stdout:7/563: fsync d0/d3/d8/d95/d63/d7f/d9c/f83 0 2024-06-03T01:20:54.797 INFO:tasks.workunit.client.1.smithi044.stdout:6/604: truncate d2/d23/d55/d5f/db2/d3f/f9a 1012748 0 2024-06-03T01:20:54.797 INFO:tasks.workunit.client.0.smithi032.stdout:2/91: dread d5/f19 [0,4194304] 0 2024-06-03T01:20:54.798 INFO:tasks.workunit.client.0.smithi032.stdout:6/80: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:54.798 INFO:tasks.workunit.client.1.smithi044.stdout:9/360: dwrite d6/dd/d51/d16/d61/f6c [0,4194304] 0 2024-06-03T01:20:54.799 INFO:tasks.workunit.client.1.smithi044.stdout:5/483: dwrite d2/d11/d12/d2b/d4a/d57/f50 [0,4194304] 0 2024-06-03T01:20:54.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/571: dwrite d3/da6/fac [0,4194304] 0 2024-06-03T01:20:54.800 INFO:tasks.workunit.client.0.smithi032.stdout:8/34: symlink lb 0 2024-06-03T01:20:54.800 INFO:tasks.workunit.client.0.smithi032.stdout:5/41: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.800 INFO:tasks.workunit.client.1.smithi044.stdout:5/484: write d2/dc/f49 [935358,69615] 0 2024-06-03T01:20:54.803 INFO:tasks.workunit.client.0.smithi032.stdout:7/58: truncate dc/dd/f11 2704392 0 2024-06-03T01:20:54.804 INFO:tasks.workunit.client.0.smithi032.stdout:2/92: dread d5/fd [0,4194304] 0 2024-06-03T01:20:54.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/137: mknod d0/d23/d25/c28 0 2024-06-03T01:20:54.813 INFO:tasks.workunit.client.0.smithi032.stdout:9/138: symlink d0/d23/l29 0 2024-06-03T01:20:54.813 INFO:tasks.workunit.client.0.smithi032.stdout:9/139: write d0/f1 [4638668,36478] 0 2024-06-03T01:20:54.815 INFO:tasks.workunit.client.0.smithi032.stdout:9/140: dread d0/d11/d1d/f1e [0,4194304] 0 2024-06-03T01:20:54.816 INFO:tasks.workunit.client.0.smithi032.stdout:5/42: dread f4 [0,4194304] 0 2024-06-03T01:20:54.822 INFO:tasks.workunit.client.1.smithi044.stdout:3/369: symlink df/d13/d1b/d59/l77 0 2024-06-03T01:20:54.823 INFO:tasks.workunit.client.0.smithi032.stdout:8/35: getdents . 0 2024-06-03T01:20:54.823 INFO:tasks.workunit.client.0.smithi032.stdout:8/36: readlink l3 0 2024-06-03T01:20:54.823 INFO:tasks.workunit.client.1.smithi044.stdout:7/564: symlink d0/d3/d30/lbc 0 2024-06-03T01:20:54.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/605: link d2/d23/d44/d4d/da0/fce d2/d23/d30/d1e/d34/d39/d9b/db4/fd9 0 2024-06-03T01:20:54.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/606: write d2/d23/d30/d1e/d34/d39/d64/f9e [529128,91599] 0 2024-06-03T01:20:54.828 INFO:tasks.workunit.client.0.smithi032.stdout:4/48: rmdir d4 39 2024-06-03T01:20:54.830 INFO:tasks.workunit.client.0.smithi032.stdout:4/49: mknod d4/db/da/c12 0 2024-06-03T01:20:54.830 INFO:tasks.workunit.client.0.smithi032.stdout:4/50: write d4/f6 [863622,40091] 0 2024-06-03T01:20:54.831 INFO:tasks.workunit.client.0.smithi032.stdout:9/141: dread d0/f6 [4194304,4194304] 0 2024-06-03T01:20:54.832 INFO:tasks.workunit.client.0.smithi032.stdout:9/142: write d0/d11/d1d/f1e [812837,55160] 0 2024-06-03T01:20:54.832 INFO:tasks.workunit.client.0.smithi032.stdout:9/143: creat d0/d23/d24/f2a x:0 0 0 2024-06-03T01:20:54.832 INFO:tasks.workunit.client.1.smithi044.stdout:9/361: rename d6/dd/d51/d16/c3a to d6/dd/d45/c75 0 2024-06-03T01:20:54.832 INFO:tasks.workunit.client.1.smithi044.stdout:9/362: creat d6/dd/d51/d46/f76 x:0 0 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:5/43: dread f4 [0,4194304] 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:2/93: dwrite d5/fd [0,4194304] 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:6/81: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:2/94: fsync d5/ff 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:6/82: read d8/ff [2987518,130724] 0 2024-06-03T01:20:54.838 INFO:tasks.workunit.client.0.smithi032.stdout:6/83: chown d8/le 1559 1 2024-06-03T01:20:54.839 INFO:tasks.workunit.client.0.smithi032.stdout:9/144: rename d0/d11/d1d/l20 to d0/d23/d25/l2b 0 2024-06-03T01:20:54.839 INFO:tasks.workunit.client.0.smithi032.stdout:9/145: write d0/f1b [4739013,6118] 0 2024-06-03T01:20:54.839 INFO:tasks.workunit.client.0.smithi032.stdout:5/44: dread f7 [0,4194304] 0 2024-06-03T01:20:54.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/485: mknod d2/d11/d1e/d2f/c9c 0 2024-06-03T01:20:54.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/486: write d2/dc/d28/d4c/f88 [81126,11646] 0 2024-06-03T01:20:54.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/487: stat d2/d11 0 2024-06-03T01:20:54.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/488: write d2/d11/d12/f6c [5050245,25428] 0 2024-06-03T01:20:54.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/489: truncate d2/d11/d1e/d21/d24/f38 5128650 0 2024-06-03T01:20:54.842 INFO:tasks.workunit.client.1.smithi044.stdout:7/565: unlink d0/d16/fba 0 2024-06-03T01:20:54.842 INFO:tasks.workunit.client.1.smithi044.stdout:7/566: fdatasync d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/f80 0 2024-06-03T01:20:54.842 INFO:tasks.workunit.client.1.smithi044.stdout:7/567: fsync d0/f62 0 2024-06-03T01:20:54.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/95: unlink d5/l12 0 2024-06-03T01:20:54.843 INFO:tasks.workunit.client.1.smithi044.stdout:9/363: symlink d6/dd/d51/d16/d4d/d67/l77 0 2024-06-03T01:20:54.843 INFO:tasks.workunit.client.1.smithi044.stdout:9/364: readlink d6/dd/d51/d16/d4d/d67/l77 0 2024-06-03T01:20:54.844 INFO:tasks.workunit.client.1.smithi044.stdout:9/365: chown d6/dd/d51/d16/d27/c2e 1 1 2024-06-03T01:20:54.845 INFO:tasks.workunit.client.0.smithi032.stdout:9/146: mknod d0/d23/d24/d26/c2c 0 2024-06-03T01:20:54.848 INFO:tasks.workunit.client.0.smithi032.stdout:5/45: mkdir da/dd 0 2024-06-03T01:20:54.848 INFO:tasks.workunit.client.0.smithi032.stdout:5/46: getdents da/dd 0 2024-06-03T01:20:54.848 INFO:tasks.workunit.client.0.smithi032.stdout:5/47: fdatasync f7 0 2024-06-03T01:20:54.849 INFO:tasks.workunit.client.1.smithi044.stdout:9/366: truncate f4 3910565 0 2024-06-03T01:20:54.850 INFO:tasks.workunit.client.0.smithi032.stdout:9/147: symlink d0/d23/l2d 0 2024-06-03T01:20:54.850 INFO:tasks.workunit.client.0.smithi032.stdout:9/148: chown d0/l16 100661295 1 2024-06-03T01:20:54.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/367: creat d6/dd/d51/d16/d27/d5e/d71/f78 x:0 0 0 2024-06-03T01:20:54.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/149: symlink d0/d23/l2e 0 2024-06-03T01:20:54.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/150: truncate d0/d23/d24/f2a 303487 0 2024-06-03T01:20:54.854 INFO:tasks.workunit.client.1.smithi044.stdout:9/368: mkdir d6/dd/d51/d16/d5f/d79 0 2024-06-03T01:20:54.855 INFO:tasks.workunit.client.1.smithi044.stdout:9/369: chown d6/dd/d51/d16/c1c 16346312 1 2024-06-03T01:20:54.855 INFO:tasks.workunit.client.0.smithi032.stdout:5/48: dread f4 [0,4194304] 0 2024-06-03T01:20:54.855 INFO:tasks.workunit.client.0.smithi032.stdout:9/151: rename d0/c1a to d0/d23/d24/c2f 0 2024-06-03T01:20:54.857 INFO:tasks.workunit.client.0.smithi032.stdout:5/49: creat da/dd/fe x:0 0 0 2024-06-03T01:20:54.857 INFO:tasks.workunit.client.0.smithi032.stdout:5/50: write f4 [5200525,61885] 0 2024-06-03T01:20:54.859 INFO:tasks.workunit.client.1.smithi044.stdout:7/568: dread d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [0,4194304] 0 2024-06-03T01:20:54.859 INFO:tasks.workunit.client.0.smithi032.stdout:2/96: dread d5/f13 [0,4194304] 0 2024-06-03T01:20:54.859 INFO:tasks.workunit.client.0.smithi032.stdout:2/97: chown c4 425069630 1 2024-06-03T01:20:54.860 INFO:tasks.workunit.client.0.smithi032.stdout:5/51: mkdir da/dd/df 0 2024-06-03T01:20:54.860 INFO:tasks.workunit.client.0.smithi032.stdout:5/52: stat l5 0 2024-06-03T01:20:54.861 INFO:tasks.workunit.client.0.smithi032.stdout:5/53: mkdir da/dd/df/d10 0 2024-06-03T01:20:54.861 INFO:tasks.workunit.client.0.smithi032.stdout:5/54: stat da/dd/fe 0 2024-06-03T01:20:54.862 INFO:tasks.workunit.client.0.smithi032.stdout:5/55: rmdir da 39 2024-06-03T01:20:54.862 INFO:tasks.workunit.client.0.smithi032.stdout:5/56: chown da/dd/df 28 1 2024-06-03T01:20:54.862 INFO:tasks.workunit.client.0.smithi032.stdout:5/57: fsync f7 0 2024-06-03T01:20:54.862 INFO:tasks.workunit.client.0.smithi032.stdout:9/152: dread d0/f1 [0,4194304] 0 2024-06-03T01:20:54.863 INFO:tasks.workunit.client.0.smithi032.stdout:5/58: mknod da/dd/df/c11 0 2024-06-03T01:20:54.869 INFO:tasks.workunit.client.0.smithi032.stdout:8/37: dwrite f8 [0,4194304] 0 2024-06-03T01:20:54.869 INFO:tasks.workunit.client.0.smithi032.stdout:8/38: truncate f2 5429416 0 2024-06-03T01:20:54.870 INFO:tasks.workunit.client.0.smithi032.stdout:8/39: mkdir dc 0 2024-06-03T01:20:54.876 INFO:tasks.workunit.client.0.smithi032.stdout:7/59: truncate dc/dd/f11 2650930 0 2024-06-03T01:20:54.878 INFO:tasks.workunit.client.1.smithi044.stdout:7/569: read d0/d3/d8/d95/d63/d7f/d9c/f83 [3597114,172] 0 2024-06-03T01:20:54.880 INFO:tasks.workunit.client.0.smithi032.stdout:4/51: dwrite d4/db/f11 [0,4194304] 0 2024-06-03T01:20:54.880 INFO:tasks.workunit.client.1.smithi044.stdout:7/570: rename d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fb9 to d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fbd 0 2024-06-03T01:20:54.881 INFO:tasks.workunit.client.1.smithi044.stdout:7/571: read d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 [56934,27317] 0 2024-06-03T01:20:54.881 INFO:tasks.workunit.client.0.smithi032.stdout:9/153: dread d0/ff [0,4194304] 0 2024-06-03T01:20:54.881 INFO:tasks.workunit.client.0.smithi032.stdout:6/84: dwrite d8/fb [0,4194304] 0 2024-06-03T01:20:54.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/490: dwrite d2/dc/d28/d66/f8e [0,4194304] 0 2024-06-03T01:20:54.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/491: stat d2/dc/d28/c55 0 2024-06-03T01:20:54.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/492: chown d2/d11/d12/d2b/f72 1814391 1 2024-06-03T01:20:54.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/493: chown d2/f13 16540 1 2024-06-03T01:20:54.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/494: readlink d2/d11/d1e/d2f/l3d 0 2024-06-03T01:20:54.883 INFO:tasks.workunit.client.1.smithi044.stdout:7/572: creat d0/d3/d8/da0/fbe x:0 0 0 2024-06-03T01:20:54.884 INFO:tasks.workunit.client.0.smithi032.stdout:9/154: symlink d0/d23/d25/l30 0 2024-06-03T01:20:54.886 INFO:tasks.workunit.client.0.smithi032.stdout:6/85: truncate d8/d11/f12 401660 0 2024-06-03T01:20:54.889 INFO:tasks.workunit.client.0.smithi032.stdout:9/155: creat d0/d11/f31 x:0 0 0 2024-06-03T01:20:54.889 INFO:tasks.workunit.client.0.smithi032.stdout:2/98: truncate d5/fd 3382876 0 2024-06-03T01:20:54.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/607: dwrite d2/d23/d30/d1e/f47 [0,4194304] 0 2024-06-03T01:20:54.890 INFO:tasks.workunit.client.0.smithi032.stdout:9/156: symlink d0/d11/l32 0 2024-06-03T01:20:54.891 INFO:tasks.workunit.client.0.smithi032.stdout:9/157: read - d0/f19 zero size 2024-06-03T01:20:54.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/158: symlink d0/d11/d1d/l33 0 2024-06-03T01:20:54.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/159: mknod d0/d11/c34 0 2024-06-03T01:20:54.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/160: write d0/d23/d24/f27 [187026,117085] 0 2024-06-03T01:20:54.895 INFO:tasks.workunit.client.1.smithi044.stdout:7/573: write d0/d3/d8/d95/d63/d89/fa5 [5002078,96246] 0 2024-06-03T01:20:54.896 INFO:tasks.workunit.client.0.smithi032.stdout:8/40: dwrite fa [0,4194304] 0 2024-06-03T01:20:54.896 INFO:tasks.workunit.client.0.smithi032.stdout:8/41: truncate f9 365809 0 2024-06-03T01:20:54.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/608: unlink d2/d23/f2e 0 2024-06-03T01:20:54.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/609: truncate d2/d23/d30/d1e/d34/d39/d7d/fac 171125 0 2024-06-03T01:20:54.897 INFO:tasks.workunit.client.1.smithi044.stdout:6/610: readlink d2/d23/l8b 0 2024-06-03T01:20:54.897 INFO:tasks.workunit.client.1.smithi044.stdout:6/611: stat d2/d23/d30/f1d 0 2024-06-03T01:20:54.901 INFO:tasks.workunit.client.0.smithi032.stdout:4/52: write d4/db/f11 [1404629,26804] 0 2024-06-03T01:20:54.903 INFO:tasks.workunit.client.0.smithi032.stdout:5/59: getdents da/dd/df 0 2024-06-03T01:20:54.903 INFO:tasks.workunit.client.0.smithi032.stdout:5/60: chown l9 0 1 2024-06-03T01:20:54.904 INFO:tasks.workunit.client.1.smithi044.stdout:5/495: rename d2/dc to d2/d11/d1e/d21/d24/d93/d9d 0 2024-06-03T01:20:54.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/99: dread d5/f19 [0,4194304] 0 2024-06-03T01:20:54.913 INFO:tasks.workunit.client.0.smithi032.stdout:8/42: dread fa [0,4194304] 0 2024-06-03T01:20:54.913 INFO:tasks.workunit.client.0.smithi032.stdout:8/43: chown c6 25478 1 2024-06-03T01:20:54.916 INFO:tasks.workunit.client.0.smithi032.stdout:6/86: dwrite d8/d11/f17 [0,4194304] 0 2024-06-03T01:20:54.921 INFO:tasks.workunit.client.1.smithi044.stdout:6/612: dwrite d2/d23/f33 [0,4194304] 0 2024-06-03T01:20:54.922 INFO:tasks.workunit.client.0.smithi032.stdout:4/53: creat d4/f13 x:0 0 0 2024-06-03T01:20:54.922 INFO:tasks.workunit.client.0.smithi032.stdout:4/54: readlink d4/db/le 0 2024-06-03T01:20:54.922 INFO:tasks.workunit.client.0.smithi032.stdout:4/55: chown d4/db/da 102279 1 2024-06-03T01:20:54.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/613: mkdir d2/d23/d44/dda 0 2024-06-03T01:20:54.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/614: truncate d2/d23/d44/f94 149563 0 2024-06-03T01:20:54.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/615: readlink d2/d23/d55/d5f/db2/l8 0 2024-06-03T01:20:54.924 INFO:tasks.workunit.client.1.smithi044.stdout:6/616: mkdir d2/d23/d55/d5f/d99/dcc/ddb 0 2024-06-03T01:20:54.924 INFO:tasks.workunit.client.0.smithi032.stdout:2/100: link d5/da/d1a/c1d d5/da/d1a/c21 0 2024-06-03T01:20:54.925 INFO:tasks.workunit.client.0.smithi032.stdout:9/161: rmdir d0 39 2024-06-03T01:20:54.926 INFO:tasks.workunit.client.0.smithi032.stdout:7/60: link dc/dd/f11 dc/d12/f13 0 2024-06-03T01:20:54.931 INFO:tasks.workunit.client.0.smithi032.stdout:4/56: mknod d4/db/c14 0 2024-06-03T01:20:54.932 INFO:tasks.workunit.client.0.smithi032.stdout:2/101: rename d5/da/d1a/c1d to d5/da/d1a/c22 0 2024-06-03T01:20:54.933 INFO:tasks.workunit.client.0.smithi032.stdout:9/162: creat d0/d11/f35 x:0 0 0 2024-06-03T01:20:54.934 INFO:tasks.workunit.client.0.smithi032.stdout:7/61: stat lb 0 2024-06-03T01:20:54.935 INFO:tasks.workunit.client.1.smithi044.stdout:6/617: dread d2/d23/d30/d1e/f49 [0,4194304] 0 2024-06-03T01:20:54.935 INFO:tasks.workunit.client.0.smithi032.stdout:6/87: link d8/lc d8/d18/d19/l1a 0 2024-06-03T01:20:54.936 INFO:tasks.workunit.client.0.smithi032.stdout:2/102: link d5/da/f20 d5/f23 0 2024-06-03T01:20:54.938 INFO:tasks.workunit.client.0.smithi032.stdout:9/163: rename d0/l16 to d0/d23/d24/d26/l36 0 2024-06-03T01:20:54.941 INFO:tasks.workunit.client.0.smithi032.stdout:2/103: read d5/f13 [3494969,20189] 0 2024-06-03T01:20:54.941 INFO:tasks.workunit.client.0.smithi032.stdout:2/104: fsync d5/fe 0 2024-06-03T01:20:54.941 INFO:tasks.workunit.client.0.smithi032.stdout:2/105: chown d5/da/d1a/c22 316 1 2024-06-03T01:20:54.947 INFO:tasks.workunit.client.1.smithi044.stdout:5/496: dwrite d2/d11/d1e/d21/f75 [0,4194304] 0 2024-06-03T01:20:54.947 INFO:tasks.workunit.client.0.smithi032.stdout:4/57: read d4/db/da/fd [2564042,80415] 0 2024-06-03T01:20:54.948 INFO:tasks.workunit.client.0.smithi032.stdout:8/44: dwrite fa [0,4194304] 0 2024-06-03T01:20:54.948 INFO:tasks.workunit.client.0.smithi032.stdout:7/62: dread dc/d12/f13 [0,4194304] 0 2024-06-03T01:20:54.948 INFO:tasks.workunit.client.0.smithi032.stdout:5/61: dwrite f4 [0,4194304] 0 2024-06-03T01:20:54.949 INFO:tasks.workunit.client.0.smithi032.stdout:2/106: write d5/da/f18 [927338,18108] 0 2024-06-03T01:20:54.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/618: write d2/d23/d30/d1e/d34/d39/f3c [4192397,27082] 0 2024-06-03T01:20:54.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/619: dread - d2/d23/d55/d5f/db2/d3f/fc5 zero size 2024-06-03T01:20:54.949 INFO:tasks.workunit.client.1.smithi044.stdout:7/574: dwrite d0/d3/d8/d32/d33/f67 [0,4194304] 0 2024-06-03T01:20:54.950 INFO:tasks.workunit.client.0.smithi032.stdout:4/58: creat d4/db/da/f15 x:0 0 0 2024-06-03T01:20:54.950 INFO:tasks.workunit.client.0.smithi032.stdout:4/59: write d4/f7 [453580,12264] 0 2024-06-03T01:20:54.950 INFO:tasks.workunit.client.1.smithi044.stdout:5/497: mkdir d2/d11/d1e/d21/d24/d9e 0 2024-06-03T01:20:54.950 INFO:tasks.workunit.client.1.smithi044.stdout:5/498: creat d2/f9f x:0 0 0 2024-06-03T01:20:54.951 INFO:tasks.workunit.client.1.smithi044.stdout:5/499: chown d2/d11/d1e/l36 212160780 1 2024-06-03T01:20:54.951 INFO:tasks.workunit.client.0.smithi032.stdout:4/60: dread d4/db/ff [0,4194304] 0 2024-06-03T01:20:54.952 INFO:tasks.workunit.client.0.smithi032.stdout:8/45: creat dc/fd x:0 0 0 2024-06-03T01:20:54.953 INFO:tasks.workunit.client.1.smithi044.stdout:6/620: creat d2/d23/d30/d1e/d34/fdc x:0 0 0 2024-06-03T01:20:54.953 INFO:tasks.workunit.client.0.smithi032.stdout:0/39: sync 2024-06-03T01:20:54.953 INFO:tasks.workunit.client.0.smithi032.stdout:7/63: getdents dc/d12 0 2024-06-03T01:20:54.955 INFO:tasks.workunit.client.0.smithi032.stdout:3/67: sync 2024-06-03T01:20:54.955 INFO:tasks.workunit.client.1.smithi044.stdout:7/575: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:54.958 INFO:tasks.workunit.client.0.smithi032.stdout:5/62: creat da/f12 x:0 0 0 2024-06-03T01:20:54.958 INFO:tasks.workunit.client.0.smithi032.stdout:1/18: sync 2024-06-03T01:20:54.958 INFO:tasks.workunit.client.0.smithi032.stdout:1/19: dwrite - no filename 2024-06-03T01:20:54.958 INFO:tasks.workunit.client.0.smithi032.stdout:1/20: write - no filename 2024-06-03T01:20:54.958 INFO:tasks.workunit.client.0.smithi032.stdout:1/21: write - no filename 2024-06-03T01:20:54.959 INFO:tasks.workunit.client.0.smithi032.stdout:1/22: rmdir - no directory 2024-06-03T01:20:54.959 INFO:tasks.workunit.client.1.smithi044.stdout:2/386: sync 2024-06-03T01:20:54.959 INFO:tasks.workunit.client.1.smithi044.stdout:2/387: fdatasync db/d1b/d2a/d67/f7c 0 2024-06-03T01:20:54.959 INFO:tasks.workunit.client.1.smithi044.stdout:1/545: sync 2024-06-03T01:20:54.959 INFO:tasks.workunit.client.1.smithi044.stdout:1/546: chown d0/d3/d20/f9c 7776776 1 2024-06-03T01:20:54.960 INFO:tasks.workunit.client.0.smithi032.stdout:2/107: symlink d5/l24 0 2024-06-03T01:20:54.960 INFO:tasks.workunit.client.0.smithi032.stdout:2/108: dread - d5/da/f20 zero size 2024-06-03T01:20:54.960 INFO:tasks.workunit.client.1.smithi044.stdout:2/388: unlink db/d1b/d27/f5f 0 2024-06-03T01:20:54.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/547: creat d0/d66/fbf x:0 0 0 2024-06-03T01:20:54.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/548: readlink d0/d3/d20/d40/d45/l9b 0 2024-06-03T01:20:54.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/549: dread - d0/d3/d20/f9c zero size 2024-06-03T01:20:54.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/550: creat d0/d3/d20/d40/d45/d4b/d5c/d94/fc0 x:0 0 0 2024-06-03T01:20:54.963 INFO:tasks.workunit.client.0.smithi032.stdout:4/61: rename d4/f13 to d4/db/da/f16 0 2024-06-03T01:20:54.966 INFO:tasks.workunit.client.0.smithi032.stdout:8/46: dread f8 [0,4194304] 0 2024-06-03T01:20:54.966 INFO:tasks.workunit.client.0.smithi032.stdout:8/47: chown c6 20061 1 2024-06-03T01:20:54.968 INFO:tasks.workunit.client.1.smithi044.stdout:6/621: dread d2/d23/d55/d5f/db2/d3f/f9a [0,4194304] 0 2024-06-03T01:20:54.968 INFO:tasks.workunit.client.1.smithi044.stdout:6/622: dread - d2/d23/d30/d1e/d38/f78 zero size 2024-06-03T01:20:54.968 INFO:tasks.workunit.client.1.smithi044.stdout:6/623: readlink d2/d23/d30/d1e/d34/lcf 0 2024-06-03T01:20:54.969 INFO:tasks.workunit.client.1.smithi044.stdout:4/480: sync 2024-06-03T01:20:54.969 INFO:tasks.workunit.client.0.smithi032.stdout:7/64: creat dc/f14 x:0 0 0 2024-06-03T01:20:54.971 INFO:tasks.workunit.client.1.smithi044.stdout:6/624: rename d2/d23/d55/d5f/db2/f8d to d2/d23/d30/d1e/d34/d39/d9b/db4/fdd 0 2024-06-03T01:20:54.971 INFO:tasks.workunit.client.1.smithi044.stdout:6/625: write d2/d23/d55/d5f/db2/d3f/fa6 [3303743,53620] 0 2024-06-03T01:20:54.971 INFO:tasks.workunit.client.0.smithi032.stdout:5/63: symlink da/dd/df/d10/l13 0 2024-06-03T01:20:54.972 INFO:tasks.workunit.client.1.smithi044.stdout:4/481: mknod d17/d1c/d75/ca3 0 2024-06-03T01:20:54.972 INFO:tasks.workunit.client.1.smithi044.stdout:4/482: write d17/d23/d59/f5f [4164831,27325] 0 2024-06-03T01:20:54.972 INFO:tasks.workunit.client.0.smithi032.stdout:1/23: creat f5 x:0 0 0 2024-06-03T01:20:54.974 INFO:tasks.workunit.client.0.smithi032.stdout:8/48: truncate f8 1503393 0 2024-06-03T01:20:54.974 INFO:tasks.workunit.client.0.smithi032.stdout:8/49: chown l3 4695 1 2024-06-03T01:20:54.974 INFO:tasks.workunit.client.0.smithi032.stdout:8/50: readlink l4 0 2024-06-03T01:20:54.975 INFO:tasks.workunit.client.1.smithi044.stdout:4/483: dread d17/d23/d28/d2b/f93 [0,4194304] 0 2024-06-03T01:20:54.976 INFO:tasks.workunit.client.0.smithi032.stdout:9/164: getdents d0/d23/d24/d26 0 2024-06-03T01:20:54.976 INFO:tasks.workunit.client.0.smithi032.stdout:9/165: chown d0/d23/d24/c2f 0 1 2024-06-03T01:20:54.977 INFO:tasks.workunit.client.1.smithi044.stdout:2/389: read db/dd/f11 [1622932,129336] 0 2024-06-03T01:20:54.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/484: getdents d17/d23/d28/d2b 0 2024-06-03T01:20:54.977 INFO:tasks.workunit.client.1.smithi044.stdout:7/576: dwrite d0/d3/d8/d32/f90 [0,4194304] 0 2024-06-03T01:20:54.977 INFO:tasks.workunit.client.0.smithi032.stdout:0/40: mkdir d1/da 0 2024-06-03T01:20:54.979 INFO:tasks.workunit.client.0.smithi032.stdout:3/68: getdents de 0 2024-06-03T01:20:54.979 INFO:tasks.workunit.client.1.smithi044.stdout:4/485: unlink d17/d23/d58/fa1 0 2024-06-03T01:20:54.980 INFO:tasks.workunit.client.1.smithi044.stdout:4/486: readlink d17/d23/d28/d2b/l29 0 2024-06-03T01:20:54.983 INFO:tasks.workunit.client.0.smithi032.stdout:5/64: creat da/dd/df/d10/f14 x:0 0 0 2024-06-03T01:20:54.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/487: symlink d17/d23/d58/la4 0 2024-06-03T01:20:54.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/488: creat d17/d23/fa5 x:0 0 0 2024-06-03T01:20:54.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/24: link c2 c6 0 2024-06-03T01:20:54.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/489: truncate d17/d63/f65 675483 0 2024-06-03T01:20:54.989 INFO:tasks.workunit.client.1.smithi044.stdout:2/390: write db/f26 [10834415,120733] 0 2024-06-03T01:20:54.989 INFO:tasks.workunit.client.1.smithi044.stdout:2/391: fsync db/dd/f60 0 2024-06-03T01:20:54.990 INFO:tasks.workunit.client.1.smithi044.stdout:4/490: rmdir d17/d23/d6a 39 2024-06-03T01:20:54.990 INFO:tasks.workunit.client.1.smithi044.stdout:4/491: chown d17/d1c/d70 77666 1 2024-06-03T01:20:54.991 INFO:tasks.workunit.client.1.smithi044.stdout:2/392: symlink db/l84 0 2024-06-03T01:20:54.993 INFO:tasks.workunit.client.1.smithi044.stdout:4/492: symlink d17/d23/d59/la6 0 2024-06-03T01:20:54.993 INFO:tasks.workunit.client.1.smithi044.stdout:2/393: write db/d1b/d2a/f2b [537890,117832] 0 2024-06-03T01:20:54.993 INFO:tasks.workunit.client.0.smithi032.stdout:9/166: unlink d0/d23/l2e 0 2024-06-03T01:20:54.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/493: mknod d17/d23/d59/ca7 0 2024-06-03T01:20:54.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/494: readlink d17/d23/d35/l6b 0 2024-06-03T01:20:54.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/495: fsync d17/d23/d28/d2b/f3d 0 2024-06-03T01:20:54.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/496: chown fc 577 1 2024-06-03T01:20:54.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/497: creat d17/d1c/d7e/fa8 x:0 0 0 2024-06-03T01:20:54.995 INFO:tasks.workunit.client.1.smithi044.stdout:2/394: symlink db/d1b/d2a/d52/d54/l85 0 2024-06-03T01:20:54.995 INFO:tasks.workunit.client.1.smithi044.stdout:2/395: stat db/dd/f60 0 2024-06-03T01:20:54.995 INFO:tasks.workunit.client.0.smithi032.stdout:3/69: write f8 [1456924,113359] 0 2024-06-03T01:20:54.997 INFO:tasks.workunit.client.1.smithi044.stdout:4/498: symlink d17/d1c/d75/d7a/la9 0 2024-06-03T01:20:54.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/167: rename d0/d11/f35 to d0/d23/d25/f37 0 2024-06-03T01:20:54.999 INFO:tasks.workunit.client.1.smithi044.stdout:4/499: mknod d17/d1c/d70/d7d/caa 0 2024-06-03T01:20:54.999 INFO:tasks.workunit.client.1.smithi044.stdout:4/500: write d17/d23/f9b [75782,54375] 0 2024-06-03T01:20:54.999 INFO:tasks.workunit.client.1.smithi044.stdout:4/501: chown d17/d23/d28/d2b/d8e 1872 1 2024-06-03T01:20:54.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/41: symlink d1/lb 0 2024-06-03T01:20:55.000 INFO:tasks.workunit.client.0.smithi032.stdout:3/70: creat de/f15 x:0 0 0 2024-06-03T01:20:55.002 INFO:tasks.workunit.client.0.smithi032.stdout:9/168: mkdir d0/d11/d1d/d38 0 2024-06-03T01:20:55.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/42: creat d1/fc x:0 0 0 2024-06-03T01:20:55.005 INFO:tasks.workunit.client.0.smithi032.stdout:3/71: truncate de/f10 1030173 0 2024-06-03T01:20:55.006 INFO:tasks.workunit.client.0.smithi032.stdout:9/169: creat d0/d23/d24/d26/f39 x:0 0 0 2024-06-03T01:20:55.006 INFO:tasks.workunit.client.0.smithi032.stdout:9/170: chown d0/ff 0 1 2024-06-03T01:20:55.008 INFO:tasks.workunit.client.0.smithi032.stdout:9/171: symlink d0/d11/d1d/d38/l3a 0 2024-06-03T01:20:55.010 INFO:tasks.workunit.client.0.smithi032.stdout:9/172: creat d0/d23/d24/f3b x:0 0 0 2024-06-03T01:20:55.010 INFO:tasks.workunit.client.0.smithi032.stdout:9/173: mknod d0/d11/c3c 0 2024-06-03T01:20:55.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/174: chown d0/d11/d1d/l33 0 1 2024-06-03T01:20:55.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/175: truncate d0/f19 590197 0 2024-06-03T01:20:55.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/176: dread - d0/d23/d25/f37 zero size 2024-06-03T01:20:55.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/177: creat d0/d23/f3d x:0 0 0 2024-06-03T01:20:55.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/178: rmdir d0/d11/d1d 39 2024-06-03T01:20:55.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/179: fdatasync d0/f5 0 2024-06-03T01:20:55.014 INFO:tasks.workunit.client.0.smithi032.stdout:9/180: creat d0/d11/d1d/d38/f3e x:0 0 0 2024-06-03T01:20:55.014 INFO:tasks.workunit.client.0.smithi032.stdout:9/181: fsync d0/d23/d24/f2a 0 2024-06-03T01:20:55.024 INFO:tasks.workunit.client.0.smithi032.stdout:2/109: dwrite d5/f13 [0,4194304] 0 2024-06-03T01:20:55.039 INFO:tasks.workunit.client.1.smithi044.stdout:7/577: dwrite d0/d3/d8/d95/d84/f93 [0,4194304] 0 2024-06-03T01:20:55.039 INFO:tasks.workunit.client.1.smithi044.stdout:7/578: stat d0/d3/d30/cb5 0 2024-06-03T01:20:55.039 INFO:tasks.workunit.client.1.smithi044.stdout:7/579: truncate d0/f62 703775 0 2024-06-03T01:20:55.042 INFO:tasks.workunit.client.1.smithi044.stdout:7/580: mknod d0/d3/d8/cbf 0 2024-06-03T01:20:55.042 INFO:tasks.workunit.client.1.smithi044.stdout:6/626: dwrite d2/d23/d30/d1e/d34/d39/d9b/f9c [0,4194304] 0 2024-06-03T01:20:55.043 INFO:tasks.workunit.client.0.smithi032.stdout:7/65: dwrite f8 [0,4194304] 0 2024-06-03T01:20:55.043 INFO:tasks.workunit.client.1.smithi044.stdout:4/502: dwrite d17/d23/f20 [0,4194304] 0 2024-06-03T01:20:55.044 INFO:tasks.workunit.client.0.smithi032.stdout:0/43: dwrite d1/f9 [0,4194304] 0 2024-06-03T01:20:55.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/44: mknod d1/da/cd 0 2024-06-03T01:20:55.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/45: rename d1/da to d1/da/de 22 2024-06-03T01:20:55.047 INFO:tasks.workunit.client.1.smithi044.stdout:9/370: sync 2024-06-03T01:20:55.047 INFO:tasks.workunit.client.1.smithi044.stdout:8/492: sync 2024-06-03T01:20:55.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/370: sync 2024-06-03T01:20:55.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/371: fdatasync df/d13/f25 0 2024-06-03T01:20:55.048 INFO:tasks.workunit.client.1.smithi044.stdout:0/572: sync 2024-06-03T01:20:55.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/493: mknod d5/d20/d47/c92 0 2024-06-03T01:20:55.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/494: chown d5/d55/d56/d5c/d68/c8d 900846 1 2024-06-03T01:20:55.051 INFO:tasks.workunit.client.1.smithi044.stdout:3/372: rmdir df/d1e/d3a 39 2024-06-03T01:20:55.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/581: write d0/d3/d8/d32/d33/f5b [2464668,96354] 0 2024-06-03T01:20:55.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/582: truncate d0/d3/d8/d32/d33/f67 5201430 0 2024-06-03T01:20:55.052 INFO:tasks.workunit.client.1.smithi044.stdout:0/573: rmdir d3/d6b 39 2024-06-03T01:20:55.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/495: symlink d5/d20/d47/d64/l93 0 2024-06-03T01:20:55.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/496: readlink d5/d16/l85 0 2024-06-03T01:20:55.054 INFO:tasks.workunit.client.1.smithi044.stdout:8/497: creat d5/d55/d74/f94 x:0 0 0 2024-06-03T01:20:55.055 INFO:tasks.workunit.client.1.smithi044.stdout:6/627: dread d2/d23/d30/d1e/f47 [0,4194304] 0 2024-06-03T01:20:55.056 INFO:tasks.workunit.client.1.smithi044.stdout:6/628: readlink d2/d23/d55/d5f/d85/lc1 0 2024-06-03T01:20:55.056 INFO:tasks.workunit.client.1.smithi044.stdout:7/583: rename d0/d3/d8/d8a to d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0 0 2024-06-03T01:20:55.056 INFO:tasks.workunit.client.1.smithi044.stdout:0/574: read - d3/f18 zero size 2024-06-03T01:20:55.056 INFO:tasks.workunit.client.1.smithi044.stdout:0/575: creat d3/fc6 x:0 0 0 2024-06-03T01:20:55.058 INFO:tasks.workunit.client.1.smithi044.stdout:8/498: link d5/d50/l54 d5/d16/d1d/d6b/l95 0 2024-06-03T01:20:55.059 INFO:tasks.workunit.client.1.smithi044.stdout:2/396: dread db/f26 [4194304,4194304] 0 2024-06-03T01:20:55.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/629: mknod d2/d23/d30/d1e/d34/d39/da8/cde 0 2024-06-03T01:20:55.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/630: readlink d2/d23/d30/d1e/d38/d6c/l1b 0 2024-06-03T01:20:55.061 INFO:tasks.workunit.client.0.smithi032.stdout:2/110: dwrite f3 [4194304,4194304] 0 2024-06-03T01:20:55.061 INFO:tasks.workunit.client.0.smithi032.stdout:2/111: write d5/f19 [1328473,118492] 0 2024-06-03T01:20:55.062 INFO:tasks.workunit.client.1.smithi044.stdout:0/576: creat d3/d9/fc7 x:0 0 0 2024-06-03T01:20:55.063 INFO:tasks.workunit.client.1.smithi044.stdout:0/577: creat d3/d9/fc8 x:0 0 0 2024-06-03T01:20:55.063 INFO:tasks.workunit.client.1.smithi044.stdout:0/578: stat d3/d6b/d95/db3 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:7/66: dread dc/d12/f13 [0,4194304] 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:9/182: dwrite d0/f1b [0,4194304] 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:7/67: creat dc/f15 x:0 0 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:7/68: stat dc/d12 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/65: dwrite da/dd/df/d10/f14 [0,4194304] 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/66: creat da/f15 x:0 0 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.1.smithi044.stdout:6/631: mkdir d2/d23/d30/ddf 0 2024-06-03T01:20:55.064 INFO:tasks.workunit.client.1.smithi044.stdout:9/371: write f4 [1313069,7041] 0 2024-06-03T01:20:55.065 INFO:tasks.workunit.client.0.smithi032.stdout:8/51: write f1 [372859,95420] 0 2024-06-03T01:20:55.067 INFO:tasks.workunit.client.0.smithi032.stdout:2/112: truncate d5/fd 1093190 0 2024-06-03T01:20:55.067 INFO:tasks.workunit.client.0.smithi032.stdout:2/113: creat d5/f25 x:0 0 0 2024-06-03T01:20:55.068 INFO:tasks.workunit.client.1.smithi044.stdout:0/579: symlink d3/da6/d56/d8e/lc9 0 2024-06-03T01:20:55.068 INFO:tasks.workunit.client.1.smithi044.stdout:0/580: chown d3/da6/d56/d62/fa9 12918070 1 2024-06-03T01:20:55.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/632: mkdir d2/d23/d55/d5f/d99/dcc/de0 0 2024-06-03T01:20:55.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/633: dread - d2/d23/d30/d1e/d38/d6c/d5c/d79/fb8 zero size 2024-06-03T01:20:55.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/634: write d2/fe [2837585,27903] 0 2024-06-03T01:20:55.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/635: chown d2/c97 0 1 2024-06-03T01:20:55.072 INFO:tasks.workunit.client.0.smithi032.stdout:0/46: write d1/f9 [1206873,96062] 0 2024-06-03T01:20:55.072 INFO:tasks.workunit.client.0.smithi032.stdout:0/47: write d1/f8 [736571,57204] 0 2024-06-03T01:20:55.072 INFO:tasks.workunit.client.0.smithi032.stdout:0/48: readlink d1/lb 0 2024-06-03T01:20:55.075 INFO:tasks.workunit.client.1.smithi044.stdout:4/503: dread d17/d23/d28/d2b/f2e [0,4194304] 0 2024-06-03T01:20:55.075 INFO:tasks.workunit.client.1.smithi044.stdout:4/504: chown d17/d1c/d70/l7f 84 1 2024-06-03T01:20:55.076 INFO:tasks.workunit.client.1.smithi044.stdout:0/581: dread d3/da6/d56/f5d [0,4194304] 0 2024-06-03T01:20:55.077 INFO:tasks.workunit.client.1.smithi044.stdout:9/372: unlink c5 0 2024-06-03T01:20:55.079 INFO:tasks.workunit.client.1.smithi044.stdout:4/505: symlink d17/d23/d6a/d84/lab 0 2024-06-03T01:20:55.080 INFO:tasks.workunit.client.0.smithi032.stdout:7/69: write dc/dd/f11 [3250185,83078] 0 2024-06-03T01:20:55.080 INFO:tasks.workunit.client.0.smithi032.stdout:7/70: chown f7 172 1 2024-06-03T01:20:55.082 INFO:tasks.workunit.client.1.smithi044.stdout:3/373: dwrite df/d13/f3b [0,4194304] 0 2024-06-03T01:20:55.082 INFO:tasks.workunit.client.1.smithi044.stdout:3/374: write df/d13/d39/d3d/d50/f6f [861956,74944] 0 2024-06-03T01:20:55.083 INFO:tasks.workunit.client.1.smithi044.stdout:0/582: rename d3/da6/d20/d25/l5f to d3/d6b/d95/db3/lca 0 2024-06-03T01:20:55.084 INFO:tasks.workunit.client.1.smithi044.stdout:2/397: dwrite db/d1b/d3f/f61 [0,4194304] 0 2024-06-03T01:20:55.084 INFO:tasks.workunit.client.1.smithi044.stdout:2/398: write db/d1b/d2a/d2f/d4c/f77 [1043712,24217] 0 2024-06-03T01:20:55.085 INFO:tasks.workunit.client.0.smithi032.stdout:5/67: mkdir da/dd/df/d10/d16 0 2024-06-03T01:20:55.087 INFO:tasks.workunit.client.0.smithi032.stdout:8/52: mkdir dc/de 0 2024-06-03T01:20:55.088 INFO:tasks.workunit.client.1.smithi044.stdout:4/506: dread d17/d23/d35/f41 [0,4194304] 0 2024-06-03T01:20:55.091 INFO:tasks.workunit.client.0.smithi032.stdout:2/114: creat d5/da/d1a/f26 x:0 0 0 2024-06-03T01:20:55.091 INFO:tasks.workunit.client.0.smithi032.stdout:2/115: write d5/f25 [682752,70220] 0 2024-06-03T01:20:55.091 INFO:tasks.workunit.client.0.smithi032.stdout:2/116: chown d5/da/d1a 7829 1 2024-06-03T01:20:55.093 INFO:tasks.workunit.client.1.smithi044.stdout:2/399: creat db/d1b/d5d/f86 x:0 0 0 2024-06-03T01:20:55.093 INFO:tasks.workunit.client.1.smithi044.stdout:2/400: fsync db/d1b/d2a/d52/d54/f56 0 2024-06-03T01:20:55.094 INFO:tasks.workunit.client.1.smithi044.stdout:9/373: getdents d6/dd/d51/d16/d61 0 2024-06-03T01:20:55.094 INFO:tasks.workunit.client.1.smithi044.stdout:9/374: chown d6/dd/d51 21977 1 2024-06-03T01:20:55.094 INFO:tasks.workunit.client.1.smithi044.stdout:9/375: read - d6/dd/d51/d3b/f6a zero size 2024-06-03T01:20:55.095 INFO:tasks.workunit.client.1.smithi044.stdout:9/376: chown d6/dd/d51/d16/f34 7974530 1 2024-06-03T01:20:55.095 INFO:tasks.workunit.client.0.smithi032.stdout:9/183: symlink d0/l3f 0 2024-06-03T01:20:55.097 INFO:tasks.workunit.client.1.smithi044.stdout:4/507: symlink d17/d1c/d70/lac 0 2024-06-03T01:20:55.097 INFO:tasks.workunit.client.0.smithi032.stdout:7/71: link dc/f14 dc/f16 0 2024-06-03T01:20:55.099 INFO:tasks.workunit.client.1.smithi044.stdout:9/377: link d6/dd/d51/db/c10 d6/d26/c7a 0 2024-06-03T01:20:55.099 INFO:tasks.workunit.client.1.smithi044.stdout:9/378: creat d6/dd/d51/d16/f7b x:0 0 0 2024-06-03T01:20:55.100 INFO:tasks.workunit.client.1.smithi044.stdout:4/508: symlink d17/d1c/d9c/lad 0 2024-06-03T01:20:55.102 INFO:tasks.workunit.client.1.smithi044.stdout:9/379: truncate d6/dd/d51/d46/f48 197618 0 2024-06-03T01:20:55.103 INFO:tasks.workunit.client.0.smithi032.stdout:5/68: rename f7 to da/f17 0 2024-06-03T01:20:55.103 INFO:tasks.workunit.client.0.smithi032.stdout:5/69: dread - da/dd/fe zero size 2024-06-03T01:20:55.104 INFO:tasks.workunit.client.1.smithi044.stdout:7/584: dwrite d0/d3/d8/d95/f65 [0,4194304] 0 2024-06-03T01:20:55.104 INFO:tasks.workunit.client.1.smithi044.stdout:4/509: rmdir d17/d1c/d7e 39 2024-06-03T01:20:55.104 INFO:tasks.workunit.client.1.smithi044.stdout:4/510: fsync d17/d1c/d75/f4b 0 2024-06-03T01:20:55.104 INFO:tasks.workunit.client.1.smithi044.stdout:4/511: dread - d17/d1c/d75/f48 zero size 2024-06-03T01:20:55.104 INFO:tasks.workunit.client.1.smithi044.stdout:4/512: chown d17/d23/fa5 3 1 2024-06-03T01:20:55.106 INFO:tasks.workunit.client.0.smithi032.stdout:8/53: unlink c6 0 2024-06-03T01:20:55.106 INFO:tasks.workunit.client.0.smithi032.stdout:8/54: stat f1 0 2024-06-03T01:20:55.107 INFO:tasks.workunit.client.1.smithi044.stdout:7/585: unlink d0/d3/l7e 0 2024-06-03T01:20:55.108 INFO:tasks.workunit.client.1.smithi044.stdout:7/586: truncate d0/d16/d23/f24 4724953 0 2024-06-03T01:20:55.110 INFO:tasks.workunit.client.0.smithi032.stdout:7/72: mknod dc/d12/c17 0 2024-06-03T01:20:55.110 INFO:tasks.workunit.client.0.smithi032.stdout:5/70: mkdir da/dd/df/d10/d16/d18 0 2024-06-03T01:20:55.114 INFO:tasks.workunit.client.0.smithi032.stdout:9/184: rename d0/c10 to d0/d23/c40 0 2024-06-03T01:20:55.114 INFO:tasks.workunit.client.0.smithi032.stdout:5/71: rename da to da/dd/df/d10/d16/d19 22 2024-06-03T01:20:55.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/185: rmdir d0/d23/d25 39 2024-06-03T01:20:55.116 INFO:tasks.workunit.client.0.smithi032.stdout:5/72: rmdir da/dd/df/d10/d16 39 2024-06-03T01:20:55.118 INFO:tasks.workunit.client.0.smithi032.stdout:5/73: truncate da/f17 392242 0 2024-06-03T01:20:55.120 INFO:tasks.workunit.client.1.smithi044.stdout:8/499: dwrite d5/d20/d47/d64/f8b [0,4194304] 0 2024-06-03T01:20:55.123 INFO:tasks.workunit.client.1.smithi044.stdout:8/500: mkdir d5/d20/d96 0 2024-06-03T01:20:55.124 INFO:tasks.workunit.client.1.smithi044.stdout:6/636: dwrite d2/d23/d30/d1e/d34/d39/d9b/db4/f6d [0,4194304] 0 2024-06-03T01:20:55.126 INFO:tasks.workunit.client.1.smithi044.stdout:8/501: symlink d5/d20/d47/l97 0 2024-06-03T01:20:55.129 INFO:tasks.workunit.client.0.smithi032.stdout:9/186: dread d0/fe [4194304,4194304] 0 2024-06-03T01:20:55.130 INFO:tasks.workunit.client.1.smithi044.stdout:9/380: dwrite d6/dd/d51/d16/f3f [0,4194304] 0 2024-06-03T01:20:55.131 INFO:tasks.workunit.client.0.smithi032.stdout:0/49: dread d1/f9 [0,4194304] 0 2024-06-03T01:20:55.131 INFO:tasks.workunit.client.0.smithi032.stdout:0/50: write d1/f6 [314022,104506] 0 2024-06-03T01:20:55.131 INFO:tasks.workunit.client.0.smithi032.stdout:0/51: dread - d1/fc zero size 2024-06-03T01:20:55.132 INFO:tasks.workunit.client.0.smithi032.stdout:0/52: write d1/f8 [84143,11564] 0 2024-06-03T01:20:55.133 INFO:tasks.workunit.client.0.smithi032.stdout:0/53: getdents d1 0 2024-06-03T01:20:55.134 INFO:tasks.workunit.client.1.smithi044.stdout:6/637: link d2/d23/d30/d1e/d38/d6c/l1b d2/d23/d30/d1e/d34/d39/le1 0 2024-06-03T01:20:55.135 INFO:tasks.workunit.client.1.smithi044.stdout:8/502: mkdir d5/d20/d2b/d98 0 2024-06-03T01:20:55.136 INFO:tasks.workunit.client.1.smithi044.stdout:3/375: dwrite df/d13/d1b/f64 [0,4194304] 0 2024-06-03T01:20:55.136 INFO:tasks.workunit.client.0.smithi032.stdout:0/54: rename d1/lb to d1/lf 0 2024-06-03T01:20:55.137 INFO:tasks.workunit.client.0.smithi032.stdout:0/55: mkdir d1/da/d10 0 2024-06-03T01:20:55.137 INFO:tasks.workunit.client.0.smithi032.stdout:0/56: write d1/f6 [1440222,84232] 0 2024-06-03T01:20:55.137 INFO:tasks.workunit.client.1.smithi044.stdout:9/381: unlink d6/dd/d51/d3b/f47 0 2024-06-03T01:20:55.140 INFO:tasks.workunit.client.1.smithi044.stdout:2/401: dwrite db/d1b/d5d/f86 [0,4194304] 0 2024-06-03T01:20:55.140 INFO:tasks.workunit.client.0.smithi032.stdout:9/187: dread d0/f1 [0,4194304] 0 2024-06-03T01:20:55.141 INFO:tasks.workunit.client.0.smithi032.stdout:9/188: mkdir d0/d23/d24/d26/d41 0 2024-06-03T01:20:55.141 INFO:tasks.workunit.client.0.smithi032.stdout:9/189: fdatasync d0/f1 0 2024-06-03T01:20:55.142 INFO:tasks.workunit.client.1.smithi044.stdout:8/503: creat d5/d55/d74/d87/f99 x:0 0 0 2024-06-03T01:20:55.142 INFO:tasks.workunit.client.1.smithi044.stdout:8/504: read d5/d20/d2b/f30 [4107,93748] 0 2024-06-03T01:20:55.142 INFO:tasks.workunit.client.1.smithi044.stdout:8/505: creat d5/d20/d47/f9a x:0 0 0 2024-06-03T01:20:55.144 INFO:tasks.workunit.client.0.smithi032.stdout:9/190: symlink d0/d11/d1d/l42 0 2024-06-03T01:20:55.145 INFO:tasks.workunit.client.1.smithi044.stdout:6/638: dread d2/d23/d55/d5f/db2/d3f/fa6 [0,4194304] 0 2024-06-03T01:20:55.146 INFO:tasks.workunit.client.1.smithi044.stdout:6/639: chown d2/d23/d30/d1e/d34/d39/dcd 170960 1 2024-06-03T01:20:55.146 INFO:tasks.workunit.client.0.smithi032.stdout:9/191: truncate d0/f6 2481390 0 2024-06-03T01:20:55.148 INFO:tasks.workunit.client.0.smithi032.stdout:9/192: mknod d0/d11/d1d/c43 0 2024-06-03T01:20:55.149 INFO:tasks.workunit.client.1.smithi044.stdout:9/382: dread d6/dd/d51/f9 [4194304,4194304] 0 2024-06-03T01:20:55.149 INFO:tasks.workunit.client.0.smithi032.stdout:9/193: mknod d0/d11/c44 0 2024-06-03T01:20:55.153 INFO:tasks.workunit.client.1.smithi044.stdout:0/583: dwrite d3/d9/d55/d5a/f93 [0,4194304] 0 2024-06-03T01:20:55.153 INFO:tasks.workunit.client.1.smithi044.stdout:7/587: dwrite d0/d3/d8/d95/f53 [0,4194304] 0 2024-06-03T01:20:55.153 INFO:tasks.workunit.client.1.smithi044.stdout:4/513: dwrite d17/d23/d35/f4f [0,4194304] 0 2024-06-03T01:20:55.154 INFO:tasks.workunit.client.0.smithi032.stdout:5/74: dread f4 [0,4194304] 0 2024-06-03T01:20:55.154 INFO:tasks.workunit.client.0.smithi032.stdout:5/75: write f4 [5413494,47623] 0 2024-06-03T01:20:55.154 INFO:tasks.workunit.client.0.smithi032.stdout:7/73: dwrite dc/f14 [0,4194304] 0 2024-06-03T01:20:55.154 INFO:tasks.workunit.client.0.smithi032.stdout:8/55: dwrite f2 [4194304,4194304] 0 2024-06-03T01:20:55.154 INFO:tasks.workunit.client.0.smithi032.stdout:9/194: mknod d0/d23/d24/d26/c45 0 2024-06-03T01:20:55.155 INFO:tasks.workunit.client.1.smithi044.stdout:8/506: mkdir d5/d16/d81/d9b 0 2024-06-03T01:20:55.155 INFO:tasks.workunit.client.1.smithi044.stdout:8/507: stat d5/d16/d81/d9b 0 2024-06-03T01:20:55.155 INFO:tasks.workunit.client.1.smithi044.stdout:8/508: chown d5/d20/d47/f49 959 1 2024-06-03T01:20:55.161 INFO:tasks.workunit.client.0.smithi032.stdout:0/57: dread d1/f6 [0,4194304] 0 2024-06-03T01:20:55.163 INFO:tasks.workunit.client.0.smithi032.stdout:2/117: fsync d5/da/d1a/f26 0 2024-06-03T01:20:55.166 INFO:tasks.workunit.client.1.smithi044.stdout:0/584: symlink d3/d9/d55/d7b/lcb 0 2024-06-03T01:20:55.169 INFO:tasks.workunit.client.1.smithi044.stdout:7/588: unlink d0/d3/d30/lbc 0 2024-06-03T01:20:55.169 INFO:tasks.workunit.client.1.smithi044.stdout:7/589: write d0/d3/d8/f1b [4901715,93719] 0 2024-06-03T01:20:55.169 INFO:tasks.workunit.client.1.smithi044.stdout:7/590: chown d0/d3/d8/d32/f38 937616182 1 2024-06-03T01:20:55.170 INFO:tasks.workunit.client.0.smithi032.stdout:2/118: dread d5/da/f10 [0,4194304] 0 2024-06-03T01:20:55.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/74: creat dc/d12/f18 x:0 0 0 2024-06-03T01:20:55.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/75: write dc/d12/f13 [1680467,21381] 0 2024-06-03T01:20:55.177 INFO:tasks.workunit.client.0.smithi032.stdout:9/195: rmdir d0 39 2024-06-03T01:20:55.177 INFO:tasks.workunit.client.1.smithi044.stdout:3/376: dwrite f9 [4194304,4194304] 0 2024-06-03T01:20:55.178 INFO:tasks.workunit.client.1.smithi044.stdout:3/377: stat df/d13/d1b/d4b/c51 0 2024-06-03T01:20:55.178 INFO:tasks.workunit.client.1.smithi044.stdout:4/514: mkdir d17/d63/dae 0 2024-06-03T01:20:55.178 INFO:tasks.workunit.client.1.smithi044.stdout:4/515: write d17/d1c/d7e/fa2 [687438,82240] 0 2024-06-03T01:20:55.178 INFO:tasks.workunit.client.1.smithi044.stdout:4/516: chown d17/d23/d28/f3e 105991820 1 2024-06-03T01:20:55.178 INFO:tasks.workunit.client.1.smithi044.stdout:4/517: chown d17/d23/d59/f82 38509478 1 2024-06-03T01:20:55.179 INFO:tasks.workunit.client.0.smithi032.stdout:0/58: mknod d1/da/c11 0 2024-06-03T01:20:55.180 INFO:tasks.workunit.client.1.smithi044.stdout:6/640: dwrite d2/d23/d30/d1e/d34/d39/f56 [0,4194304] 0 2024-06-03T01:20:55.180 INFO:tasks.workunit.client.1.smithi044.stdout:8/509: creat d5/d16/d81/f9c x:0 0 0 2024-06-03T01:20:55.180 INFO:tasks.workunit.client.0.smithi032.stdout:2/119: link d5/da/l1f d5/da/d1a/l27 0 2024-06-03T01:20:55.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/76: mknod dc/dd/c19 0 2024-06-03T01:20:55.181 INFO:tasks.workunit.client.1.smithi044.stdout:2/402: dwrite db/d1b/d3f/f61 [0,4194304] 0 2024-06-03T01:20:55.182 INFO:tasks.workunit.client.0.smithi032.stdout:5/76: dwrite da/dd/fe [0,4194304] 0 2024-06-03T01:20:55.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/585: mknod d3/d9/d36/ccc 0 2024-06-03T01:20:55.183 INFO:tasks.workunit.client.1.smithi044.stdout:0/586: dread - d3/d6b/d95/fb2 zero size 2024-06-03T01:20:55.183 INFO:tasks.workunit.client.0.smithi032.stdout:5/77: write da/dd/df/d10/f14 [4319057,69337] 0 2024-06-03T01:20:55.183 INFO:tasks.workunit.client.0.smithi032.stdout:9/196: rename d0/d11/f14 to d0/d11/f46 0 2024-06-03T01:20:55.184 INFO:tasks.workunit.client.0.smithi032.stdout:0/59: rmdir d1 39 2024-06-03T01:20:55.185 INFO:tasks.workunit.client.1.smithi044.stdout:4/518: write d17/d23/d35/f41 [1124815,109294] 0 2024-06-03T01:20:55.186 INFO:tasks.workunit.client.1.smithi044.stdout:4/519: chown d17/d23/d6a/d84/lab 5 1 2024-06-03T01:20:55.186 INFO:tasks.workunit.client.1.smithi044.stdout:3/378: mkdir df/d1e/d78 0 2024-06-03T01:20:55.186 INFO:tasks.workunit.client.1.smithi044.stdout:3/379: readlink df/l27 0 2024-06-03T01:20:55.186 INFO:tasks.workunit.client.0.smithi032.stdout:8/56: dwrite f9 [0,4194304] 0 2024-06-03T01:20:55.187 INFO:tasks.workunit.client.1.smithi044.stdout:9/383: dwrite d6/dd/d51/db/f15 [4194304,4194304] 0 2024-06-03T01:20:55.188 INFO:tasks.workunit.client.0.smithi032.stdout:5/78: mkdir da/dd/df/d10/d16/d18/d1a 0 2024-06-03T01:20:55.188 INFO:tasks.workunit.client.0.smithi032.stdout:5/79: read - da/f12 zero size 2024-06-03T01:20:55.188 INFO:tasks.workunit.client.1.smithi044.stdout:6/641: mknod d2/d23/d30/d1e/d34/d39/d9b/db4/dba/ce2 0 2024-06-03T01:20:55.190 INFO:tasks.workunit.client.0.smithi032.stdout:8/57: dread f8 [0,4194304] 0 2024-06-03T01:20:55.192 INFO:tasks.workunit.client.1.smithi044.stdout:8/510: mknod d5/d55/d56/d5c/c9d 0 2024-06-03T01:20:55.192 INFO:tasks.workunit.client.1.smithi044.stdout:8/511: dread - d5/d16/d1d/d6b/f80 zero size 2024-06-03T01:20:55.192 INFO:tasks.workunit.client.1.smithi044.stdout:8/512: chown d5/d55/d74/f78 57379370 1 2024-06-03T01:20:55.197 INFO:tasks.workunit.client.0.smithi032.stdout:2/120: write f3 [3883445,120340] 0 2024-06-03T01:20:55.198 INFO:tasks.workunit.client.1.smithi044.stdout:3/380: dread df/d1e/f45 [0,4194304] 0 2024-06-03T01:20:55.200 INFO:tasks.workunit.client.1.smithi044.stdout:0/587: creat d3/d5c/fcd x:0 0 0 2024-06-03T01:20:55.201 INFO:tasks.workunit.client.0.smithi032.stdout:7/77: dread dc/d12/f13 [0,4194304] 0 2024-06-03T01:20:55.204 INFO:tasks.workunit.client.1.smithi044.stdout:9/384: link d6/dd/f17 d6/dd/d51/d16/d5f/f7c 0 2024-06-03T01:20:55.206 INFO:tasks.workunit.client.1.smithi044.stdout:8/513: dread d5/d20/d2b/f48 [0,4194304] 0 2024-06-03T01:20:55.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/642: truncate d2/d23/d30/d1e/d38/d6c/d5c/fab 158163 0 2024-06-03T01:20:55.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/643: creat d2/d23/d30/d1e/d34/d39/d7d/fe3 x:0 0 0 2024-06-03T01:20:55.207 INFO:tasks.workunit.client.1.smithi044.stdout:6/644: write d2/d23/d30/f7f [290131,5692] 0 2024-06-03T01:20:55.208 INFO:tasks.workunit.client.1.smithi044.stdout:9/385: dread d6/f12 [0,4194304] 0 2024-06-03T01:20:55.210 INFO:tasks.workunit.client.1.smithi044.stdout:3/381: unlink df/d13/d39/d3d/d43/f6e 0 2024-06-03T01:20:55.210 INFO:tasks.workunit.client.1.smithi044.stdout:4/520: dwrite d17/d23/d6a/f9a [0,4194304] 0 2024-06-03T01:20:55.211 INFO:tasks.workunit.client.0.smithi032.stdout:5/80: symlink da/dd/df/l1b 0 2024-06-03T01:20:55.214 INFO:tasks.workunit.client.1.smithi044.stdout:0/588: symlink d3/d6b/d95/dae/lce 0 2024-06-03T01:20:55.215 INFO:tasks.workunit.client.1.smithi044.stdout:0/589: truncate d3/d9/d55/d5a/d29/f4f 609581 0 2024-06-03T01:20:55.216 INFO:tasks.workunit.client.0.smithi032.stdout:9/197: dwrite d0/f1 [0,4194304] 0 2024-06-03T01:20:55.216 INFO:tasks.workunit.client.0.smithi032.stdout:9/198: dread - d0/d23/d24/f3b zero size 2024-06-03T01:20:55.216 INFO:tasks.workunit.client.0.smithi032.stdout:9/199: write d0/d11/f31 [557044,93781] 0 2024-06-03T01:20:55.216 INFO:tasks.workunit.client.0.smithi032.stdout:9/200: readlink d0/d11/l13 0 2024-06-03T01:20:55.216 INFO:tasks.workunit.client.0.smithi032.stdout:8/58: link c5 dc/cf 0 2024-06-03T01:20:55.218 INFO:tasks.workunit.client.1.smithi044.stdout:6/645: symlink d2/d23/d30/d1e/d34/d39/da8/le4 0 2024-06-03T01:20:55.218 INFO:tasks.workunit.client.1.smithi044.stdout:6/646: chown d2/d23/d30/ddf 14461 1 2024-06-03T01:20:55.219 INFO:tasks.workunit.client.0.smithi032.stdout:2/121: truncate d5/da/f18 3619370 0 2024-06-03T01:20:55.221 INFO:tasks.workunit.client.1.smithi044.stdout:0/590: rmdir d3/d6b/d95/dae 39 2024-06-03T01:20:55.221 INFO:tasks.workunit.client.1.smithi044.stdout:0/591: chown d3/da6/d20/d25/f65 5 1 2024-06-03T01:20:55.224 INFO:tasks.workunit.client.1.smithi044.stdout:9/386: dread d6/dd/d51/d16/f37 [0,4194304] 0 2024-06-03T01:20:55.224 INFO:tasks.workunit.client.1.smithi044.stdout:6/647: rename d2/d23/f18 to d2/d23/d30/d1e/d38/d77/fe5 0 2024-06-03T01:20:55.225 INFO:tasks.workunit.client.1.smithi044.stdout:6/648: read - d2/d23/d55/d5f/d53/f7e zero size 2024-06-03T01:20:55.225 INFO:tasks.workunit.client.1.smithi044.stdout:4/521: write d17/d1c/d75/f5d [3862596,85175] 0 2024-06-03T01:20:55.225 INFO:tasks.workunit.client.1.smithi044.stdout:2/403: dwrite db/f19 [0,4194304] 0 2024-06-03T01:20:55.227 INFO:tasks.workunit.client.1.smithi044.stdout:0/592: mknod d3/d6b/ccf 0 2024-06-03T01:20:55.229 INFO:tasks.workunit.client.0.smithi032.stdout:7/78: mknod dc/d12/c1a 0 2024-06-03T01:20:55.229 INFO:tasks.workunit.client.1.smithi044.stdout:9/387: unlink d6/dd/d51/f9 0 2024-06-03T01:20:55.231 INFO:tasks.workunit.client.0.smithi032.stdout:0/60: symlink d1/l12 0 2024-06-03T01:20:55.231 INFO:tasks.workunit.client.0.smithi032.stdout:0/61: write d1/fc [435641,112044] 0 2024-06-03T01:20:55.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/649: creat d2/d23/d44/d4d/fe6 x:0 0 0 2024-06-03T01:20:55.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/650: write d2/d23/d55/d5f/f63 [4461348,76329] 0 2024-06-03T01:20:55.233 INFO:tasks.workunit.client.1.smithi044.stdout:6/651: write d2/d23/d30/d1e/d38/d6c/f84 [163085,83037] 0 2024-06-03T01:20:55.233 INFO:tasks.workunit.client.1.smithi044.stdout:6/652: truncate d2/d23/d30/f1f 8527048 0 2024-06-03T01:20:55.235 INFO:tasks.workunit.client.1.smithi044.stdout:2/404: symlink db/d1b/d2a/d2f/l87 0 2024-06-03T01:20:55.238 INFO:tasks.workunit.client.1.smithi044.stdout:0/593: link d3/d9/d55/d73/f77 d3/da6/d56/fd0 0 2024-06-03T01:20:55.240 INFO:tasks.workunit.client.1.smithi044.stdout:9/388: mknod d6/dd/d51/d16/d4d/c7d 0 2024-06-03T01:20:55.241 INFO:tasks.workunit.client.1.smithi044.stdout:6/653: mknod d2/d23/d30/ce7 0 2024-06-03T01:20:55.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/654: truncate d2/d23/d30/d1e/d38/d6c/f12 4934752 0 2024-06-03T01:20:55.242 INFO:tasks.workunit.client.1.smithi044.stdout:1/551: sync 2024-06-03T01:20:55.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/500: sync 2024-06-03T01:20:55.242 INFO:tasks.workunit.client.1.smithi044.stdout:1/552: read d0/d3/d20/d7/d4e/f60 [460963,60249] 0 2024-06-03T01:20:55.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/501: fsync d2/d11/d1e/d2f/f64 0 2024-06-03T01:20:55.243 INFO:tasks.workunit.client.0.smithi032.stdout:5/81: dwrite da/f12 [0,4194304] 0 2024-06-03T01:20:55.243 INFO:tasks.workunit.client.1.smithi044.stdout:3/382: dwrite df/d13/d1b/f20 [0,4194304] 0 2024-06-03T01:20:55.244 INFO:tasks.workunit.client.0.smithi032.stdout:9/201: link d0/f19 d0/d11/d1d/f47 0 2024-06-03T01:20:55.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/522: dwrite d17/d1c/d75/f5c [0,4194304] 0 2024-06-03T01:20:55.245 INFO:tasks.workunit.client.0.smithi032.stdout:8/59: creat dc/de/f10 x:0 0 0 2024-06-03T01:20:55.246 INFO:tasks.workunit.client.1.smithi044.stdout:0/594: mkdir d3/d6b/d95/dae/dd1 0 2024-06-03T01:20:55.246 INFO:tasks.workunit.client.1.smithi044.stdout:9/389: creat d6/dd/d51/d16/d61/f7e x:0 0 0 2024-06-03T01:20:55.246 INFO:tasks.workunit.client.1.smithi044.stdout:9/390: stat d6/dd/l1a 0 2024-06-03T01:20:55.247 INFO:tasks.workunit.client.1.smithi044.stdout:6/655: symlink d2/d23/d30/d1e/d34/d39/le8 0 2024-06-03T01:20:55.248 INFO:tasks.workunit.client.0.smithi032.stdout:2/122: rename d5/da/c16 to d5/da/c28 0 2024-06-03T01:20:55.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/553: symlink d0/d3/d20/d40/d45/d4b/d80/lc1 0 2024-06-03T01:20:55.250 INFO:tasks.workunit.client.0.smithi032.stdout:9/202: write d0/f6 [2417799,3628] 0 2024-06-03T01:20:55.250 INFO:tasks.workunit.client.1.smithi044.stdout:5/502: stat d2/d11/d1e/d3b/l5a 0 2024-06-03T01:20:55.252 INFO:tasks.workunit.client.0.smithi032.stdout:0/62: mknod d1/da/d10/c13 0 2024-06-03T01:20:55.254 INFO:tasks.workunit.client.0.smithi032.stdout:0/63: write d1/f6 [56287,117071] 0 2024-06-03T01:20:55.256 INFO:tasks.workunit.client.0.smithi032.stdout:5/82: link da/dd/df/d10/f14 da/dd/df/f1c 0 2024-06-03T01:20:55.259 INFO:tasks.workunit.client.1.smithi044.stdout:2/405: dwrite db/d1b/d3f/f4a [0,4194304] 0 2024-06-03T01:20:55.259 INFO:tasks.workunit.client.1.smithi044.stdout:2/406: chown db/d15/d49/d7b 1031858 1 2024-06-03T01:20:55.259 INFO:tasks.workunit.client.1.smithi044.stdout:2/407: stat db/f48 0 2024-06-03T01:20:55.261 INFO:tasks.workunit.client.1.smithi044.stdout:4/523: read d17/d1c/d75/f5d [6451817,2503] 0 2024-06-03T01:20:55.261 INFO:tasks.workunit.client.0.smithi032.stdout:8/60: write f2 [4372111,57258] 0 2024-06-03T01:20:55.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/595: link d3/d9/f4d d3/d6b/d95/fd2 0 2024-06-03T01:20:55.263 INFO:tasks.workunit.client.1.smithi044.stdout:2/408: dread db/d1b/d2a/f2b [0,4194304] 0 2024-06-03T01:20:55.264 INFO:tasks.workunit.client.0.smithi032.stdout:5/83: getdents da/dd/df/d10 0 2024-06-03T01:20:55.267 INFO:tasks.workunit.client.1.smithi044.stdout:9/391: rename d6/dd/d51/db/l23 to d6/dd/d51/d46/l7f 0 2024-06-03T01:20:55.267 INFO:tasks.workunit.client.1.smithi044.stdout:9/392: chown d6/dd/d51/d16/d27 2831868 1 2024-06-03T01:20:55.270 INFO:tasks.workunit.client.1.smithi044.stdout:6/656: getdents d2/d23/d30/d1e/d34/d39/d9b 0 2024-06-03T01:20:55.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/383: dwrite df/d13/d39/d3d/f44 [0,4194304] 0 2024-06-03T01:20:55.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/384: fdatasync df/d13/d1b/d4b/f4f 0 2024-06-03T01:20:55.270 INFO:tasks.workunit.client.1.smithi044.stdout:6/657: chown d2/d23/d30/d1e/d38/d6c/f21 252689 1 2024-06-03T01:20:55.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/385: creat df/d1e/f79 x:0 0 0 2024-06-03T01:20:55.272 INFO:tasks.workunit.client.0.smithi032.stdout:2/123: dwrite d5/da/f1e [0,4194304] 0 2024-06-03T01:20:55.272 INFO:tasks.workunit.client.0.smithi032.stdout:2/124: write d5/fe [250484,110704] 0 2024-06-03T01:20:55.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/554: mkdir d0/d66/dc2 0 2024-06-03T01:20:55.272 INFO:tasks.workunit.client.1.smithi044.stdout:1/555: chown d0/d3/d20/d40/d45/c7c 29 1 2024-06-03T01:20:55.273 INFO:tasks.workunit.client.1.smithi044.stdout:1/556: creat d0/d3/d20/d40/d45/d4b/d80/fc3 x:0 0 0 2024-06-03T01:20:55.273 INFO:tasks.workunit.client.0.smithi032.stdout:9/203: dread d0/fa [0,4194304] 0 2024-06-03T01:20:55.273 INFO:tasks.workunit.client.0.smithi032.stdout:9/204: creat d0/d11/f48 x:0 0 0 2024-06-03T01:20:55.274 INFO:tasks.workunit.client.0.smithi032.stdout:9/205: write d0/d11/d1d/f1e [206728,41464] 0 2024-06-03T01:20:55.274 INFO:tasks.workunit.client.0.smithi032.stdout:9/206: chown d0/d11/f31 1667 1 2024-06-03T01:20:55.276 INFO:tasks.workunit.client.0.smithi032.stdout:0/64: dwrite d1/f9 [4194304,4194304] 0 2024-06-03T01:20:55.277 INFO:tasks.workunit.client.1.smithi044.stdout:5/503: unlink d2/f8 0 2024-06-03T01:20:55.277 INFO:tasks.workunit.client.1.smithi044.stdout:5/504: readlink d2/d11/l52 0 2024-06-03T01:20:55.280 INFO:tasks.workunit.client.1.smithi044.stdout:6/658: write d2/d23/d30/f1f [7619256,34697] 0 2024-06-03T01:20:55.280 INFO:tasks.workunit.client.0.smithi032.stdout:9/207: dread d0/ff [0,4194304] 0 2024-06-03T01:20:55.280 INFO:tasks.workunit.client.0.smithi032.stdout:9/208: fsync d0/d23/d24/d26/f39 0 2024-06-03T01:20:55.280 INFO:tasks.workunit.client.0.smithi032.stdout:9/209: write d0/f5 [2301516,44984] 0 2024-06-03T01:20:55.281 INFO:tasks.workunit.client.0.smithi032.stdout:9/210: read d0/f19 [79449,13432] 0 2024-06-03T01:20:55.285 INFO:tasks.workunit.client.0.smithi032.stdout:9/211: read d0/d23/d24/f27 [34988,89330] 0 2024-06-03T01:20:55.285 INFO:tasks.workunit.client.0.smithi032.stdout:7/79: truncate dc/dd/f11 2693391 0 2024-06-03T01:20:55.287 INFO:tasks.workunit.client.1.smithi044.stdout:4/524: mknod d17/d23/caf 0 2024-06-03T01:20:55.288 INFO:tasks.workunit.client.0.smithi032.stdout:9/212: link d0/d11/f48 d0/d11/d1d/f49 0 2024-06-03T01:20:55.289 INFO:tasks.workunit.client.0.smithi032.stdout:6/88: sync 2024-06-03T01:20:55.290 INFO:tasks.workunit.client.0.smithi032.stdout:3/72: sync 2024-06-03T01:20:55.290 INFO:tasks.workunit.client.0.smithi032.stdout:3/73: chown f9 5829637 1 2024-06-03T01:20:55.290 INFO:tasks.workunit.client.0.smithi032.stdout:3/74: creat de/f16 x:0 0 0 2024-06-03T01:20:55.290 INFO:tasks.workunit.client.0.smithi032.stdout:3/75: write f8 [1505481,16759] 0 2024-06-03T01:20:55.291 INFO:tasks.workunit.client.1.smithi044.stdout:0/596: symlink d3/d9/d36/d82/ld3 0 2024-06-03T01:20:55.291 INFO:tasks.workunit.client.0.smithi032.stdout:4/62: sync 2024-06-03T01:20:55.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/25: sync 2024-06-03T01:20:55.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/26: dread - f5 zero size 2024-06-03T01:20:55.293 INFO:tasks.workunit.client.1.smithi044.stdout:2/409: creat db/d15/d49/d7b/f88 x:0 0 0 2024-06-03T01:20:55.293 INFO:tasks.workunit.client.0.smithi032.stdout:8/61: getdents dc/de 0 2024-06-03T01:20:55.295 INFO:tasks.workunit.client.1.smithi044.stdout:9/393: creat d6/dd/d51/d16/d5f/f80 x:0 0 0 2024-06-03T01:20:55.295 INFO:tasks.workunit.client.0.smithi032.stdout:6/89: mknod d8/d18/c1b 0 2024-06-03T01:20:55.298 INFO:tasks.workunit.client.0.smithi032.stdout:4/63: dread d4/f6 [0,4194304] 0 2024-06-03T01:20:55.299 INFO:tasks.workunit.client.0.smithi032.stdout:1/27: symlink l7 0 2024-06-03T01:20:55.300 INFO:tasks.workunit.client.0.smithi032.stdout:8/62: creat dc/de/f11 x:0 0 0 2024-06-03T01:20:55.302 INFO:tasks.workunit.client.1.smithi044.stdout:2/410: write db/d1b/d3f/f61 [345474,3702] 0 2024-06-03T01:20:55.303 INFO:tasks.workunit.client.0.smithi032.stdout:4/64: symlink d4/db/da/l17 0 2024-06-03T01:20:55.304 INFO:tasks.workunit.client.0.smithi032.stdout:1/28: rename c6 to c8 0 2024-06-03T01:20:55.304 INFO:tasks.workunit.client.1.smithi044.stdout:3/386: symlink df/d13/d39/d3d/d43/d72/l7a 0 2024-06-03T01:20:55.304 INFO:tasks.workunit.client.1.smithi044.stdout:3/387: stat df/d13/d1b 0 2024-06-03T01:20:55.308 INFO:tasks.workunit.client.0.smithi032.stdout:2/125: rename f3 to d5/da/f29 0 2024-06-03T01:20:55.310 INFO:tasks.workunit.client.0.smithi032.stdout:2/126: mkdir d5/da/d1a/d2a 0 2024-06-03T01:20:55.310 INFO:tasks.workunit.client.0.smithi032.stdout:2/127: dread - d5/da/f20 zero size 2024-06-03T01:20:55.310 INFO:tasks.workunit.client.0.smithi032.stdout:2/128: readlink d5/l14 0 2024-06-03T01:20:55.311 INFO:tasks.workunit.client.1.smithi044.stdout:6/659: dwrite d2/d23/d30/d1e/f47 [0,4194304] 0 2024-06-03T01:20:55.312 INFO:tasks.workunit.client.0.smithi032.stdout:8/63: write f2 [2900888,116437] 0 2024-06-03T01:20:55.314 INFO:tasks.workunit.client.0.smithi032.stdout:7/80: truncate f8 3573262 0 2024-06-03T01:20:55.314 INFO:tasks.workunit.client.0.smithi032.stdout:7/81: fsync fa 0 2024-06-03T01:20:55.316 INFO:tasks.workunit.client.0.smithi032.stdout:6/90: dread d8/f9 [0,4194304] 0 2024-06-03T01:20:55.318 INFO:tasks.workunit.client.1.smithi044.stdout:1/557: mknod d0/d3/d20/d7/d13/d7a/cc4 0 2024-06-03T01:20:55.319 INFO:tasks.workunit.client.0.smithi032.stdout:7/82: truncate dc/f16 1885329 0 2024-06-03T01:20:55.319 INFO:tasks.workunit.client.0.smithi032.stdout:7/83: write dc/f15 [885094,58332] 0 2024-06-03T01:20:55.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/505: symlink d2/d11/d1e/la0 0 2024-06-03T01:20:55.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/506: fdatasync d2/d11/d12/f1b 0 2024-06-03T01:20:55.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/507: getdents d2/d11/d12/d2b/d4a 0 2024-06-03T01:20:55.322 INFO:tasks.workunit.client.0.smithi032.stdout:0/65: dwrite d1/f8 [0,4194304] 0 2024-06-03T01:20:55.322 INFO:tasks.workunit.client.0.smithi032.stdout:2/129: read d5/da/f29 [6874771,61959] 0 2024-06-03T01:20:55.323 INFO:tasks.workunit.client.0.smithi032.stdout:7/84: mkdir dc/d12/d1b 0 2024-06-03T01:20:55.323 INFO:tasks.workunit.client.0.smithi032.stdout:7/85: chown dc/d12/f18 2491 1 2024-06-03T01:20:55.324 INFO:tasks.workunit.client.1.smithi044.stdout:4/525: rename d17/d23/d28/c6c to d17/d23/d28/d2b/d8e/cb0 0 2024-06-03T01:20:55.324 INFO:tasks.workunit.client.1.smithi044.stdout:4/526: chown d17 46429 1 2024-06-03T01:20:55.324 INFO:tasks.workunit.client.1.smithi044.stdout:4/527: read d17/d23/d35/f3c [356381,118750] 0 2024-06-03T01:20:55.324 INFO:tasks.workunit.client.1.smithi044.stdout:4/528: chown d17/d23/d28/d2b/l29 7258 1 2024-06-03T01:20:55.326 INFO:tasks.workunit.client.0.smithi032.stdout:7/86: symlink dc/d12/d1b/l1c 0 2024-06-03T01:20:55.329 INFO:tasks.workunit.client.0.smithi032.stdout:9/213: dwrite d0/d11/d1d/f47 [0,4194304] 0 2024-06-03T01:20:55.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/597: link d3/d9/d55/d73/f77 d3/d9/d55/fd4 0 2024-06-03T01:20:55.331 INFO:tasks.workunit.client.0.smithi032.stdout:7/87: creat dc/d12/d1b/f1d x:0 0 0 2024-06-03T01:20:55.331 INFO:tasks.workunit.client.0.smithi032.stdout:7/88: readlink dc/d12/d1b/l1c 0 2024-06-03T01:20:55.332 INFO:tasks.workunit.client.1.smithi044.stdout:9/394: rmdir d6/dd/d51/d16/d4d/d55/d68/d74 0 2024-06-03T01:20:55.332 INFO:tasks.workunit.client.1.smithi044.stdout:9/395: dread - d6/dd/d51/d16/d4d/d55/f5b zero size 2024-06-03T01:20:55.334 INFO:tasks.workunit.client.1.smithi044.stdout:3/388: mkdir df/d13/d1b/d56/d7b 0 2024-06-03T01:20:55.334 INFO:tasks.workunit.client.0.smithi032.stdout:4/65: dwrite d4/db/fc [4194304,4194304] 0 2024-06-03T01:20:55.336 INFO:tasks.workunit.client.0.smithi032.stdout:9/214: mkdir d0/d11/d1d/d4a 0 2024-06-03T01:20:55.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/66: dread d1/f9 [4194304,4194304] 0 2024-06-03T01:20:55.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/67: readlink d1/l3 0 2024-06-03T01:20:55.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/68: chown d1/f6 980 1 2024-06-03T01:20:55.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/660: creat d2/d23/d30/d1e/d34/d39/d64/fe9 x:0 0 0 2024-06-03T01:20:55.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/661: getdents d2/d23/d30/d1e/d34/d39/d64/dc7 0 2024-06-03T01:20:55.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/662: readlink d2/d23/d30/d1e/d34/d39/da8/le4 0 2024-06-03T01:20:55.339 INFO:tasks.workunit.client.0.smithi032.stdout:4/66: stat d4/db/le 0 2024-06-03T01:20:55.343 INFO:tasks.workunit.client.0.smithi032.stdout:0/69: symlink d1/da/l14 0 2024-06-03T01:20:55.344 INFO:tasks.workunit.client.1.smithi044.stdout:4/529: symlink d17/d23/d28/lb1 0 2024-06-03T01:20:55.344 INFO:tasks.workunit.client.1.smithi044.stdout:4/530: chown d17/d23/d28/l38 12396299 1 2024-06-03T01:20:55.344 INFO:tasks.workunit.client.1.smithi044.stdout:0/598: dread d3/da6/fac [0,4194304] 0 2024-06-03T01:20:55.345 INFO:tasks.workunit.client.0.smithi032.stdout:2/130: dread d5/fd [0,4194304] 0 2024-06-03T01:20:55.346 INFO:tasks.workunit.client.1.smithi044.stdout:3/389: mkdir df/d13/d39/d3d/d43/d71/d7c 0 2024-06-03T01:20:55.347 INFO:tasks.workunit.client.0.smithi032.stdout:4/67: symlink d4/db/l18 0 2024-06-03T01:20:55.347 INFO:tasks.workunit.client.0.smithi032.stdout:4/68: creat d4/f19 x:0 0 0 2024-06-03T01:20:55.347 INFO:tasks.workunit.client.0.smithi032.stdout:4/69: stat d4/f6 0 2024-06-03T01:20:55.348 INFO:tasks.workunit.client.1.smithi044.stdout:4/531: mknod d17/d23/d6a/cb2 0 2024-06-03T01:20:55.349 INFO:tasks.workunit.client.1.smithi044.stdout:0/599: rmdir d3/da6/dc1 39 2024-06-03T01:20:55.349 INFO:tasks.workunit.client.0.smithi032.stdout:2/131: link d5/da/d1a/c21 d5/da/d1a/d2a/c2b 0 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/70: creat d4/db/f1a x:0 0 0 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/71: fdatasync d4/db/f1a 0 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/72: creat d4/db/da/f1b x:0 0 0 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/73: readlink d4/l5 0 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/74: chown d4/db/da/c12 0 1 2024-06-03T01:20:55.354 INFO:tasks.workunit.client.0.smithi032.stdout:4/75: truncate d4/db/da/f1b 409865 0 2024-06-03T01:20:55.355 INFO:tasks.workunit.client.1.smithi044.stdout:0/600: symlink d3/da6/d20/d25/ld5 0 2024-06-03T01:20:55.355 INFO:tasks.workunit.client.1.smithi044.stdout:0/601: creat d3/d5c/fd6 x:0 0 0 2024-06-03T01:20:55.357 INFO:tasks.workunit.client.1.smithi044.stdout:4/532: rename d17/d23/d58 to d17/d1c/d75/d7a/db3 0 2024-06-03T01:20:55.357 INFO:tasks.workunit.client.1.smithi044.stdout:4/533: fdatasync d17/d23/d8d/f9e 0 2024-06-03T01:20:55.358 INFO:tasks.workunit.client.0.smithi032.stdout:7/89: dread dc/d12/f13 [0,4194304] 0 2024-06-03T01:20:55.358 INFO:tasks.workunit.client.0.smithi032.stdout:2/132: rename d5/fd to d5/da/f2c 0 2024-06-03T01:20:55.360 INFO:tasks.workunit.client.1.smithi044.stdout:3/390: dread df/d13/d1b/f20 [0,4194304] 0 2024-06-03T01:20:55.360 INFO:tasks.workunit.client.1.smithi044.stdout:0/602: rename d3/da6/d21/f9b to d3/d9/d36/fd7 0 2024-06-03T01:20:55.361 INFO:tasks.workunit.client.0.smithi032.stdout:4/76: link d4/f19 d4/db/f1c 0 2024-06-03T01:20:55.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/90: dread f8 [0,4194304] 0 2024-06-03T01:20:55.363 INFO:tasks.workunit.client.0.smithi032.stdout:4/77: mkdir d4/db/da/d1d 0 2024-06-03T01:20:55.363 INFO:tasks.workunit.client.1.smithi044.stdout:4/534: link d17/d23/fa5 d17/d23/d35/fb4 0 2024-06-03T01:20:55.363 INFO:tasks.workunit.client.1.smithi044.stdout:4/535: dread - d17/f69 zero size 2024-06-03T01:20:55.364 INFO:tasks.workunit.client.1.smithi044.stdout:0/603: dread d3/da6/d56/f5d [0,4194304] 0 2024-06-03T01:20:55.364 INFO:tasks.workunit.client.1.smithi044.stdout:4/536: stat d17/d23/d59/l5b 0 2024-06-03T01:20:55.364 INFO:tasks.workunit.client.1.smithi044.stdout:4/537: write d17/d23/fa5 [626337,23731] 0 2024-06-03T01:20:55.365 INFO:tasks.workunit.client.0.smithi032.stdout:4/78: mknod d4/c1e 0 2024-06-03T01:20:55.365 INFO:tasks.workunit.client.0.smithi032.stdout:4/79: readlink d4/db/da/l17 0 2024-06-03T01:20:55.365 INFO:tasks.workunit.client.0.smithi032.stdout:4/80: write d4/db/f1c [402307,32115] 0 2024-06-03T01:20:55.365 INFO:tasks.workunit.client.1.smithi044.stdout:0/604: mkdir d3/da6/d21/d94/da3/dd8 0 2024-06-03T01:20:55.366 INFO:tasks.workunit.client.0.smithi032.stdout:4/81: unlink d4/db/da/c10 0 2024-06-03T01:20:55.366 INFO:tasks.workunit.client.0.smithi032.stdout:4/82: write d4/db/f11 [1961363,108675] 0 2024-06-03T01:20:55.366 INFO:tasks.workunit.client.0.smithi032.stdout:4/83: chown d4/l5 128744919 1 2024-06-03T01:20:55.368 INFO:tasks.workunit.client.1.smithi044.stdout:4/538: link d17/d23/f9b d17/d23/d59/fb5 0 2024-06-03T01:20:55.368 INFO:tasks.workunit.client.0.smithi032.stdout:9/215: write d0/d11/d1d/f47 [225882,38518] 0 2024-06-03T01:20:55.368 INFO:tasks.workunit.client.1.smithi044.stdout:0/605: mknod d3/d9/d55/d73/da0/cd9 0 2024-06-03T01:20:55.369 INFO:tasks.workunit.client.1.smithi044.stdout:0/606: getdents d3/da6/d21/d94/da3/dd8 0 2024-06-03T01:20:55.369 INFO:tasks.workunit.client.0.smithi032.stdout:2/133: dread d5/f19 [0,4194304] 0 2024-06-03T01:20:55.370 INFO:tasks.workunit.client.1.smithi044.stdout:4/539: mknod d17/d1c/d9c/cb6 0 2024-06-03T01:20:55.370 INFO:tasks.workunit.client.1.smithi044.stdout:4/540: creat d17/d1c/fb7 x:0 0 0 2024-06-03T01:20:55.370 INFO:tasks.workunit.client.1.smithi044.stdout:0/607: mknod d3/d6b/cda 0 2024-06-03T01:20:55.372 INFO:tasks.workunit.client.0.smithi032.stdout:9/216: getdents d0/d23/d24/d26 0 2024-06-03T01:20:55.374 INFO:tasks.workunit.client.1.smithi044.stdout:0/608: mknod d3/da6/dc1/cdb 0 2024-06-03T01:20:55.374 INFO:tasks.workunit.client.1.smithi044.stdout:0/609: creat d3/da6/d20/fdc x:0 0 0 2024-06-03T01:20:55.374 INFO:tasks.workunit.client.0.smithi032.stdout:9/217: link d0/c4 d0/d11/d1d/d4a/c4b 0 2024-06-03T01:20:55.376 INFO:tasks.workunit.client.0.smithi032.stdout:9/218: truncate d0/f6 1246209 0 2024-06-03T01:20:55.378 INFO:tasks.workunit.client.0.smithi032.stdout:7/91: dread f6 [0,4194304] 0 2024-06-03T01:20:55.378 INFO:tasks.workunit.client.1.smithi044.stdout:0/610: link d3/d6b/d95/dae/cb9 d3/d6b/d95/cdd 0 2024-06-03T01:20:55.379 INFO:tasks.workunit.client.1.smithi044.stdout:0/611: mkdir d3/d9/d55/d5a/dde 0 2024-06-03T01:20:55.379 INFO:tasks.workunit.client.1.smithi044.stdout:0/612: write d3/da6/d20/d25/f64 [729852,52374] 0 2024-06-03T01:20:55.380 INFO:tasks.workunit.client.0.smithi032.stdout:7/92: rename f8 to dc/f1e 0 2024-06-03T01:20:55.381 INFO:tasks.workunit.client.1.smithi044.stdout:3/391: dwrite df/f58 [0,4194304] 0 2024-06-03T01:20:55.381 INFO:tasks.workunit.client.1.smithi044.stdout:4/541: write d17/f31 [2905580,52128] 0 2024-06-03T01:20:55.381 INFO:tasks.workunit.client.1.smithi044.stdout:0/613: link d3/d9/d55/d5a/d29/d58/l9e d3/da6/d21/d94/da3/dad/ldf 0 2024-06-03T01:20:55.381 INFO:tasks.workunit.client.1.smithi044.stdout:0/614: chown d3/la 9509 1 2024-06-03T01:20:55.382 INFO:tasks.workunit.client.0.smithi032.stdout:8/64: dwrite f1 [0,4194304] 0 2024-06-03T01:20:55.383 INFO:tasks.workunit.client.1.smithi044.stdout:1/558: dwrite d0/d3/d20/d7/d13/f24 [0,4194304] 0 2024-06-03T01:20:55.383 INFO:tasks.workunit.client.1.smithi044.stdout:1/559: write d0/f3f [501890,65472] 0 2024-06-03T01:20:55.383 INFO:tasks.workunit.client.0.smithi032.stdout:0/70: dwrite d1/f9 [0,4194304] 0 2024-06-03T01:20:55.384 INFO:tasks.workunit.client.1.smithi044.stdout:6/663: dwrite d2/fe [8388608,4194304] 0 2024-06-03T01:20:55.384 INFO:tasks.workunit.client.1.smithi044.stdout:5/508: dwrite d2/f2e [0,4194304] 0 2024-06-03T01:20:55.384 INFO:tasks.workunit.client.0.smithi032.stdout:3/76: dwrite f8 [0,4194304] 0 2024-06-03T01:20:55.386 INFO:tasks.workunit.client.0.smithi032.stdout:0/71: dread d1/fc [0,4194304] 0 2024-06-03T01:20:55.388 INFO:tasks.workunit.client.1.smithi044.stdout:0/615: truncate d3/d9/f44 284594 0 2024-06-03T01:20:55.388 INFO:tasks.workunit.client.0.smithi032.stdout:9/219: dread d0/ff [0,4194304] 0 2024-06-03T01:20:55.388 INFO:tasks.workunit.client.0.smithi032.stdout:6/91: dwrite f4 [0,4194304] 0 2024-06-03T01:20:55.388 INFO:tasks.workunit.client.0.smithi032.stdout:6/92: stat d8/d18/d19 0 2024-06-03T01:20:55.389 INFO:tasks.workunit.client.1.smithi044.stdout:1/560: creat d0/d66/dbe/fc5 x:0 0 0 2024-06-03T01:20:55.389 INFO:tasks.workunit.client.1.smithi044.stdout:1/561: chown d0/d3/d20/d40/d45/d6f/f98 10040 1 2024-06-03T01:20:55.389 INFO:tasks.workunit.client.1.smithi044.stdout:1/562: chown d0/d3/d20/d7/f36 1317140354 1 2024-06-03T01:20:55.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/563: dread - d0/d3/d20/d7/d13/f96 zero size 2024-06-03T01:20:55.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/77: mkdir de/d17 0 2024-06-03T01:20:55.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/65: mknod dc/de/c12 0 2024-06-03T01:20:55.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/66: creat dc/f13 x:0 0 0 2024-06-03T01:20:55.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/67: chown lb 199 1 2024-06-03T01:20:55.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/616: dread d3/da6/d20/d25/f4b [0,4194304] 0 2024-06-03T01:20:55.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/617: truncate d3/d6b/d95/fb2 886008 0 2024-06-03T01:20:55.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/618: creat d3/d9/d36/fe0 x:0 0 0 2024-06-03T01:20:55.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/619: write d3/d9/f4d [852111,42750] 0 2024-06-03T01:20:55.400 INFO:tasks.workunit.client.0.smithi032.stdout:8/68: write f1 [473296,42066] 0 2024-06-03T01:20:55.402 INFO:tasks.workunit.client.0.smithi032.stdout:6/93: read d8/f9 [3035749,123394] 0 2024-06-03T01:20:55.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/72: write d1/f9 [1500353,104003] 0 2024-06-03T01:20:55.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/73: fsync d1/fc 0 2024-06-03T01:20:55.410 INFO:tasks.workunit.client.0.smithi032.stdout:3/78: dwrite fd [0,4194304] 0 2024-06-03T01:20:55.412 INFO:tasks.workunit.client.1.smithi044.stdout:5/509: mknod d2/d11/d12/d2b/d4a/ca1 0 2024-06-03T01:20:55.413 INFO:tasks.workunit.client.0.smithi032.stdout:9/220: link d0/fd d0/d23/d25/f4c 0 2024-06-03T01:20:55.414 INFO:tasks.workunit.client.1.smithi044.stdout:3/392: truncate f9 5016470 0 2024-06-03T01:20:55.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/69: mkdir dc/d14 0 2024-06-03T01:20:55.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/70: dread - dc/de/f11 zero size 2024-06-03T01:20:55.419 INFO:tasks.workunit.client.0.smithi032.stdout:6/94: mknod d8/d18/d19/c1c 0 2024-06-03T01:20:55.421 INFO:tasks.workunit.client.0.smithi032.stdout:0/74: unlink d1/da/d10/c13 0 2024-06-03T01:20:55.423 INFO:tasks.workunit.client.0.smithi032.stdout:3/79: creat de/d17/f18 x:0 0 0 2024-06-03T01:20:55.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/95: rmdir d8/d18 39 2024-06-03T01:20:55.427 INFO:tasks.workunit.client.0.smithi032.stdout:0/75: symlink d1/da/d10/l15 0 2024-06-03T01:20:55.429 INFO:tasks.workunit.client.1.smithi044.stdout:0/620: mknod d3/da6/d21/ce1 0 2024-06-03T01:20:55.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/93: unlink dc/f1e 0 2024-06-03T01:20:55.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/94: fsync f6 0 2024-06-03T01:20:55.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/95: write dc/f16 [2087097,6942] 0 2024-06-03T01:20:55.432 INFO:tasks.workunit.client.0.smithi032.stdout:8/71: dwrite dc/fd [0,4194304] 0 2024-06-03T01:20:55.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/221: mknod d0/c4d 0 2024-06-03T01:20:55.434 INFO:tasks.workunit.client.0.smithi032.stdout:9/222: write d0/d11/d1d/d38/f3e [620697,57812] 0 2024-06-03T01:20:55.435 INFO:tasks.workunit.client.0.smithi032.stdout:6/96: link f3 d8/d18/d19/f1d 0 2024-06-03T01:20:55.437 INFO:tasks.workunit.client.0.smithi032.stdout:0/76: creat d1/da/f16 x:0 0 0 2024-06-03T01:20:55.438 INFO:tasks.workunit.client.1.smithi044.stdout:5/510: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d66/fa2 x:0 0 0 2024-06-03T01:20:55.440 INFO:tasks.workunit.client.0.smithi032.stdout:7/96: dread fa [0,4194304] 0 2024-06-03T01:20:55.440 INFO:tasks.workunit.client.0.smithi032.stdout:7/97: truncate dc/f16 2444147 0 2024-06-03T01:20:55.440 INFO:tasks.workunit.client.0.smithi032.stdout:7/98: chown dc/f14 2144219 1 2024-06-03T01:20:55.440 INFO:tasks.workunit.client.0.smithi032.stdout:7/99: fdatasync dc/d12/f13 0 2024-06-03T01:20:55.440 INFO:tasks.workunit.client.1.smithi044.stdout:3/393: mknod df/d13/d39/d3d/d43/d72/c7d 0 2024-06-03T01:20:55.441 INFO:tasks.workunit.client.1.smithi044.stdout:3/394: readlink df/d13/d39/d3d/d43/d72/l7a 0 2024-06-03T01:20:55.443 INFO:tasks.workunit.client.1.smithi044.stdout:0/621: rmdir d3/da6/d56 39 2024-06-03T01:20:55.443 INFO:tasks.workunit.client.1.smithi044.stdout:0/622: chown d3/da6/d56/d62/l80 1122 1 2024-06-03T01:20:55.444 INFO:tasks.workunit.client.0.smithi032.stdout:9/223: rmdir d0 39 2024-06-03T01:20:55.445 INFO:tasks.workunit.client.0.smithi032.stdout:6/97: write d8/d18/d19/f1d [8083556,92231] 0 2024-06-03T01:20:55.446 INFO:tasks.workunit.client.0.smithi032.stdout:7/100: truncate fa 1197839 0 2024-06-03T01:20:55.446 INFO:tasks.workunit.client.1.smithi044.stdout:4/542: rename d17/d23 to d17/d1c/d75/d7a/db8 0 2024-06-03T01:20:55.449 INFO:tasks.workunit.client.0.smithi032.stdout:9/224: getdents d0/d11/d1d/d4a 0 2024-06-03T01:20:55.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/98: readlink d8/la 0 2024-06-03T01:20:55.450 INFO:tasks.workunit.client.1.smithi044.stdout:5/511: truncate d2/d11/d12/d2b/d44/f5c 2519650 0 2024-06-03T01:20:55.450 INFO:tasks.workunit.client.0.smithi032.stdout:7/101: symlink dc/l1f 0 2024-06-03T01:20:55.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/77: write d1/f8 [1457494,110639] 0 2024-06-03T01:20:55.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/78: dread - d1/da/f16 zero size 2024-06-03T01:20:55.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/79: dread - d1/da/f16 zero size 2024-06-03T01:20:55.452 INFO:tasks.workunit.client.1.smithi044.stdout:3/395: truncate df/d1e/d3a/f4a 86235 0 2024-06-03T01:20:55.457 INFO:tasks.workunit.client.1.smithi044.stdout:6/664: rename d2/d23/d55/d5f/f62 to d2/d23/d30/d1e/d34/d39/da8/fea 0 2024-06-03T01:20:55.459 INFO:tasks.workunit.client.1.smithi044.stdout:4/543: mkdir d17/d1c/d75/d7a/db8/db9 0 2024-06-03T01:20:55.459 INFO:tasks.workunit.client.1.smithi044.stdout:4/544: write d17/d1c/d75/f62 [1512187,1123] 0 2024-06-03T01:20:55.459 INFO:tasks.workunit.client.1.smithi044.stdout:4/545: truncate d17/d1c/d75/d7a/db8/d28/d2b/f92 4267069 0 2024-06-03T01:20:55.459 INFO:tasks.workunit.client.1.smithi044.stdout:4/546: write d17/d1c/d75/d7a/db8/d59/f6d [745247,125249] 0 2024-06-03T01:20:55.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/80: creat d1/f17 x:0 0 0 2024-06-03T01:20:55.461 INFO:tasks.workunit.client.1.smithi044.stdout:1/564: rename d0/f15 to d0/d3/d20/d7/d2f/db7/fc6 0 2024-06-03T01:20:55.463 INFO:tasks.workunit.client.0.smithi032.stdout:3/80: rmdir de 39 2024-06-03T01:20:55.463 INFO:tasks.workunit.client.0.smithi032.stdout:3/81: write de/f10 [1316927,92824] 0 2024-06-03T01:20:55.465 INFO:tasks.workunit.client.1.smithi044.stdout:4/547: symlink d17/d1c/d75/d7a/db3/lba 0 2024-06-03T01:20:55.467 INFO:tasks.workunit.client.0.smithi032.stdout:3/82: rmdir de/d17 39 2024-06-03T01:20:55.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/72: dwrite f1 [0,4194304] 0 2024-06-03T01:20:55.469 INFO:tasks.workunit.client.1.smithi044.stdout:5/512: rename d2/d11/d1e/d80 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3 0 2024-06-03T01:20:55.473 INFO:tasks.workunit.client.0.smithi032.stdout:3/83: getdents de/d17 0 2024-06-03T01:20:55.473 INFO:tasks.workunit.client.0.smithi032.stdout:3/84: chown de/d17/f18 12056 1 2024-06-03T01:20:55.477 INFO:tasks.workunit.client.1.smithi044.stdout:0/623: dwrite d3/d9/f86 [0,4194304] 0 2024-06-03T01:20:55.479 INFO:tasks.workunit.client.1.smithi044.stdout:5/513: symlink d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/la4 0 2024-06-03T01:20:55.479 INFO:tasks.workunit.client.1.smithi044.stdout:5/514: write d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [641876,122539] 0 2024-06-03T01:20:55.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/73: dread f9 [0,4194304] 0 2024-06-03T01:20:55.484 INFO:tasks.workunit.client.1.smithi044.stdout:1/565: rename d0/d66/fbf to d0/d3/d20/d7/d2f/db7/fc7 0 2024-06-03T01:20:55.484 INFO:tasks.workunit.client.1.smithi044.stdout:1/566: write d0/d3/d14/f38 [9010463,90740] 0 2024-06-03T01:20:55.484 INFO:tasks.workunit.client.0.smithi032.stdout:8/74: getdents dc 0 2024-06-03T01:20:55.486 INFO:tasks.workunit.client.1.smithi044.stdout:0/624: creat d3/da6/dc1/fe2 x:0 0 0 2024-06-03T01:20:55.486 INFO:tasks.workunit.client.1.smithi044.stdout:0/625: readlink d3/d9/d55/d73/l9a 0 2024-06-03T01:20:55.489 INFO:tasks.workunit.client.1.smithi044.stdout:5/515: link d2/d11/d12/d2b/d4a/d57/f76 d2/d11/d12/d2b/d4a/fa5 0 2024-06-03T01:20:55.489 INFO:tasks.workunit.client.1.smithi044.stdout:5/516: read d2/d11/d12/f37 [3906364,21530] 0 2024-06-03T01:20:55.490 INFO:tasks.workunit.client.1.smithi044.stdout:5/517: mkdir d2/d11/d1e/d3b/da6 0 2024-06-03T01:20:55.491 INFO:tasks.workunit.client.1.smithi044.stdout:6/665: dwrite d2/d23/d30/f1d [0,4194304] 0 2024-06-03T01:20:55.491 INFO:tasks.workunit.client.1.smithi044.stdout:6/666: stat d2/d23/d55/d5f/db2/l40 0 2024-06-03T01:20:55.492 INFO:tasks.workunit.client.1.smithi044.stdout:5/518: creat d2/d11/d1e/d21/d24/fa7 x:0 0 0 2024-06-03T01:20:55.492 INFO:tasks.workunit.client.1.smithi044.stdout:5/519: chown d2/d11/d1e/d3b/l5a 0 1 2024-06-03T01:20:55.493 INFO:tasks.workunit.client.1.smithi044.stdout:5/520: chown d2/f3c 7 1 2024-06-03T01:20:55.497 INFO:tasks.workunit.client.1.smithi044.stdout:6/667: truncate d2/d23/d30/d1e/d34/d39/d64/dc7/f5d 3530428 0 2024-06-03T01:20:55.497 INFO:tasks.workunit.client.1.smithi044.stdout:6/668: dread - d2/d23/d30/d1e/d34/d39/d9b/db4/fd9 zero size 2024-06-03T01:20:55.497 INFO:tasks.workunit.client.1.smithi044.stdout:6/669: chown d2/d23/d30/f45 23 1 2024-06-03T01:20:55.497 INFO:tasks.workunit.client.1.smithi044.stdout:6/670: read d2/d23/d30/d1e/d34/d39/d64/f9f [148470,102483] 0 2024-06-03T01:20:55.503 INFO:tasks.workunit.client.1.smithi044.stdout:7/591: sync 2024-06-03T01:20:55.503 INFO:tasks.workunit.client.1.smithi044.stdout:8/514: sync 2024-06-03T01:20:55.504 INFO:tasks.workunit.client.1.smithi044.stdout:2/411: sync 2024-06-03T01:20:55.504 INFO:tasks.workunit.client.1.smithi044.stdout:9/396: sync 2024-06-03T01:20:55.505 INFO:tasks.workunit.client.1.smithi044.stdout:1/567: dread d0/d3/d20/d40/d45/d4b/d80/fb4 [0,4194304] 0 2024-06-03T01:20:55.506 INFO:tasks.workunit.client.1.smithi044.stdout:1/568: fsync d0/d66/dbe/d9e/fa9 0 2024-06-03T01:20:55.506 INFO:tasks.workunit.client.1.smithi044.stdout:2/412: creat db/d15/d49/d6c/d72/d7a/d4e/d7e/f89 x:0 0 0 2024-06-03T01:20:55.506 INFO:tasks.workunit.client.1.smithi044.stdout:2/413: creat db/d15/d49/d7b/f8a x:0 0 0 2024-06-03T01:20:55.512 INFO:tasks.workunit.client.1.smithi044.stdout:3/396: dwrite df/d1e/f35 [0,4194304] 0 2024-06-03T01:20:55.515 INFO:tasks.workunit.client.1.smithi044.stdout:3/397: mknod df/d13/d6d/c7e 0 2024-06-03T01:20:55.523 INFO:tasks.workunit.client.1.smithi044.stdout:8/515: write d5/d20/d33/f63 [3648131,85378] 0 2024-06-03T01:20:55.523 INFO:tasks.workunit.client.1.smithi044.stdout:8/516: readlink d5/d20/d33/l7d 0 2024-06-03T01:20:55.527 INFO:tasks.workunit.client.1.smithi044.stdout:8/517: creat d5/d20/f9e x:0 0 0 2024-06-03T01:20:55.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/398: read df/d13/d39/d3d/d43/f5b [964358,24083] 0 2024-06-03T01:20:55.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/399: creat df/d13/d1b/d56/f7f x:0 0 0 2024-06-03T01:20:55.529 INFO:tasks.workunit.client.1.smithi044.stdout:8/518: symlink d5/d16/d81/l9f 0 2024-06-03T01:20:55.529 INFO:tasks.workunit.client.1.smithi044.stdout:8/519: fsync d5/d16/d1d/d32/f7a 0 2024-06-03T01:20:55.529 INFO:tasks.workunit.client.1.smithi044.stdout:3/400: symlink df/d13/d1b/d56/d5c/l80 0 2024-06-03T01:20:55.530 INFO:tasks.workunit.client.1.smithi044.stdout:8/520: rmdir d5/d50/d79 39 2024-06-03T01:20:55.531 INFO:tasks.workunit.client.0.smithi032.stdout:6/99: dwrite d8/d11/f12 [0,4194304] 0 2024-06-03T01:20:55.532 INFO:tasks.workunit.client.0.smithi032.stdout:9/225: dwrite d0/d11/f22 [4194304,4194304] 0 2024-06-03T01:20:55.532 INFO:tasks.workunit.client.0.smithi032.stdout:0/81: dwrite d1/fc [0,4194304] 0 2024-06-03T01:20:55.532 INFO:tasks.workunit.client.0.smithi032.stdout:0/82: truncate d1/da/f16 27523 0 2024-06-03T01:20:55.532 INFO:tasks.workunit.client.0.smithi032.stdout:9/226: write d0/fd [1536505,29754] 0 2024-06-03T01:20:55.532 INFO:tasks.workunit.client.0.smithi032.stdout:0/83: write d1/f17 [379039,74401] 0 2024-06-03T01:20:55.533 INFO:tasks.workunit.client.0.smithi032.stdout:0/84: readlink d1/lf 0 2024-06-03T01:20:55.533 INFO:tasks.workunit.client.1.smithi044.stdout:3/401: unlink df/d13/d1b/d4b/c51 0 2024-06-03T01:20:55.533 INFO:tasks.workunit.client.1.smithi044.stdout:3/402: truncate df/d1e/d2a/f30 156004 0 2024-06-03T01:20:55.533 INFO:tasks.workunit.client.1.smithi044.stdout:8/521: dread d5/d16/f24 [0,4194304] 0 2024-06-03T01:20:55.535 INFO:tasks.workunit.client.0.smithi032.stdout:6/100: mknod d8/d11/c1e 0 2024-06-03T01:20:55.536 INFO:tasks.workunit.client.1.smithi044.stdout:8/522: dread d5/d16/f3a [0,4194304] 0 2024-06-03T01:20:55.539 INFO:tasks.workunit.client.0.smithi032.stdout:3/85: dwrite de/f15 [0,4194304] 0 2024-06-03T01:20:55.542 INFO:tasks.workunit.client.0.smithi032.stdout:9/227: mknod d0/d23/c4e 0 2024-06-03T01:20:55.543 INFO:tasks.workunit.client.0.smithi032.stdout:0/85: creat d1/f18 x:0 0 0 2024-06-03T01:20:55.543 INFO:tasks.workunit.client.1.smithi044.stdout:8/523: link d5/d55/d74/f94 d5/d20/d47/d64/fa0 0 2024-06-03T01:20:55.546 INFO:tasks.workunit.client.0.smithi032.stdout:3/86: symlink de/l19 0 2024-06-03T01:20:55.547 INFO:tasks.workunit.client.0.smithi032.stdout:0/86: creat d1/da/d10/f19 x:0 0 0 2024-06-03T01:20:55.547 INFO:tasks.workunit.client.0.smithi032.stdout:0/87: creat d1/da/f1a x:0 0 0 2024-06-03T01:20:55.547 INFO:tasks.workunit.client.0.smithi032.stdout:0/88: truncate d1/da/f1a 160582 0 2024-06-03T01:20:55.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/403: read df/d13/d1b/f34 [2260573,25151] 0 2024-06-03T01:20:55.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/404: write df/d13/d1b/d4b/d69/f76 [1029384,83026] 0 2024-06-03T01:20:55.548 INFO:tasks.workunit.client.1.smithi044.stdout:4/548: dwrite d17/d1c/d75/f43 [0,4194304] 0 2024-06-03T01:20:55.548 INFO:tasks.workunit.client.0.smithi032.stdout:7/102: dwrite dc/d12/f13 [4194304,4194304] 0 2024-06-03T01:20:55.549 INFO:tasks.workunit.client.1.smithi044.stdout:2/414: dwrite db/dd/f60 [0,4194304] 0 2024-06-03T01:20:55.549 INFO:tasks.workunit.client.1.smithi044.stdout:6/671: dwrite d2/d23/d55/d5f/f59 [0,4194304] 0 2024-06-03T01:20:55.549 INFO:tasks.workunit.client.1.smithi044.stdout:7/592: dwrite d0/d3/d30/f73 [0,4194304] 0 2024-06-03T01:20:55.549 INFO:tasks.workunit.client.1.smithi044.stdout:1/569: dwrite d0/d3/d20/d7/d13/f4a [4194304,4194304] 0 2024-06-03T01:20:55.550 INFO:tasks.workunit.client.1.smithi044.stdout:7/593: fsync d0/d3/d8/d32/d33/fb1 0 2024-06-03T01:20:55.550 INFO:tasks.workunit.client.1.smithi044.stdout:9/397: dwrite d6/dd/d51/db/f22 [0,4194304] 0 2024-06-03T01:20:55.550 INFO:tasks.workunit.client.1.smithi044.stdout:9/398: creat d6/dd/d45/f81 x:0 0 0 2024-06-03T01:20:55.550 INFO:tasks.workunit.client.1.smithi044.stdout:0/626: dwrite d3/f3a [0,4194304] 0 2024-06-03T01:20:55.550 INFO:tasks.workunit.client.1.smithi044.stdout:0/627: truncate d3/da6/d56/d62/fa1 847272 0 2024-06-03T01:20:55.551 INFO:tasks.workunit.client.1.smithi044.stdout:3/405: mkdir df/d1e/d2a/d70/d81 0 2024-06-03T01:20:55.557 INFO:tasks.workunit.client.0.smithi032.stdout:6/101: dwrite d8/d11/f17 [4194304,4194304] 0 2024-06-03T01:20:55.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/524: dread d5/d16/f3c [0,4194304] 0 2024-06-03T01:20:55.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/525: getdents d5/d16/d81/d9b 0 2024-06-03T01:20:55.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/526: chown d5/d55/d74/f94 1289 1 2024-06-03T01:20:55.561 INFO:tasks.workunit.client.1.smithi044.stdout:2/415: creat db/d1b/d2a/d52/f8b x:0 0 0 2024-06-03T01:20:55.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/89: rmdir d1/da/d10 39 2024-06-03T01:20:55.566 INFO:tasks.workunit.client.1.smithi044.stdout:7/594: truncate d0/d3/d8/d95/d63/d7f/d9c/f50 1068423 0 2024-06-03T01:20:55.567 INFO:tasks.workunit.client.0.smithi032.stdout:0/90: creat d1/f1b x:0 0 0 2024-06-03T01:20:55.569 INFO:tasks.workunit.client.0.smithi032.stdout:0/91: unlink d1/f8 0 2024-06-03T01:20:55.569 INFO:tasks.workunit.client.0.smithi032.stdout:0/92: write d1/da/f1a [688214,122156] 0 2024-06-03T01:20:55.570 INFO:tasks.workunit.client.0.smithi032.stdout:3/87: dwrite de/f15 [0,4194304] 0 2024-06-03T01:20:55.570 INFO:tasks.workunit.client.0.smithi032.stdout:3/88: write fb [750380,125972] 0 2024-06-03T01:20:55.570 INFO:tasks.workunit.client.0.smithi032.stdout:6/102: read d8/d11/f12 [3365681,105395] 0 2024-06-03T01:20:55.571 INFO:tasks.workunit.client.1.smithi044.stdout:1/570: link d0/d3/d20/d40/d45/d4b/d5c/f63 d0/d3/d20/d40/d45/d4b/d5a/fc8 0 2024-06-03T01:20:55.571 INFO:tasks.workunit.client.1.smithi044.stdout:6/672: dwrite d2/d23/d30/d1e/d38/d77/fbd [0,4194304] 0 2024-06-03T01:20:55.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/93: truncate d1/f6 1387830 0 2024-06-03T01:20:55.578 INFO:tasks.workunit.client.1.smithi044.stdout:9/399: unlink d6/d26/f4f 0 2024-06-03T01:20:55.586 INFO:tasks.workunit.client.1.smithi044.stdout:0/628: mkdir d3/d5c/de3 0 2024-06-03T01:20:55.586 INFO:tasks.workunit.client.1.smithi044.stdout:0/629: truncate d3/f85 1061474 0 2024-06-03T01:20:55.586 INFO:tasks.workunit.client.1.smithi044.stdout:3/406: mknod df/d13/d1b/d4b/c82 0 2024-06-03T01:20:55.587 INFO:tasks.workunit.client.1.smithi044.stdout:4/549: rename c16 to d17/d1c/d70/d7d/cbb 0 2024-06-03T01:20:55.588 INFO:tasks.workunit.client.1.smithi044.stdout:8/527: mknod d5/d20/d2b/d58/d89/ca1 0 2024-06-03T01:20:55.591 INFO:tasks.workunit.client.1.smithi044.stdout:7/595: getdents d0/d3/d8/d95/d63/d89 0 2024-06-03T01:20:55.591 INFO:tasks.workunit.client.1.smithi044.stdout:7/596: stat d0/d16/d23/l26 0 2024-06-03T01:20:55.593 INFO:tasks.workunit.client.0.smithi032.stdout:6/103: dread f4 [0,4194304] 0 2024-06-03T01:20:55.593 INFO:tasks.workunit.client.1.smithi044.stdout:6/673: symlink d2/d23/d30/d1e/d34/d39/dcd/dd1/leb 0 2024-06-03T01:20:55.594 INFO:tasks.workunit.client.1.smithi044.stdout:6/674: read d2/d23/d30/d1e/d34/d39/d64/dc7/f5d [560433,3145] 0 2024-06-03T01:20:55.595 INFO:tasks.workunit.client.1.smithi044.stdout:0/630: mkdir d3/da6/dc1/dc2/de4 0 2024-06-03T01:20:55.595 INFO:tasks.workunit.client.0.smithi032.stdout:6/104: creat d8/d18/f1f x:0 0 0 2024-06-03T01:20:55.596 INFO:tasks.workunit.client.1.smithi044.stdout:3/407: creat df/d13/d39/d3d/d43/d71/f83 x:0 0 0 2024-06-03T01:20:55.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/89: write fd [2858344,20604] 0 2024-06-03T01:20:55.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/90: truncate fa 469444 0 2024-06-03T01:20:55.598 INFO:tasks.workunit.client.0.smithi032.stdout:6/105: getdents d8/d11 0 2024-06-03T01:20:55.598 INFO:tasks.workunit.client.0.smithi032.stdout:6/106: chown d8/d18/d19 23 1 2024-06-03T01:20:55.598 INFO:tasks.workunit.client.0.smithi032.stdout:6/107: chown d8/f9 1641 1 2024-06-03T01:20:55.600 INFO:tasks.workunit.client.1.smithi044.stdout:1/571: rename d0/d66/dbe/d9e/lb2 to d0/d66/dc2/lc9 0 2024-06-03T01:20:55.600 INFO:tasks.workunit.client.1.smithi044.stdout:1/572: truncate d0/d3/d20/d40/d45/d4b/f90 923896 0 2024-06-03T01:20:55.601 INFO:tasks.workunit.client.0.smithi032.stdout:3/91: unlink de/f16 0 2024-06-03T01:20:55.601 INFO:tasks.workunit.client.1.smithi044.stdout:4/550: truncate f5 3076602 0 2024-06-03T01:20:55.601 INFO:tasks.workunit.client.0.smithi032.stdout:3/92: truncate f9 254814 0 2024-06-03T01:20:55.602 INFO:tasks.workunit.client.0.smithi032.stdout:3/93: fdatasync de/f15 0 2024-06-03T01:20:55.602 INFO:tasks.workunit.client.0.smithi032.stdout:3/94: chown de/c12 5652 1 2024-06-03T01:20:55.603 INFO:tasks.workunit.client.1.smithi044.stdout:2/416: rmdir db/d1b/d3f 39 2024-06-03T01:20:55.603 INFO:tasks.workunit.client.1.smithi044.stdout:2/417: chown db/d1b/d2a/c42 5 1 2024-06-03T01:20:55.605 INFO:tasks.workunit.client.1.smithi044.stdout:7/597: creat d0/d3/d8/d95/d63/fc1 x:0 0 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:7/598: read d0/d16/f54 [65892,72753] 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:7/599: stat d0/d3/d8/d95/d84/l85 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:6/675: getdents d2/d23/d30/d1e/d34/d39/d9b/db4/dba 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:9/400: dwrite d6/d26/f4a [4194304,4194304] 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:9/401: write d6/dd/f36 [919891,34276] 0 2024-06-03T01:20:55.607 INFO:tasks.workunit.client.1.smithi044.stdout:9/402: write d6/dd/d45/f56 [970749,118037] 0 2024-06-03T01:20:55.608 INFO:tasks.workunit.client.0.smithi032.stdout:0/94: dwrite d1/f9 [0,4194304] 0 2024-06-03T01:20:55.608 INFO:tasks.workunit.client.0.smithi032.stdout:0/95: chown d1/da/d10/l15 7193 1 2024-06-03T01:20:55.608 INFO:tasks.workunit.client.1.smithi044.stdout:0/631: link d3/d9/d55/d5a/l51 d3/d6b/d95/le5 0 2024-06-03T01:20:55.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/96: dread d1/da/f1a [0,4194304] 0 2024-06-03T01:20:55.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/97: write d1/f1b [130866,102989] 0 2024-06-03T01:20:55.610 INFO:tasks.workunit.client.0.smithi032.stdout:0/98: creat d1/f1c x:0 0 0 2024-06-03T01:20:55.611 INFO:tasks.workunit.client.0.smithi032.stdout:3/95: dread f9 [0,4194304] 0 2024-06-03T01:20:55.611 INFO:tasks.workunit.client.0.smithi032.stdout:0/99: mknod d1/da/d10/c1d 0 2024-06-03T01:20:55.615 INFO:tasks.workunit.client.0.smithi032.stdout:0/100: dread d1/f6 [0,4194304] 0 2024-06-03T01:20:55.615 INFO:tasks.workunit.client.0.smithi032.stdout:0/101: stat d1/da/c11 0 2024-06-03T01:20:55.621 INFO:tasks.workunit.client.1.smithi044.stdout:3/408: mkdir df/d1e/d3a/d84 0 2024-06-03T01:20:55.621 INFO:tasks.workunit.client.1.smithi044.stdout:3/409: write df/f1c [8704451,38102] 0 2024-06-03T01:20:55.622 INFO:tasks.workunit.client.0.smithi032.stdout:3/96: dread f5 [0,4194304] 0 2024-06-03T01:20:55.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/97: dread f9 [0,4194304] 0 2024-06-03T01:20:55.625 INFO:tasks.workunit.client.0.smithi032.stdout:0/102: mknod d1/da/d10/c1e 0 2024-06-03T01:20:55.626 INFO:tasks.workunit.client.1.smithi044.stdout:1/573: rename d0/d3/d20/d40/f55 to d0/d3/d20/d40/d45/d4b/d5c/d94/fca 0 2024-06-03T01:20:55.626 INFO:tasks.workunit.client.1.smithi044.stdout:1/574: fsync d0/d16/f26 0 2024-06-03T01:20:55.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/551: mknod d17/d1c/d75/cbc 0 2024-06-03T01:20:55.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/552: chown d17/d1c/d9c/lad 3293 1 2024-06-03T01:20:55.630 INFO:tasks.workunit.client.1.smithi044.stdout:8/528: rmdir d5/d20/d47 39 2024-06-03T01:20:55.630 INFO:tasks.workunit.client.1.smithi044.stdout:8/529: chown d5/d16/d81/l9f 6066413 1 2024-06-03T01:20:55.633 INFO:tasks.workunit.client.0.smithi032.stdout:3/98: link de/f11 de/f1a 0 2024-06-03T01:20:55.634 INFO:tasks.workunit.client.1.smithi044.stdout:3/410: dread df/d13/f3f [0,4194304] 0 2024-06-03T01:20:55.634 INFO:tasks.workunit.client.1.smithi044.stdout:7/600: creat d0/d3/d30/fc2 x:0 0 0 2024-06-03T01:20:55.634 INFO:tasks.workunit.client.0.smithi032.stdout:3/99: truncate de/f10 1385945 0 2024-06-03T01:20:55.636 INFO:tasks.workunit.client.1.smithi044.stdout:9/403: symlink d6/dd/d51/d3b/l82 0 2024-06-03T01:20:55.636 INFO:tasks.workunit.client.0.smithi032.stdout:3/100: creat de/d17/f1b x:0 0 0 2024-06-03T01:20:55.636 INFO:tasks.workunit.client.0.smithi032.stdout:3/101: creat de/d17/f1c x:0 0 0 2024-06-03T01:20:55.638 INFO:tasks.workunit.client.1.smithi044.stdout:0/632: creat d3/d6b/fe6 x:0 0 0 2024-06-03T01:20:55.638 INFO:tasks.workunit.client.1.smithi044.stdout:0/633: write d3/f91 [2307233,10130] 0 2024-06-03T01:20:55.638 INFO:tasks.workunit.client.0.smithi032.stdout:3/102: mkdir de/d17/d1d 0 2024-06-03T01:20:55.638 INFO:tasks.workunit.client.0.smithi032.stdout:3/103: chown de/d17/f1c 16707432 1 2024-06-03T01:20:55.640 INFO:tasks.workunit.client.1.smithi044.stdout:1/575: mkdir d0/d3/d20/d40/dcb 0 2024-06-03T01:20:55.644 INFO:tasks.workunit.client.1.smithi044.stdout:8/530: symlink d5/d55/d56/d5c/la2 0 2024-06-03T01:20:55.646 INFO:tasks.workunit.client.1.smithi044.stdout:8/531: dread d5/d55/f5d [0,4194304] 0 2024-06-03T01:20:55.650 INFO:tasks.workunit.client.1.smithi044.stdout:2/418: getdents db/d15/d49 0 2024-06-03T01:20:55.651 INFO:tasks.workunit.client.1.smithi044.stdout:2/419: write db/d1b/d3f/f41 [163019,62429] 0 2024-06-03T01:20:55.651 INFO:tasks.workunit.client.1.smithi044.stdout:2/420: fdatasync db/d1b/d27/f39 0 2024-06-03T01:20:55.651 INFO:tasks.workunit.client.1.smithi044.stdout:2/421: write db/d1b/d2a/d52/f8b [176687,21200] 0 2024-06-03T01:20:55.651 INFO:tasks.workunit.client.0.smithi032.stdout:0/103: dwrite d1/f1c [0,4194304] 0 2024-06-03T01:20:55.652 INFO:tasks.workunit.client.1.smithi044.stdout:4/553: dwrite d17/d1c/d75/d7a/db8/f57 [0,4194304] 0 2024-06-03T01:20:55.652 INFO:tasks.workunit.client.1.smithi044.stdout:4/554: readlink d17/d1c/d75/d7a/la9 0 2024-06-03T01:20:55.652 INFO:tasks.workunit.client.1.smithi044.stdout:4/555: readlink d17/d63/l8a 0 2024-06-03T01:20:55.654 INFO:tasks.workunit.client.1.smithi044.stdout:3/411: rename l4 to df/d1e/d78/l85 0 2024-06-03T01:20:55.655 INFO:tasks.workunit.client.0.smithi032.stdout:0/104: unlink d1/l12 0 2024-06-03T01:20:55.655 INFO:tasks.workunit.client.0.smithi032.stdout:0/105: fdatasync d1/f6 0 2024-06-03T01:20:55.655 INFO:tasks.workunit.client.0.smithi032.stdout:0/106: fsync d1/f6 0 2024-06-03T01:20:55.655 INFO:tasks.workunit.client.0.smithi032.stdout:0/107: write d1/f1b [969332,88106] 0 2024-06-03T01:20:55.656 INFO:tasks.workunit.client.1.smithi044.stdout:7/601: symlink d0/d3/d8/d95/d63/lc3 0 2024-06-03T01:20:55.657 INFO:tasks.workunit.client.1.smithi044.stdout:0/634: mkdir d3/da6/d56/d62/de7 0 2024-06-03T01:20:55.659 INFO:tasks.workunit.client.1.smithi044.stdout:1/576: symlink d0/d3/d20/d40/d45/d4b/d80/lcc 0 2024-06-03T01:20:55.660 INFO:tasks.workunit.client.1.smithi044.stdout:1/577: stat d0/d3/d14/c85 0 2024-06-03T01:20:55.660 INFO:tasks.workunit.client.1.smithi044.stdout:1/578: dread - d0/f6c zero size 2024-06-03T01:20:55.662 INFO:tasks.workunit.client.1.smithi044.stdout:8/532: creat d5/d20/d2b/d58/fa3 x:0 0 0 2024-06-03T01:20:55.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/556: dread d17/d1c/d75/d7a/db8/f57 [0,4194304] 0 2024-06-03T01:20:55.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/557: write d17/f69 [130780,125130] 0 2024-06-03T01:20:55.667 INFO:tasks.workunit.client.0.smithi032.stdout:3/104: dwrite f5 [0,4194304] 0 2024-06-03T01:20:55.667 INFO:tasks.workunit.client.0.smithi032.stdout:3/105: creat de/d17/f1e x:0 0 0 2024-06-03T01:20:55.667 INFO:tasks.workunit.client.1.smithi044.stdout:2/422: creat db/d15/d49/d6c/f8c x:0 0 0 2024-06-03T01:20:55.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/412: write df/d13/f3b [4055493,113848] 0 2024-06-03T01:20:55.670 INFO:tasks.workunit.client.1.smithi044.stdout:9/404: rename d6/dd/d51/d3b/l82 to d6/dd/d51/d46/l83 0 2024-06-03T01:20:55.671 INFO:tasks.workunit.client.0.smithi032.stdout:3/106: dread fa [0,4194304] 0 2024-06-03T01:20:55.671 INFO:tasks.workunit.client.1.smithi044.stdout:1/579: rmdir d0/d3/d14/d81 39 2024-06-03T01:20:55.673 INFO:tasks.workunit.client.0.smithi032.stdout:3/107: rmdir de/d17/d1d 0 2024-06-03T01:20:55.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/533: creat d5/fa4 x:0 0 0 2024-06-03T01:20:55.674 INFO:tasks.workunit.client.1.smithi044.stdout:9/405: dread d6/dd/d51/d16/f1b [0,4194304] 0 2024-06-03T01:20:55.674 INFO:tasks.workunit.client.1.smithi044.stdout:9/406: creat d6/f84 x:0 0 0 2024-06-03T01:20:55.680 INFO:tasks.workunit.client.1.smithi044.stdout:4/558: link d17/f94 d17/d1c/d9c/fbd 0 2024-06-03T01:20:55.682 INFO:tasks.workunit.client.1.smithi044.stdout:1/580: mkdir d0/d3/d20/d7/d13/d2d/dcd 0 2024-06-03T01:20:55.684 INFO:tasks.workunit.client.1.smithi044.stdout:2/423: write db/d15/f70 [1241742,14342] 0 2024-06-03T01:20:55.685 INFO:tasks.workunit.client.0.smithi032.stdout:0/108: dwrite d1/f18 [0,4194304] 0 2024-06-03T01:20:55.686 INFO:tasks.workunit.client.0.smithi032.stdout:0/109: write d1/f6 [615104,14527] 0 2024-06-03T01:20:55.687 INFO:tasks.workunit.client.0.smithi032.stdout:0/110: creat d1/da/f1f x:0 0 0 2024-06-03T01:20:55.687 INFO:tasks.workunit.client.0.smithi032.stdout:0/111: chown d1/da/f1a 2874 1 2024-06-03T01:20:55.688 INFO:tasks.workunit.client.1.smithi044.stdout:4/559: mknod d17/d1c/d75/d7a/db8/db9/cbe 0 2024-06-03T01:20:55.688 INFO:tasks.workunit.client.1.smithi044.stdout:7/602: dwrite d0/d16/fb8 [0,4194304] 0 2024-06-03T01:20:55.688 INFO:tasks.workunit.client.1.smithi044.stdout:7/603: creat d0/d16/fc4 x:0 0 0 2024-06-03T01:20:55.688 INFO:tasks.workunit.client.1.smithi044.stdout:7/604: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/f81 zero size 2024-06-03T01:20:55.689 INFO:tasks.workunit.client.0.smithi032.stdout:3/108: getdents de/d17 0 2024-06-03T01:20:55.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/413: dwrite df/d1e/d3a/f4a [0,4194304] 0 2024-06-03T01:20:55.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/414: creat df/d13/d1b/d56/f86 x:0 0 0 2024-06-03T01:20:55.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/415: write df/d13/d1b/d56/f7f [418348,96792] 0 2024-06-03T01:20:55.692 INFO:tasks.workunit.client.0.smithi032.stdout:0/112: symlink d1/da/d10/l20 0 2024-06-03T01:20:55.692 INFO:tasks.workunit.client.1.smithi044.stdout:8/534: dwrite d5/f15 [0,4194304] 0 2024-06-03T01:20:55.692 INFO:tasks.workunit.client.1.smithi044.stdout:8/535: write d5/d16/d1d/f38 [464310,26972] 0 2024-06-03T01:20:55.693 INFO:tasks.workunit.client.0.smithi032.stdout:0/113: creat d1/f21 x:0 0 0 2024-06-03T01:20:55.702 INFO:tasks.workunit.client.0.smithi032.stdout:3/109: dread f8 [0,4194304] 0 2024-06-03T01:20:55.706 INFO:tasks.workunit.client.1.smithi044.stdout:4/560: rename fc to d17/d63/dae/fbf 0 2024-06-03T01:20:55.709 INFO:tasks.workunit.client.1.smithi044.stdout:9/407: rmdir d6 39 2024-06-03T01:20:55.709 INFO:tasks.workunit.client.1.smithi044.stdout:9/408: write d6/dd/d51/d16/d27/f39 [1484208,7452] 0 2024-06-03T01:20:55.712 INFO:tasks.workunit.client.1.smithi044.stdout:1/581: getdents d0/d3/d20/d7 0 2024-06-03T01:20:55.713 INFO:tasks.workunit.client.1.smithi044.stdout:7/605: dwrite d0/d3/d8/d32/f38 [0,4194304] 0 2024-06-03T01:20:55.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/114: dwrite d1/f18 [0,4194304] 0 2024-06-03T01:20:55.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/115: readlink d1/da/d10/l20 0 2024-06-03T01:20:55.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/116: truncate d1/da/f16 196107 0 2024-06-03T01:20:55.715 INFO:tasks.workunit.client.0.smithi032.stdout:0/117: fsync d1/f6 0 2024-06-03T01:20:55.715 INFO:tasks.workunit.client.0.smithi032.stdout:0/118: truncate d1/da/f1f 697418 0 2024-06-03T01:20:55.715 INFO:tasks.workunit.client.1.smithi044.stdout:8/536: link d5/d16/d81/f83 d5/d16/d1d/fa5 0 2024-06-03T01:20:55.720 INFO:tasks.workunit.client.1.smithi044.stdout:4/561: rename d17/d1c/l66 to d17/d1c/d75/d7a/db8/d28/d2b/lc0 0 2024-06-03T01:20:55.722 INFO:tasks.workunit.client.1.smithi044.stdout:9/409: symlink d6/d38/l85 0 2024-06-03T01:20:55.722 INFO:tasks.workunit.client.1.smithi044.stdout:9/410: stat d6/dd/d51/d16/f34 0 2024-06-03T01:20:55.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/537: creat d5/d20/d2b/d58/d89/fa6 x:0 0 0 2024-06-03T01:20:55.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/538: readlink d5/d16/d1d/d32/l37 0 2024-06-03T01:20:55.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/539: readlink d5/d16/l6c 0 2024-06-03T01:20:55.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/540: chown d5/d16/d1d/d6b 40010178 1 2024-06-03T01:20:55.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/541: truncate d5/d20/d47/f71 557200 0 2024-06-03T01:20:55.725 INFO:tasks.workunit.client.1.smithi044.stdout:4/562: creat d17/d1c/d9c/fc1 x:0 0 0 2024-06-03T01:20:55.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/119: write d1/f1c [4071011,27713] 0 2024-06-03T01:20:55.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/120: fsync d1/da/f1a 0 2024-06-03T01:20:55.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/121: write d1/f1b [167550,47438] 0 2024-06-03T01:20:55.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/122: truncate d1/f6 1763480 0 2024-06-03T01:20:55.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/582: dwrite d0/d16/f26 [0,4194304] 0 2024-06-03T01:20:55.737 INFO:tasks.workunit.client.0.smithi032.stdout:0/123: dread d1/f1c [0,4194304] 0 2024-06-03T01:20:55.738 INFO:tasks.workunit.client.0.smithi032.stdout:0/124: dread d1/da/f16 [0,4194304] 0 2024-06-03T01:20:55.739 INFO:tasks.workunit.client.1.smithi044.stdout:4/563: write d17/d1c/d75/d7a/db8/d59/f81 [367250,20852] 0 2024-06-03T01:20:55.740 INFO:tasks.workunit.client.0.smithi032.stdout:0/125: symlink d1/da/d10/l22 0 2024-06-03T01:20:55.741 INFO:tasks.workunit.client.0.smithi032.stdout:0/126: mknod d1/da/c23 0 2024-06-03T01:20:55.742 INFO:tasks.workunit.client.0.smithi032.stdout:0/127: chown d1/da/d10/l20 17 1 2024-06-03T01:20:55.743 INFO:tasks.workunit.client.0.smithi032.stdout:0/128: link d1/da/d10/c1d d1/da/d10/c24 0 2024-06-03T01:20:55.743 INFO:tasks.workunit.client.0.smithi032.stdout:0/129: fdatasync d1/da/f1a 0 2024-06-03T01:20:55.744 INFO:tasks.workunit.client.0.smithi032.stdout:0/130: rename d1/da/cd to d1/da/d10/c25 0 2024-06-03T01:20:55.744 INFO:tasks.workunit.client.1.smithi044.stdout:8/542: dwrite d5/d55/f86 [0,4194304] 0 2024-06-03T01:20:55.745 INFO:tasks.workunit.client.0.smithi032.stdout:0/131: dread d1/da/f1f [0,4194304] 0 2024-06-03T01:20:55.745 INFO:tasks.workunit.client.0.smithi032.stdout:0/132: read d1/f1c [2870243,86614] 0 2024-06-03T01:20:55.746 INFO:tasks.workunit.client.1.smithi044.stdout:4/564: dread d17/d1c/f3a [4194304,4194304] 0 2024-06-03T01:20:55.748 INFO:tasks.workunit.client.1.smithi044.stdout:8/543: link d5/d50/f57 d5/d20/d2b/d98/fa7 0 2024-06-03T01:20:55.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/565: getdents d17/d1c/d75 0 2024-06-03T01:20:55.759 INFO:tasks.workunit.client.1.smithi044.stdout:1/583: dwrite d0/d3/f62 [0,4194304] 0 2024-06-03T01:20:55.760 INFO:tasks.workunit.client.1.smithi044.stdout:8/544: truncate d5/d16/d1d/f38 84120 0 2024-06-03T01:20:55.761 INFO:tasks.workunit.client.0.smithi032.stdout:0/133: dwrite d1/fc [0,4194304] 0 2024-06-03T01:20:55.762 INFO:tasks.workunit.client.1.smithi044.stdout:1/584: creat d0/d3/d20/d7/d13/d2d/fce x:0 0 0 2024-06-03T01:20:55.763 INFO:tasks.workunit.client.0.smithi032.stdout:0/134: dread d1/da/f16 [0,4194304] 0 2024-06-03T01:20:55.763 INFO:tasks.workunit.client.0.smithi032.stdout:0/135: truncate d1/f1b 1911221 0 2024-06-03T01:20:55.764 INFO:tasks.workunit.client.1.smithi044.stdout:8/545: link d5/d20/d2b/l76 d5/d16/d1d/d59/la8 0 2024-06-03T01:20:55.765 INFO:tasks.workunit.client.1.smithi044.stdout:1/585: creat d0/d3/d14/d81/fcf x:0 0 0 2024-06-03T01:20:55.766 INFO:tasks.workunit.client.1.smithi044.stdout:8/546: symlink d5/d16/la9 0 2024-06-03T01:20:55.768 INFO:tasks.workunit.client.1.smithi044.stdout:4/566: dwrite d17/d1c/d75/d7a/db8/d28/d2b/f4a [4194304,4194304] 0 2024-06-03T01:20:55.768 INFO:tasks.workunit.client.1.smithi044.stdout:1/586: creat d0/d66/dbe/d9e/fd0 x:0 0 0 2024-06-03T01:20:55.770 INFO:tasks.workunit.client.1.smithi044.stdout:8/547: symlink d5/d55/laa 0 2024-06-03T01:20:55.770 INFO:tasks.workunit.client.1.smithi044.stdout:8/548: fdatasync d5/d20/d2b/d58/f84 0 2024-06-03T01:20:55.772 INFO:tasks.workunit.client.0.smithi032.stdout:0/136: link d1/da/d10/c25 d1/da/c26 0 2024-06-03T01:20:55.775 INFO:tasks.workunit.client.1.smithi044.stdout:1/587: creat d0/d66/dc2/fd1 x:0 0 0 2024-06-03T01:20:55.775 INFO:tasks.workunit.client.1.smithi044.stdout:1/588: chown d0/d3/d20/l48 81428717 1 2024-06-03T01:20:55.784 INFO:tasks.workunit.client.1.smithi044.stdout:4/567: rmdir d17 39 2024-06-03T01:20:55.784 INFO:tasks.workunit.client.0.smithi032.stdout:1/29: sync 2024-06-03T01:20:55.784 INFO:tasks.workunit.client.0.smithi032.stdout:5/84: sync 2024-06-03T01:20:55.784 INFO:tasks.workunit.client.0.smithi032.stdout:4/84: sync 2024-06-03T01:20:55.785 INFO:tasks.workunit.client.0.smithi032.stdout:0/137: dread d1/f9 [0,4194304] 0 2024-06-03T01:20:55.785 INFO:tasks.workunit.client.0.smithi032.stdout:0/138: chown d1/da/d10/f19 15730086 1 2024-06-03T01:20:55.785 INFO:tasks.workunit.client.0.smithi032.stdout:0/139: dread - d1/f21 zero size 2024-06-03T01:20:55.786 INFO:tasks.workunit.client.0.smithi032.stdout:1/30: link l7 l9 0 2024-06-03T01:20:55.789 INFO:tasks.workunit.client.0.smithi032.stdout:5/85: creat da/dd/df/d10/d16/d18/f1d x:0 0 0 2024-06-03T01:20:55.789 INFO:tasks.workunit.client.0.smithi032.stdout:5/86: readlink da/lc 0 2024-06-03T01:20:55.789 INFO:tasks.workunit.client.0.smithi032.stdout:4/85: creat d4/db/da/d1d/f1f x:0 0 0 2024-06-03T01:20:55.791 INFO:tasks.workunit.client.0.smithi032.stdout:0/140: getdents d1/da/d10 0 2024-06-03T01:20:55.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/86: link d4/db/c14 d4/c20 0 2024-06-03T01:20:55.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/87: creat d4/db/da/d1d/f21 x:0 0 0 2024-06-03T01:20:55.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/88: chown d4/db/da/fd 4798 1 2024-06-03T01:20:55.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/89: readlink d4/db/l18 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:1/589: rename d0/d3/d20/d7/d4e to d0/dd2 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:8/549: dwrite d5/d16/f7c [0,4194304] 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:1/590: creat d0/d3/d20/d40/d45/d6f/fd3 x:0 0 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:1/591: dread - d0/f7d zero size 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/568: dread d17/d1c/d75/f5d [0,4194304] 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/569: readlink d17/d1c/d75/d7a/db8/d28/lb1 0 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/570: chown d17/d63 5 1 2024-06-03T01:20:55.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/571: fsync d17/d1c/d75/f4b 0 2024-06-03T01:20:55.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/572: fdatasync d17/d1c/fb7 0 2024-06-03T01:20:55.797 INFO:tasks.workunit.client.0.smithi032.stdout:0/141: creat d1/da/d10/f27 x:0 0 0 2024-06-03T01:20:55.799 INFO:tasks.workunit.client.0.smithi032.stdout:4/90: unlink d4/db/da/f16 0 2024-06-03T01:20:55.799 INFO:tasks.workunit.client.1.smithi044.stdout:1/592: link d0/d3/d20/d7/d2f/f59 d0/d3/d20/d40/d45/d4b/fd4 0 2024-06-03T01:20:55.800 INFO:tasks.workunit.client.1.smithi044.stdout:1/593: creat d0/d3/d14/d81/fd5 x:0 0 0 2024-06-03T01:20:55.800 INFO:tasks.workunit.client.0.smithi032.stdout:0/142: mkdir d1/da/d28 0 2024-06-03T01:20:55.800 INFO:tasks.workunit.client.1.smithi044.stdout:8/550: rmdir d5/d16/d81 39 2024-06-03T01:20:55.800 INFO:tasks.workunit.client.1.smithi044.stdout:8/551: creat d5/d20/d2b/d58/d89/fab x:0 0 0 2024-06-03T01:20:55.800 INFO:tasks.workunit.client.1.smithi044.stdout:8/552: write d5/d20/d47/f49 [367800,35395] 0 2024-06-03T01:20:55.803 INFO:tasks.workunit.client.1.smithi044.stdout:4/573: dread d17/d1c/d75/f62 [0,4194304] 0 2024-06-03T01:20:55.804 INFO:tasks.workunit.client.1.smithi044.stdout:1/594: mkdir d0/d3/d14/d30/dd6 0 2024-06-03T01:20:55.804 INFO:tasks.workunit.client.0.smithi032.stdout:5/87: dread da/f12 [0,4194304] 0 2024-06-03T01:20:55.805 INFO:tasks.workunit.client.0.smithi032.stdout:5/88: chown da/dd/df 31992 1 2024-06-03T01:20:55.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/553: creat d5/d55/fac x:0 0 0 2024-06-03T01:20:55.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/554: creat d5/d55/fad x:0 0 0 2024-06-03T01:20:55.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/555: creat d5/d55/d56/d5c/fae x:0 0 0 2024-06-03T01:20:55.810 INFO:tasks.workunit.client.1.smithi044.stdout:1/595: unlink d0/d3/d20/d40/d45/d6f/c95 0 2024-06-03T01:20:55.812 INFO:tasks.workunit.client.1.smithi044.stdout:6/676: sync 2024-06-03T01:20:55.812 INFO:tasks.workunit.client.1.smithi044.stdout:5/521: sync 2024-06-03T01:20:55.812 INFO:tasks.workunit.client.1.smithi044.stdout:6/677: truncate d2/d23/d30/d1e/d34/d39/d7d/fac 678693 0 2024-06-03T01:20:55.815 INFO:tasks.workunit.client.0.smithi032.stdout:5/89: dread da/f12 [0,4194304] 0 2024-06-03T01:20:55.815 INFO:tasks.workunit.client.0.smithi032.stdout:1/31: dwrite f5 [0,4194304] 0 2024-06-03T01:20:55.818 INFO:tasks.workunit.client.1.smithi044.stdout:8/556: symlink d5/d55/d56/d5c/d68/laf 0 2024-06-03T01:20:55.820 INFO:tasks.workunit.client.1.smithi044.stdout:5/522: dread d2/fa [0,4194304] 0 2024-06-03T01:20:55.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/678: creat d2/d23/d55/d5f/db2/fec x:0 0 0 2024-06-03T01:20:55.822 INFO:tasks.workunit.client.1.smithi044.stdout:8/557: mkdir d5/d55/d74/d82/db0 0 2024-06-03T01:20:55.824 INFO:tasks.workunit.client.1.smithi044.stdout:4/574: dwrite d17/d1c/d75/d7a/db8/f68 [0,4194304] 0 2024-06-03T01:20:55.825 INFO:tasks.workunit.client.0.smithi032.stdout:0/143: dwrite d1/f17 [0,4194304] 0 2024-06-03T01:20:55.825 INFO:tasks.workunit.client.0.smithi032.stdout:0/144: truncate d1/da/f1a 1268259 0 2024-06-03T01:20:55.830 INFO:tasks.workunit.client.0.smithi032.stdout:1/32: getdents . 0 2024-06-03T01:20:55.831 INFO:tasks.workunit.client.1.smithi044.stdout:1/596: rmdir d0/d3/d14 39 2024-06-03T01:20:55.831 INFO:tasks.workunit.client.1.smithi044.stdout:1/597: chown d0/d3/d20/d40/d45/d4b/c8f 200099097 1 2024-06-03T01:20:55.833 INFO:tasks.workunit.client.0.smithi032.stdout:1/33: creat fa x:0 0 0 2024-06-03T01:20:55.836 INFO:tasks.workunit.client.1.smithi044.stdout:8/558: creat d5/d20/d33/fb1 x:0 0 0 2024-06-03T01:20:55.836 INFO:tasks.workunit.client.1.smithi044.stdout:8/559: stat d5/d20/d2b/d58/d89/ca1 0 2024-06-03T01:20:55.836 INFO:tasks.workunit.client.1.smithi044.stdout:8/560: fsync d5/d16/d81/f9c 0 2024-06-03T01:20:55.839 INFO:tasks.workunit.client.0.smithi032.stdout:5/90: dwrite da/dd/fe [0,4194304] 0 2024-06-03T01:20:55.840 INFO:tasks.workunit.client.1.smithi044.stdout:1/598: link d0/d3/d14/d81/f56 d0/d3/d20/fd7 0 2024-06-03T01:20:55.840 INFO:tasks.workunit.client.1.smithi044.stdout:1/599: readlink d0/d3/d20/d40/d45/lac 0 2024-06-03T01:20:55.840 INFO:tasks.workunit.client.1.smithi044.stdout:1/600: write d0/d3/d20/d7/d2f/db7/fc7 [651167,112043] 0 2024-06-03T01:20:55.840 INFO:tasks.workunit.client.1.smithi044.stdout:1/601: creat d0/d3/d20/d7/d13/fd8 x:0 0 0 2024-06-03T01:20:55.843 INFO:tasks.workunit.client.0.smithi032.stdout:1/34: rename c2 to cb 0 2024-06-03T01:20:55.843 INFO:tasks.workunit.client.1.smithi044.stdout:4/575: unlink c6 0 2024-06-03T01:20:55.843 INFO:tasks.workunit.client.1.smithi044.stdout:1/602: mkdir d0/d3/d20/d7/d13/d2d/dd9 0 2024-06-03T01:20:55.844 INFO:tasks.workunit.client.1.smithi044.stdout:6/679: dwrite d2/d23/d55/d5f/d53/f7e [0,4194304] 0 2024-06-03T01:20:55.844 INFO:tasks.workunit.client.0.smithi032.stdout:0/145: getdents d1/da 0 2024-06-03T01:20:55.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/576: symlink d17/d63/lc2 0 2024-06-03T01:20:55.849 INFO:tasks.workunit.client.1.smithi044.stdout:4/577: chown d17 431 1 2024-06-03T01:20:55.849 INFO:tasks.workunit.client.1.smithi044.stdout:1/603: mknod d0/d3/d20/d40/cda 0 2024-06-03T01:20:55.850 INFO:tasks.workunit.client.0.smithi032.stdout:0/146: mkdir d1/da/d10/d29 0 2024-06-03T01:20:55.850 INFO:tasks.workunit.client.1.smithi044.stdout:8/561: dread d5/d20/d33/f4b [0,4194304] 0 2024-06-03T01:20:55.851 INFO:tasks.workunit.client.0.smithi032.stdout:1/35: link l4 lc 0 2024-06-03T01:20:55.851 INFO:tasks.workunit.client.0.smithi032.stdout:1/36: write fa [428478,28079] 0 2024-06-03T01:20:55.854 INFO:tasks.workunit.client.0.smithi032.stdout:1/37: fdatasync f5 0 2024-06-03T01:20:55.854 INFO:tasks.workunit.client.0.smithi032.stdout:1/38: fsync f5 0 2024-06-03T01:20:55.855 INFO:tasks.workunit.client.0.smithi032.stdout:1/39: mknod cd 0 2024-06-03T01:20:55.857 INFO:tasks.workunit.client.1.smithi044.stdout:6/680: dread d2/d23/d55/d5f/f59 [0,4194304] 0 2024-06-03T01:20:55.858 INFO:tasks.workunit.client.0.smithi032.stdout:1/40: mknod ce 0 2024-06-03T01:20:55.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/147: write d1/fc [3817665,23006] 0 2024-06-03T01:20:55.859 INFO:tasks.workunit.client.0.smithi032.stdout:0/148: getdents d1/da 0 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/681: mknod d2/d23/d30/d1e/ced 0 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/682: chown d2/d23/d30/d1e/d38 247804565 1 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/683: write d2/d23/d30/d1e/fbc [5175525,15109] 0 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/684: stat d2/d23/d30/d1e/d34/d39/dcd/dd1/leb 0 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/685: dread - d2/d23/d44/d4d/da0/fae zero size 2024-06-03T01:20:55.860 INFO:tasks.workunit.client.1.smithi044.stdout:4/578: dread d17/d1c/d75/d7a/db8/d59/f5f [0,4194304] 0 2024-06-03T01:20:55.861 INFO:tasks.workunit.client.0.smithi032.stdout:0/149: unlink d1/da/l14 0 2024-06-03T01:20:55.861 INFO:tasks.workunit.client.0.smithi032.stdout:0/150: write d1/da/f16 [493910,47146] 0 2024-06-03T01:20:55.861 INFO:tasks.workunit.client.0.smithi032.stdout:0/151: write d1/f6 [288514,105317] 0 2024-06-03T01:20:55.862 INFO:tasks.workunit.client.0.smithi032.stdout:0/152: creat d1/da/d10/f2a x:0 0 0 2024-06-03T01:20:55.862 INFO:tasks.workunit.client.0.smithi032.stdout:0/153: write d1/f6 [2230038,52623] 0 2024-06-03T01:20:55.862 INFO:tasks.workunit.client.0.smithi032.stdout:0/154: write d1/f18 [4546667,3552] 0 2024-06-03T01:20:55.863 INFO:tasks.workunit.client.1.smithi044.stdout:6/686: getdents d2/d23 0 2024-06-03T01:20:55.866 INFO:tasks.workunit.client.1.smithi044.stdout:6/687: mkdir d2/d23/d30/d1e/d34/d39/d7d/da7/db1/dee 0 2024-06-03T01:20:55.866 INFO:tasks.workunit.client.1.smithi044.stdout:4/579: dread d17/d1c/d75/d7a/db8/d59/f7b [0,4194304] 0 2024-06-03T01:20:55.867 INFO:tasks.workunit.client.1.smithi044.stdout:4/580: fdatasync d17/d1c/d75/d7a/db8/d28/d2b/f34 0 2024-06-03T01:20:55.867 INFO:tasks.workunit.client.1.smithi044.stdout:6/688: rmdir d2/d23/d30/d1e/d38/d6c/dc2 39 2024-06-03T01:20:55.869 INFO:tasks.workunit.client.0.smithi032.stdout:5/91: dwrite da/dd/df/d10/f14 [0,4194304] 0 2024-06-03T01:20:55.869 INFO:tasks.workunit.client.1.smithi044.stdout:4/581: creat d17/d1c/d75/d7a/db8/db9/fc3 x:0 0 0 2024-06-03T01:20:55.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/92: symlink da/l1e 0 2024-06-03T01:20:55.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/93: truncate da/f15 81411 0 2024-06-03T01:20:55.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/94: truncate da/f12 4563802 0 2024-06-03T01:20:55.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/95: fdatasync da/dd/df/d10/f14 0 2024-06-03T01:20:55.872 INFO:tasks.workunit.client.1.smithi044.stdout:4/582: dread d17/d1c/d75/d7a/db8/d59/fb5 [0,4194304] 0 2024-06-03T01:20:55.872 INFO:tasks.workunit.client.1.smithi044.stdout:4/583: chown d17/d1c/d75/d7a/db3/f85 0 1 2024-06-03T01:20:55.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/689: rename d2/d23/d55/d5f/db2/d3f/c8c to d2/d23/d30/d1e/d34/d39/dcd/cef 0 2024-06-03T01:20:55.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/690: chown d2/d23/d30/d1e/fbc 8 1 2024-06-03T01:20:55.873 INFO:tasks.workunit.client.0.smithi032.stdout:5/96: link da/l1e da/dd/df/d10/l1f 0 2024-06-03T01:20:55.873 INFO:tasks.workunit.client.0.smithi032.stdout:5/97: fsync da/dd/df/f1c 0 2024-06-03T01:20:55.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/562: dwrite d5/d20/d47/f71 [0,4194304] 0 2024-06-03T01:20:55.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/563: chown d5/f2d 86 1 2024-06-03T01:20:55.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/564: write d5/d55/d74/d87/f99 [769541,66707] 0 2024-06-03T01:20:55.874 INFO:tasks.workunit.client.1.smithi044.stdout:6/691: creat d2/ff0 x:0 0 0 2024-06-03T01:20:55.875 INFO:tasks.workunit.client.0.smithi032.stdout:1/41: fdatasync fa 0 2024-06-03T01:20:55.876 INFO:tasks.workunit.client.1.smithi044.stdout:1/604: dwrite d0/d66/dbe/fc5 [0,4194304] 0 2024-06-03T01:20:55.878 INFO:tasks.workunit.client.1.smithi044.stdout:4/584: rename d17/d1c/d75/f5c to d17/d1c/d75/d7a/db8/d8d/fc4 0 2024-06-03T01:20:55.882 INFO:tasks.workunit.client.0.smithi032.stdout:5/98: write da/dd/fe [779819,77126] 0 2024-06-03T01:20:55.885 INFO:tasks.workunit.client.0.smithi032.stdout:5/99: mknod da/dd/df/d10/d16/d18/d1a/c20 0 2024-06-03T01:20:55.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/565: dread d5/d20/d33/f3f [0,4194304] 0 2024-06-03T01:20:55.887 INFO:tasks.workunit.client.0.smithi032.stdout:0/155: dread d1/f18 [0,4194304] 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/605: mknod d0/d3/d20/d40/dcb/cdb 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/606: stat d0/d3/d20/d7/f34 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/607: creat d0/d66/fdc x:0 0 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/608: creat d0/d3/d20/d40/fdd x:0 0 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/609: creat d0/d3/d20/d7/d13/fde x:0 0 0 2024-06-03T01:20:55.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/610: fdatasync d0/d3/d20/d7/d2f/db7/fc6 0 2024-06-03T01:20:55.890 INFO:tasks.workunit.client.0.smithi032.stdout:5/100: mknod da/dd/df/d10/d16/c21 0 2024-06-03T01:20:55.890 INFO:tasks.workunit.client.1.smithi044.stdout:4/585: symlink d17/d1c/d7e/lc5 0 2024-06-03T01:20:55.891 INFO:tasks.workunit.client.0.smithi032.stdout:1/42: dwrite fa [0,4194304] 0 2024-06-03T01:20:55.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/566: symlink d5/d50/d79/lb2 0 2024-06-03T01:20:55.892 INFO:tasks.workunit.client.0.smithi032.stdout:5/101: unlink da/dd/df/d10/d16/c21 0 2024-06-03T01:20:55.895 INFO:tasks.workunit.client.0.smithi032.stdout:1/43: symlink lf 0 2024-06-03T01:20:55.896 INFO:tasks.workunit.client.0.smithi032.stdout:0/156: dread d1/f17 [0,4194304] 0 2024-06-03T01:20:55.896 INFO:tasks.workunit.client.0.smithi032.stdout:5/102: getdents da/dd/df/d10/d16/d18 0 2024-06-03T01:20:55.896 INFO:tasks.workunit.client.0.smithi032.stdout:5/103: fsync da/dd/df/d10/d16/d18/f1d 0 2024-06-03T01:20:55.898 INFO:tasks.workunit.client.1.smithi044.stdout:1/611: truncate d0/d3/d14/f50 1433404 0 2024-06-03T01:20:55.900 INFO:tasks.workunit.client.1.smithi044.stdout:4/586: mknod d17/d1c/d75/d7a/db8/d28/d2b/d8e/cc6 0 2024-06-03T01:20:55.900 INFO:tasks.workunit.client.1.smithi044.stdout:4/587: fsync d17/d1c/d7e/fa8 0 2024-06-03T01:20:55.901 INFO:tasks.workunit.client.0.smithi032.stdout:1/44: mknod c10 0 2024-06-03T01:20:55.901 INFO:tasks.workunit.client.1.smithi044.stdout:6/692: dwrite d2/d23/d30/d1e/d34/d39/d64/f9e [0,4194304] 0 2024-06-03T01:20:55.901 INFO:tasks.workunit.client.1.smithi044.stdout:6/693: write d2/d23/d30/d1e/d38/f78 [728633,1891] 0 2024-06-03T01:20:55.902 INFO:tasks.workunit.client.1.smithi044.stdout:8/567: mkdir d5/d20/db3 0 2024-06-03T01:20:55.907 INFO:tasks.workunit.client.1.smithi044.stdout:1/612: link d0/d16/f67 d0/d3/d20/fdf 0 2024-06-03T01:20:55.907 INFO:tasks.workunit.client.0.smithi032.stdout:1/45: rename f5 to f11 0 2024-06-03T01:20:55.907 INFO:tasks.workunit.client.0.smithi032.stdout:1/46: truncate fa 5045010 0 2024-06-03T01:20:55.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/588: symlink d17/d1c/d75/d7a/db8/lc7 0 2024-06-03T01:20:55.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/589: stat d17/d63/f65 0 2024-06-03T01:20:55.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/590: creat d17/d1c/d75/d7a/db8/db9/fc8 x:0 0 0 2024-06-03T01:20:55.911 INFO:tasks.workunit.client.0.smithi032.stdout:1/47: creat f12 x:0 0 0 2024-06-03T01:20:55.917 INFO:tasks.workunit.client.0.smithi032.stdout:1/48: rename lc to l13 0 2024-06-03T01:20:55.918 INFO:tasks.workunit.client.1.smithi044.stdout:6/694: rename d2/d23/d30/f74 to d2/d23/d55/ff1 0 2024-06-03T01:20:55.919 INFO:tasks.workunit.client.0.smithi032.stdout:1/49: creat f14 x:0 0 0 2024-06-03T01:20:55.922 INFO:tasks.workunit.client.0.smithi032.stdout:5/104: truncate da/f15 79756 0 2024-06-03T01:20:55.923 INFO:tasks.workunit.client.0.smithi032.stdout:0/157: dwrite d1/da/d10/f19 [0,4194304] 0 2024-06-03T01:20:55.925 INFO:tasks.workunit.client.0.smithi032.stdout:5/105: symlink da/dd/df/l22 0 2024-06-03T01:20:55.925 INFO:tasks.workunit.client.1.smithi044.stdout:1/613: unlink d0/d3/d20/fdf 0 2024-06-03T01:20:55.925 INFO:tasks.workunit.client.1.smithi044.stdout:1/614: fdatasync d0/d3/d20/f86 0 2024-06-03T01:20:55.925 INFO:tasks.workunit.client.1.smithi044.stdout:1/615: chown d0/d66/f65 1018016 1 2024-06-03T01:20:55.925 INFO:tasks.workunit.client.0.smithi032.stdout:5/106: dread da/f17 [0,4194304] 0 2024-06-03T01:20:55.926 INFO:tasks.workunit.client.0.smithi032.stdout:5/107: stat da/dd/df 0 2024-06-03T01:20:55.926 INFO:tasks.workunit.client.1.smithi044.stdout:8/568: dwrite d5/d20/d2b/d58/f84 [0,4194304] 0 2024-06-03T01:20:55.926 INFO:tasks.workunit.client.0.smithi032.stdout:0/158: creat d1/f2b x:0 0 0 2024-06-03T01:20:55.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/591: creat d17/d1c/d75/d7a/db8/d28/d2b/d8e/fc9 x:0 0 0 2024-06-03T01:20:55.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/592: truncate d17/d1c/d75/f62 1595429 0 2024-06-03T01:20:55.929 INFO:tasks.workunit.client.0.smithi032.stdout:1/50: unlink f11 0 2024-06-03T01:20:55.929 INFO:tasks.workunit.client.0.smithi032.stdout:1/51: dread - f12 zero size 2024-06-03T01:20:55.929 INFO:tasks.workunit.client.1.smithi044.stdout:6/695: dread d2/d23/d30/d1e/d34/d39/f56 [0,4194304] 0 2024-06-03T01:20:55.929 INFO:tasks.workunit.client.1.smithi044.stdout:6/696: readlink d2/d23/d30/d1e/d38/d6c/l1b 0 2024-06-03T01:20:55.930 INFO:tasks.workunit.client.1.smithi044.stdout:1/616: mkdir d0/d3/d20/d7/d2f/de0 0 2024-06-03T01:20:55.930 INFO:tasks.workunit.client.1.smithi044.stdout:8/569: read d5/d20/d2b/d98/fa7 [110760,26776] 0 2024-06-03T01:20:55.931 INFO:tasks.workunit.client.0.smithi032.stdout:5/108: symlink da/dd/df/l23 0 2024-06-03T01:20:55.935 INFO:tasks.workunit.client.0.smithi032.stdout:0/159: truncate d1/f1b 1824096 0 2024-06-03T01:20:55.936 INFO:tasks.workunit.client.0.smithi032.stdout:5/109: mknod da/dd/df/d10/c24 0 2024-06-03T01:20:55.937 INFO:tasks.workunit.client.1.smithi044.stdout:4/593: creat d17/d1c/d75/d7a/db8/fca x:0 0 0 2024-06-03T01:20:55.938 INFO:tasks.workunit.client.1.smithi044.stdout:6/697: symlink d2/d23/d55/d5f/db2/d3f/lf2 0 2024-06-03T01:20:55.939 INFO:tasks.workunit.client.1.smithi044.stdout:6/698: getdents d2/d23/d30/ddf 0 2024-06-03T01:20:55.939 INFO:tasks.workunit.client.1.smithi044.stdout:6/699: getdents d2/d23/d30/d1e/d34/d39/d7d/da7/db1/dee 0 2024-06-03T01:20:55.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/110: mkdir da/dd/df/d10/d16/d25 0 2024-06-03T01:20:55.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/111: write da/dd/df/f1c [5389895,71999] 0 2024-06-03T01:20:55.941 INFO:tasks.workunit.client.0.smithi032.stdout:0/160: mknod d1/da/d10/c2c 0 2024-06-03T01:20:55.942 INFO:tasks.workunit.client.0.smithi032.stdout:5/112: mknod da/c26 0 2024-06-03T01:20:55.942 INFO:tasks.workunit.client.0.smithi032.stdout:5/113: chown da/dd/df/d10/d16/d25 783 1 2024-06-03T01:20:55.943 INFO:tasks.workunit.client.1.smithi044.stdout:8/570: mkdir d5/d55/d74/db4 0 2024-06-03T01:20:55.944 INFO:tasks.workunit.client.0.smithi032.stdout:5/114: write f4 [4851569,45420] 0 2024-06-03T01:20:55.945 INFO:tasks.workunit.client.0.smithi032.stdout:5/115: dread da/f17 [0,4194304] 0 2024-06-03T01:20:55.946 INFO:tasks.workunit.client.0.smithi032.stdout:1/52: dwrite f12 [0,4194304] 0 2024-06-03T01:20:55.946 INFO:tasks.workunit.client.0.smithi032.stdout:1/53: write f14 [436728,54176] 0 2024-06-03T01:20:55.947 INFO:tasks.workunit.client.1.smithi044.stdout:6/700: mkdir d2/d23/d55/d5f/d99/dcc/dd4/df3 0 2024-06-03T01:20:55.949 INFO:tasks.workunit.client.1.smithi044.stdout:1/617: getdents d0/d66/dbe/d9e 0 2024-06-03T01:20:55.949 INFO:tasks.workunit.client.1.smithi044.stdout:1/618: chown d0/d16/d64/c87 225087 1 2024-06-03T01:20:55.949 INFO:tasks.workunit.client.1.smithi044.stdout:1/619: chown d0/d3/d20/d40/d45/d4b/d5a/f72 666 1 2024-06-03T01:20:55.949 INFO:tasks.workunit.client.1.smithi044.stdout:1/620: write d0/d3/d20/d40/fdd [830498,49912] 0 2024-06-03T01:20:55.950 INFO:tasks.workunit.client.1.smithi044.stdout:4/594: dread d17/d1c/d75/d7a/db8/d28/d2b/f1b [0,4194304] 0 2024-06-03T01:20:55.953 INFO:tasks.workunit.client.0.smithi032.stdout:1/54: link f14 f15 0 2024-06-03T01:20:55.953 INFO:tasks.workunit.client.1.smithi044.stdout:1/621: mkdir d0/d3/de1 0 2024-06-03T01:20:55.955 INFO:tasks.workunit.client.1.smithi044.stdout:0/635: sync 2024-06-03T01:20:55.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/161: rmdir d1/da 39 2024-06-03T01:20:55.957 INFO:tasks.workunit.client.1.smithi044.stdout:4/595: dread d17/d1c/d75/d7a/db3/f96 [0,4194304] 0 2024-06-03T01:20:55.957 INFO:tasks.workunit.client.1.smithi044.stdout:0/636: mkdir d3/d6b/de8 0 2024-06-03T01:20:55.957 INFO:tasks.workunit.client.1.smithi044.stdout:4/596: chown d17/d1c/d75/d7a/db8/d59/f7b 197538795 1 2024-06-03T01:20:55.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/162: readlink d1/da/d10/l15 0 2024-06-03T01:20:55.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/163: creat d1/da/d10/f2d x:0 0 0 2024-06-03T01:20:55.961 INFO:tasks.workunit.client.0.smithi032.stdout:0/164: link d1/da/d10/l15 d1/da/d10/l2e 0 2024-06-03T01:20:55.963 INFO:tasks.workunit.client.0.smithi032.stdout:1/55: truncate fa 3542503 0 2024-06-03T01:20:55.968 INFO:tasks.workunit.client.1.smithi044.stdout:3/416: sync 2024-06-03T01:20:55.969 INFO:tasks.workunit.client.1.smithi044.stdout:2/424: sync 2024-06-03T01:20:55.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/571: dwrite d5/d16/f7c [4194304,4194304] 0 2024-06-03T01:20:55.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/572: write d5/d55/d74/d87/f70 [1364074,72880] 0 2024-06-03T01:20:55.972 INFO:tasks.workunit.client.1.smithi044.stdout:8/573: chown d5/d10/c1a 58315 1 2024-06-03T01:20:55.972 INFO:tasks.workunit.client.1.smithi044.stdout:3/417: creat df/d1e/d3a/f87 x:0 0 0 2024-06-03T01:20:55.972 INFO:tasks.workunit.client.1.smithi044.stdout:3/418: creat df/d1e/d3a/f88 x:0 0 0 2024-06-03T01:20:55.972 INFO:tasks.workunit.client.1.smithi044.stdout:2/425: getdents db/d15/d49/d7b 0 2024-06-03T01:20:55.972 INFO:tasks.workunit.client.1.smithi044.stdout:2/426: getdents db/d15/d49 0 2024-06-03T01:20:55.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/574: mkdir d5/d55/db5 0 2024-06-03T01:20:55.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/575: truncate d5/d16/f3a 1191021 0 2024-06-03T01:20:55.975 INFO:tasks.workunit.client.1.smithi044.stdout:1/622: write d0/d3/d20/d7/d13/f24 [946446,76933] 0 2024-06-03T01:20:55.976 INFO:tasks.workunit.client.1.smithi044.stdout:2/427: truncate db/d1b/d5d/f62 6027165 0 2024-06-03T01:20:55.977 INFO:tasks.workunit.client.1.smithi044.stdout:1/623: dread d0/dd2/f78 [4194304,4194304] 0 2024-06-03T01:20:55.980 INFO:tasks.workunit.client.1.smithi044.stdout:2/428: creat db/d15/d49/d6c/d72/d7a/f8d x:0 0 0 2024-06-03T01:20:55.980 INFO:tasks.workunit.client.1.smithi044.stdout:2/429: write db/d15/d49/d6c/d72/d7a/f68 [390456,120565] 0 2024-06-03T01:20:55.980 INFO:tasks.workunit.client.1.smithi044.stdout:2/430: write db/d1b/d2a/d52/d54/f56 [264571,20144] 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/597: dread d17/d1c/d75/d7a/db8/d28/d2b/f21 [0,4194304] 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:2/431: creat db/d15/d49/f8e x:0 0 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:2/432: readlink db/dd/l16 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:2/433: stat db/d1b/d2a/d2f/d4c/f77 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/598: symlink d17/d1c/d75/d7a/db8/d6a/lcb 0 2024-06-03T01:20:55.984 INFO:tasks.workunit.client.1.smithi044.stdout:3/419: dread df/d13/d39/d3d/d43/f5b [0,4194304] 0 2024-06-03T01:20:55.986 INFO:tasks.workunit.client.1.smithi044.stdout:3/420: getdents df/d13/d39/d3d/d43/d72 0 2024-06-03T01:20:55.986 INFO:tasks.workunit.client.1.smithi044.stdout:3/421: chown df/d13/d1b/d4b/f54 1801534766 1 2024-06-03T01:20:55.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/599: mknod d17/d1c/ccc 0 2024-06-03T01:20:55.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/600: write d17/d1c/d75/d7a/db3/f95 [803716,102832] 0 2024-06-03T01:20:55.987 INFO:tasks.workunit.client.1.smithi044.stdout:3/422: getdents df/d13/d1b/d59 0 2024-06-03T01:20:55.988 INFO:tasks.workunit.client.1.smithi044.stdout:4/601: symlink d17/d63/dae/lcd 0 2024-06-03T01:20:55.992 INFO:tasks.workunit.client.1.smithi044.stdout:3/423: creat df/d1e/d2a/f89 x:0 0 0 2024-06-03T01:20:55.992 INFO:tasks.workunit.client.1.smithi044.stdout:3/424: readlink df/d13/d39/d3d/d43/l63 0 2024-06-03T01:20:55.992 INFO:tasks.workunit.client.1.smithi044.stdout:3/425: fsync df/d13/d1b/d4b/d69/f75 0 2024-06-03T01:20:55.993 INFO:tasks.workunit.client.1.smithi044.stdout:4/602: creat d17/d1c/d75/d7a/db8/d35/fce x:0 0 0 2024-06-03T01:20:55.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/426: rename df/d13/d1b/d56/d7b to df/d1e/d8a 0 2024-06-03T01:20:55.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/427: write df/d13/d1b/f73 [610445,59847] 0 2024-06-03T01:20:55.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/428: read df/f19 [932544,44675] 0 2024-06-03T01:20:55.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/429: truncate df/d13/d1b/d4b/f54 4377473 0 2024-06-03T01:20:55.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/430: creat df/d13/d39/d3d/f8b x:0 0 0 2024-06-03T01:20:55.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/165: dwrite d1/f18 [4194304,4194304] 0 2024-06-03T01:20:55.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/166: chown d1/da 966391146 1 2024-06-03T01:20:55.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/167: truncate d1/f21 640813 0 2024-06-03T01:20:56.000 INFO:tasks.workunit.client.0.smithi032.stdout:1/56: fdatasync f14 0 2024-06-03T01:20:56.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/431: mknod df/d13/d39/d3d/d43/d72/c8c 0 2024-06-03T01:20:56.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/432: dread - df/d13/d1b/d56/f6c zero size 2024-06-03T01:20:56.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/433: chown df/d13/d1b/f5e 500 1 2024-06-03T01:20:56.004 INFO:tasks.workunit.client.0.smithi032.stdout:1/57: dread f14 [0,4194304] 0 2024-06-03T01:20:56.006 INFO:tasks.workunit.client.1.smithi044.stdout:3/434: creat df/d13/d1b/d56/d5c/f8d x:0 0 0 2024-06-03T01:20:56.009 INFO:tasks.workunit.client.1.smithi044.stdout:3/435: mkdir df/d13/d1b/d56/d8e 0 2024-06-03T01:20:56.011 INFO:tasks.workunit.client.1.smithi044.stdout:3/436: mknod df/d1e/d3a/d84/c8f 0 2024-06-03T01:20:56.013 INFO:tasks.workunit.client.0.smithi032.stdout:5/116: dwrite da/dd/df/d10/f14 [4194304,4194304] 0 2024-06-03T01:20:56.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/437: write df/d13/f24 [3028218,101210] 0 2024-06-03T01:20:56.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/438: write df/d13/d1b/d4b/d69/f76 [2052362,97042] 0 2024-06-03T01:20:56.024 INFO:tasks.workunit.client.1.smithi044.stdout:4/603: dwrite d17/d1c/d75/d7a/db8/d35/f4f [0,4194304] 0 2024-06-03T01:20:56.026 INFO:tasks.workunit.client.1.smithi044.stdout:4/604: dread d17/d1c/d75/d7a/db8/d59/f7b [0,4194304] 0 2024-06-03T01:20:56.029 INFO:tasks.workunit.client.1.smithi044.stdout:4/605: mknod d17/d63/dae/ccf 0 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: pgmap v257: 65 pgs: 65 active+clean; 1.1 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 101 MiB/s rd, 141 MiB/s wr, 818 op/s 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:20:56.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:55 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.032 INFO:tasks.workunit.client.1.smithi044.stdout:1/624: dwrite d0/dd2/f57 [0,4194304] 0 2024-06-03T01:20:56.032 INFO:tasks.workunit.client.1.smithi044.stdout:1/625: read - d0/d3/d20/d40/d45/d4b/d5c/d94/fb3 zero size 2024-06-03T01:20:56.032 INFO:tasks.workunit.client.1.smithi044.stdout:1/626: fsync d0/d3/d20/d40/d45/d6f/f98 0 2024-06-03T01:20:56.033 INFO:tasks.workunit.client.1.smithi044.stdout:2/434: dwrite db/d15/d49/d6c/d72/d7a/f81 [0,4194304] 0 2024-06-03T01:20:56.033 INFO:tasks.workunit.client.1.smithi044.stdout:4/606: link d17/d1c/d75/d7a/db8/d28/d2b/f4a d17/d63/fd0 0 2024-06-03T01:20:56.033 INFO:tasks.workunit.client.1.smithi044.stdout:4/607: fdatasync d17/d1c/d75/d7a/db8/d35/fce 0 2024-06-03T01:20:56.033 INFO:tasks.workunit.client.1.smithi044.stdout:8/576: dwrite d5/d16/f3c [0,4194304] 0 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:2/134: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:4/91: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:3/110: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:6/108: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:9/228: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/75: sync 2024-06-03T01:20:56.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/76: read - dc/de/f11 zero size 2024-06-03T01:20:56.036 INFO:tasks.workunit.client.0.smithi032.stdout:7/103: sync 2024-06-03T01:20:56.036 INFO:tasks.workunit.client.1.smithi044.stdout:2/435: truncate db/d1b/f36 364540 0 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: pgmap v257: 65 pgs: 65 active+clean; 1.1 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 101 MiB/s rd, 141 MiB/s wr, 818 op/s 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:20:56.037 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:55 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:20:56.038 INFO:tasks.workunit.client.1.smithi044.stdout:4/608: creat d17/d1c/d75/d7a/db8/d28/fd1 x:0 0 0 2024-06-03T01:20:56.038 INFO:tasks.workunit.client.1.smithi044.stdout:4/609: readlink d17/d1c/d75/d7a/db3/l72 0 2024-06-03T01:20:56.038 INFO:tasks.workunit.client.1.smithi044.stdout:1/627: dread d0/d3/d20/f6d [0,4194304] 0 2024-06-03T01:20:56.039 INFO:tasks.workunit.client.0.smithi032.stdout:2/135: unlink c4 0 2024-06-03T01:20:56.042 INFO:tasks.workunit.client.0.smithi032.stdout:6/109: mkdir d8/d11/d20 0 2024-06-03T01:20:56.042 INFO:tasks.workunit.client.0.smithi032.stdout:8/77: write dc/de/f10 [706977,57762] 0 2024-06-03T01:20:56.043 INFO:tasks.workunit.client.0.smithi032.stdout:8/78: readlink l3 0 2024-06-03T01:20:56.043 INFO:tasks.workunit.client.0.smithi032.stdout:8/79: truncate f8 2329416 0 2024-06-03T01:20:56.043 INFO:tasks.workunit.client.0.smithi032.stdout:8/80: write fa [5212875,33127] 0 2024-06-03T01:20:56.043 INFO:tasks.workunit.client.0.smithi032.stdout:8/81: fsync dc/de/f11 0 2024-06-03T01:20:56.043 INFO:tasks.workunit.client.0.smithi032.stdout:2/136: dread d5/fe [0,4194304] 0 2024-06-03T01:20:56.046 INFO:tasks.workunit.client.0.smithi032.stdout:6/110: mkdir d8/d11/d20/d21 0 2024-06-03T01:20:56.046 INFO:tasks.workunit.client.0.smithi032.stdout:6/111: truncate d8/f9 4394540 0 2024-06-03T01:20:56.049 INFO:tasks.workunit.client.0.smithi032.stdout:1/58: dwrite fa [0,4194304] 0 2024-06-03T01:20:56.049 INFO:tasks.workunit.client.0.smithi032.stdout:1/59: write f15 [1296280,41342] 0 2024-06-03T01:20:56.051 INFO:tasks.workunit.client.1.smithi044.stdout:2/436: mkdir db/d15/d49/d6c/d72/d8f 0 2024-06-03T01:20:56.051 INFO:tasks.workunit.client.0.smithi032.stdout:8/82: link f9 dc/f15 0 2024-06-03T01:20:56.051 INFO:tasks.workunit.client.0.smithi032.stdout:8/83: chown dc/de/f10 0 1 2024-06-03T01:20:56.052 INFO:tasks.workunit.client.1.smithi044.stdout:1/628: dread d0/d3/d20/d7/d2f/db7/fc6 [0,4194304] 0 2024-06-03T01:20:56.053 INFO:tasks.workunit.client.0.smithi032.stdout:2/137: unlink d5/l6 0 2024-06-03T01:20:56.054 INFO:tasks.workunit.client.1.smithi044.stdout:3/439: rmdir df/d13/d1b 39 2024-06-03T01:20:56.054 INFO:tasks.workunit.client.0.smithi032.stdout:6/112: mknod d8/d18/c22 0 2024-06-03T01:20:56.054 INFO:tasks.workunit.client.0.smithi032.stdout:6/113: read d8/fd [670733,61867] 0 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/440: dread df/d13/f16 [0,4194304] 0 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/441: dread - df/d13/d39/d3d/d43/f55 zero size 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/442: fsync df/d13/d39/d3d/f44 0 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/443: chown df/d13/d39/d3d/d43/f55 710 1 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/444: getdents df/d13/d39/d3d/d50 0 2024-06-03T01:20:56.056 INFO:tasks.workunit.client.1.smithi044.stdout:3/445: readlink df/d13/d1b/l41 0 2024-06-03T01:20:56.057 INFO:tasks.workunit.client.0.smithi032.stdout:6/114: link c2 d8/d11/d20/c23 0 2024-06-03T01:20:56.058 INFO:tasks.workunit.client.0.smithi032.stdout:8/84: rmdir dc/de 39 2024-06-03T01:20:56.059 INFO:tasks.workunit.client.0.smithi032.stdout:1/60: dread f14 [0,4194304] 0 2024-06-03T01:20:56.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/115: creat d8/d11/d20/d21/f24 x:0 0 0 2024-06-03T01:20:56.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/116: stat d8/la 0 2024-06-03T01:20:56.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/117: read d8/fd [1465276,35312] 0 2024-06-03T01:20:56.063 INFO:tasks.workunit.client.0.smithi032.stdout:5/117: dread da/dd/df/f1c [0,4194304] 0 2024-06-03T01:20:56.064 INFO:tasks.workunit.client.0.smithi032.stdout:8/85: rmdir dc 39 2024-06-03T01:20:56.065 INFO:tasks.workunit.client.1.smithi044.stdout:2/437: read db/f19 [2821489,96602] 0 2024-06-03T01:20:56.065 INFO:tasks.workunit.client.1.smithi044.stdout:2/438: write db/f48 [2540353,21000] 0 2024-06-03T01:20:56.065 INFO:tasks.workunit.client.1.smithi044.stdout:2/439: stat db/d1b/d5d/l78 0 2024-06-03T01:20:56.067 INFO:tasks.workunit.client.0.smithi032.stdout:1/61: unlink f14 0 2024-06-03T01:20:56.070 INFO:tasks.workunit.client.0.smithi032.stdout:4/92: write d4/db/fc [3833963,72400] 0 2024-06-03T01:20:56.070 INFO:tasks.workunit.client.0.smithi032.stdout:4/93: read d4/db/fc [1099526,32647] 0 2024-06-03T01:20:56.072 INFO:tasks.workunit.client.1.smithi044.stdout:8/577: rename d5/la to d5/d50/d79/d88/lb6 0 2024-06-03T01:20:56.072 INFO:tasks.workunit.client.1.smithi044.stdout:1/629: symlink d0/d66/dbe/le2 0 2024-06-03T01:20:56.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/86: link dc/cf dc/de/c16 0 2024-06-03T01:20:56.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/87: fsync dc/de/f11 0 2024-06-03T01:20:56.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/88: getdents dc/d14 0 2024-06-03T01:20:56.079 INFO:tasks.workunit.client.0.smithi032.stdout:1/62: symlink l16 0 2024-06-03T01:20:56.080 INFO:tasks.workunit.client.0.smithi032.stdout:4/94: symlink d4/db/da/d1d/l22 0 2024-06-03T01:20:56.080 INFO:tasks.workunit.client.0.smithi032.stdout:4/95: write d4/db/da/fd [833167,116609] 0 2024-06-03T01:20:56.085 INFO:tasks.workunit.client.0.smithi032.stdout:8/89: symlink dc/l17 0 2024-06-03T01:20:56.086 INFO:tasks.workunit.client.1.smithi044.stdout:3/446: mkdir df/d1e/d3a/d84/d90 0 2024-06-03T01:20:56.096 INFO:tasks.workunit.client.0.smithi032.stdout:4/96: link d4/f7 d4/db/da/d1d/f23 0 2024-06-03T01:20:56.099 INFO:tasks.workunit.client.1.smithi044.stdout:2/440: symlink db/dd/l90 0 2024-06-03T01:20:56.100 INFO:tasks.workunit.client.0.smithi032.stdout:4/97: link d4/l5 d4/db/l24 0 2024-06-03T01:20:56.101 INFO:tasks.workunit.client.0.smithi032.stdout:1/63: dwrite fa [0,4194304] 0 2024-06-03T01:20:56.101 INFO:tasks.workunit.client.0.smithi032.stdout:1/64: stat lf 0 2024-06-03T01:20:56.101 INFO:tasks.workunit.client.0.smithi032.stdout:1/65: fsync fa 0 2024-06-03T01:20:56.104 INFO:tasks.workunit.client.1.smithi044.stdout:8/578: rename d5/d10/f34 to d5/d20/db3/fb7 0 2024-06-03T01:20:56.104 INFO:tasks.workunit.client.1.smithi044.stdout:8/579: write d5/d20/d33/fb1 [837331,33262] 0 2024-06-03T01:20:56.109 INFO:tasks.workunit.client.1.smithi044.stdout:5/523: sync 2024-06-03T01:20:56.109 INFO:tasks.workunit.client.1.smithi044.stdout:9/411: sync 2024-06-03T01:20:56.109 INFO:tasks.workunit.client.1.smithi044.stdout:7/606: sync 2024-06-03T01:20:56.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/524: dread d2/f3c [0,4194304] 0 2024-06-03T01:20:56.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/525: write d2/f97 [101395,14398] 0 2024-06-03T01:20:56.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/526: write d2/fa [299322,54311] 0 2024-06-03T01:20:56.112 INFO:tasks.workunit.client.0.smithi032.stdout:4/98: getdents d4/db/da/d1d 0 2024-06-03T01:20:56.112 INFO:tasks.workunit.client.0.smithi032.stdout:4/99: readlink d4/l5 0 2024-06-03T01:20:56.112 INFO:tasks.workunit.client.0.smithi032.stdout:4/100: chown d4/db/da/f1b 4 1 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/441: mkdir db/d1b/d27/d79/d91 0 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/442: read db/f48 [2010922,12806] 0 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/443: stat db/d15/f70 0 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/444: truncate db/d15/d49/d7b/f8a 108943 0 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/445: write db/d1b/d2a/f43 [604200,7176] 0 2024-06-03T01:20:56.113 INFO:tasks.workunit.client.1.smithi044.stdout:7/607: write d0/d16/d23/f24 [30421,61718] 0 2024-06-03T01:20:56.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/580: unlink d5/d10/f13 0 2024-06-03T01:20:56.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/581: fsync d5/d55/d74/d87/f7b 0 2024-06-03T01:20:56.116 INFO:tasks.workunit.client.0.smithi032.stdout:6/118: rename d8/d18 to d8/d25 0 2024-06-03T01:20:56.116 INFO:tasks.workunit.client.0.smithi032.stdout:6/119: truncate d8/d25/f1f 556822 0 2024-06-03T01:20:56.118 INFO:tasks.workunit.client.1.smithi044.stdout:3/447: rename df/f1c to df/d13/d39/d3d/d43/d72/f91 0 2024-06-03T01:20:56.118 INFO:tasks.workunit.client.0.smithi032.stdout:1/66: getdents . 0 2024-06-03T01:20:56.119 INFO:tasks.workunit.client.0.smithi032.stdout:4/101: mkdir d4/db/da/d1d/d25 0 2024-06-03T01:20:56.124 INFO:tasks.workunit.client.1.smithi044.stdout:9/412: link d6/dd/d51/d16/d61/f6c d6/dd/d51/f86 0 2024-06-03T01:20:56.124 INFO:tasks.workunit.client.1.smithi044.stdout:1/630: getdents d0/d3/d20/d40/d45/d4b/d5c 0 2024-06-03T01:20:56.125 INFO:tasks.workunit.client.1.smithi044.stdout:2/446: dread db/dd/f60 [0,4194304] 0 2024-06-03T01:20:56.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/527: symlink d2/d11/d12/la8 0 2024-06-03T01:20:56.126 INFO:tasks.workunit.client.0.smithi032.stdout:4/102: mknod d4/db/da/d1d/d25/c26 0 2024-06-03T01:20:56.128 INFO:tasks.workunit.client.0.smithi032.stdout:1/67: dread fa [0,4194304] 0 2024-06-03T01:20:56.133 INFO:tasks.workunit.client.1.smithi044.stdout:7/608: symlink d0/d16/d23/lc5 0 2024-06-03T01:20:56.133 INFO:tasks.workunit.client.1.smithi044.stdout:7/609: dread - d0/d3/d8/d95/d63/d7f/f82 zero size 2024-06-03T01:20:56.133 INFO:tasks.workunit.client.1.smithi044.stdout:9/413: write d6/dd/d51/d16/d4d/d55/d68/f70 [2611416,2913] 0 2024-06-03T01:20:56.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/414: write d6/dd/d51/d16/d27/f53 [2737869,99799] 0 2024-06-03T01:20:56.134 INFO:tasks.workunit.client.0.smithi032.stdout:6/120: rmdir d8/d11/d20/d21 39 2024-06-03T01:20:56.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/448: truncate df/d13/d1b/f34 2089159 0 2024-06-03T01:20:56.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/449: fsync df/d13/f29 0 2024-06-03T01:20:56.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/450: write df/d13/d39/d3d/f68 [773162,120075] 0 2024-06-03T01:20:56.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/451: chown df/d13 43542142 1 2024-06-03T01:20:56.137 INFO:tasks.workunit.client.0.smithi032.stdout:4/103: fdatasync d4/db/da/d1d/f1f 0 2024-06-03T01:20:56.140 INFO:tasks.workunit.client.0.smithi032.stdout:4/104: stat l2 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/582: rename d5/d20/l4d to d5/d20/d2b/d98/lb8 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:1/631: creat d0/d3/d20/d40/dcb/fe3 x:0 0 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/583: creat d5/d20/d2b/d58/d89/fb9 x:0 0 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:1/632: write d0/d66/f65 [4921688,59036] 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:1/633: readlink d0/d3/d20/d7/l73 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:2/447: creat db/d1b/d27/d79/d91/f92 x:0 0 0 2024-06-03T01:20:56.141 INFO:tasks.workunit.client.1.smithi044.stdout:7/610: rmdir d0/d3/d8/d95/d63/d89 39 2024-06-03T01:20:56.142 INFO:tasks.workunit.client.1.smithi044.stdout:9/415: creat d6/dd/f87 x:0 0 0 2024-06-03T01:20:56.142 INFO:tasks.workunit.client.1.smithi044.stdout:7/611: chown d0/f74 2804 1 2024-06-03T01:20:56.143 INFO:tasks.workunit.client.1.smithi044.stdout:3/452: write df/d13/f1a [371246,3482] 0 2024-06-03T01:20:56.143 INFO:tasks.workunit.client.1.smithi044.stdout:3/453: fsync df/f19 0 2024-06-03T01:20:56.143 INFO:tasks.workunit.client.1.smithi044.stdout:8/584: mkdir d5/d55/d74/db4/dba 0 2024-06-03T01:20:56.145 INFO:tasks.workunit.client.1.smithi044.stdout:1/634: symlink d0/d66/dbe/le4 0 2024-06-03T01:20:56.145 INFO:tasks.workunit.client.0.smithi032.stdout:4/105: unlink d4/db/fc 0 2024-06-03T01:20:56.148 INFO:tasks.workunit.client.0.smithi032.stdout:4/106: chown d4/db/da/d1d/f21 2 1 2024-06-03T01:20:56.148 INFO:tasks.workunit.client.0.smithi032.stdout:1/68: dread fa [0,4194304] 0 2024-06-03T01:20:56.149 INFO:tasks.workunit.client.1.smithi044.stdout:9/416: creat d6/dd/d51/d46/f88 x:0 0 0 2024-06-03T01:20:56.149 INFO:tasks.workunit.client.1.smithi044.stdout:9/417: stat d6/dd/d51/d16/d27/d5e/l69 0 2024-06-03T01:20:56.149 INFO:tasks.workunit.client.1.smithi044.stdout:7/612: symlink d0/d3/d8/d95/d63/d89/da3/d9a/lc6 0 2024-06-03T01:20:56.158 INFO:tasks.workunit.client.0.smithi032.stdout:1/69: dread f12 [0,4194304] 0 2024-06-03T01:20:56.163 INFO:tasks.workunit.client.0.smithi032.stdout:3/111: dwrite f5 [4194304,4194304] 0 2024-06-03T01:20:56.168 INFO:tasks.workunit.client.1.smithi044.stdout:7/613: mkdir d0/d3/d8/da0/dc7 0 2024-06-03T01:20:56.172 INFO:tasks.workunit.client.1.smithi044.stdout:9/418: write d6/dd/f17 [3286556,116551] 0 2024-06-03T01:20:56.178 INFO:tasks.workunit.client.1.smithi044.stdout:9/419: mkdir d6/dd/d51/d16/d27/d5e/d71/d89 0 2024-06-03T01:20:56.179 INFO:tasks.workunit.client.1.smithi044.stdout:8/585: rename d5/d50 to d5/d55/d56/d5c/dbb 0 2024-06-03T01:20:56.181 INFO:tasks.workunit.client.1.smithi044.stdout:9/420: chown d6/dd/d51/db/l25 2 1 2024-06-03T01:20:56.181 INFO:tasks.workunit.client.1.smithi044.stdout:9/421: read - d6/f65 zero size 2024-06-03T01:20:56.181 INFO:tasks.workunit.client.1.smithi044.stdout:9/422: truncate d6/dd/d51/d46/f76 436537 0 2024-06-03T01:20:56.183 INFO:tasks.workunit.client.1.smithi044.stdout:9/423: write d6/dd/d51/d16/d27/f39 [1481764,26653] 0 2024-06-03T01:20:56.183 INFO:tasks.workunit.client.1.smithi044.stdout:9/424: chown d6/dd/d51/d16/d4d/d55/l6b 11529714 1 2024-06-03T01:20:56.183 INFO:tasks.workunit.client.1.smithi044.stdout:7/614: dread d0/d3/f47 [0,4194304] 0 2024-06-03T01:20:56.183 INFO:tasks.workunit.client.1.smithi044.stdout:8/586: mknod d5/d20/d33/cbc 0 2024-06-03T01:20:56.184 INFO:tasks.workunit.client.1.smithi044.stdout:8/587: fdatasync d5/d20/db3/fb7 0 2024-06-03T01:20:56.184 INFO:tasks.workunit.client.1.smithi044.stdout:8/588: write d5/d20/d2b/d58/d89/fa6 [618169,64693] 0 2024-06-03T01:20:56.185 INFO:tasks.workunit.client.1.smithi044.stdout:9/425: mknod d6/dd/d51/d16/d5f/d79/c8a 0 2024-06-03T01:20:56.186 INFO:tasks.workunit.client.1.smithi044.stdout:1/635: dwrite d0/d3/d20/d40/d45/d4b/d5c/d94/fb3 [0,4194304] 0 2024-06-03T01:20:56.187 INFO:tasks.workunit.client.1.smithi044.stdout:1/636: truncate d0/d3/d14/f8d 65132 0 2024-06-03T01:20:56.188 INFO:tasks.workunit.client.1.smithi044.stdout:3/454: dwrite df/d13/d39/d3d/d43/f5b [0,4194304] 0 2024-06-03T01:20:56.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/615: creat d0/d3/d8/d95/d63/d89/fc8 x:0 0 0 2024-06-03T01:20:56.193 INFO:tasks.workunit.client.1.smithi044.stdout:8/589: mknod d5/d16/cbd 0 2024-06-03T01:20:56.196 INFO:tasks.workunit.client.1.smithi044.stdout:9/426: write d6/d26/f4a [2604725,114531] 0 2024-06-03T01:20:56.198 INFO:tasks.workunit.client.1.smithi044.stdout:3/455: symlink df/d13/d1b/d59/l92 0 2024-06-03T01:20:56.199 INFO:tasks.workunit.client.1.smithi044.stdout:7/616: mkdir d0/d3/d8/d95/d63/d7f/d9c/da4/dc9 0 2024-06-03T01:20:56.200 INFO:tasks.workunit.client.1.smithi044.stdout:7/617: dread - d0/d3/d8/d95/d63/d89/fc8 zero size 2024-06-03T01:20:56.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/118: write da/dd/df/d10/f14 [2349404,18907] 0 2024-06-03T01:20:56.203 INFO:tasks.workunit.client.1.smithi044.stdout:8/590: rename d5/d55/d56/d5c/d68/c8d to d5/d16/d1d/d6b/cbe 0 2024-06-03T01:20:56.205 INFO:tasks.workunit.client.1.smithi044.stdout:8/591: write d5/d55/d74/d87/f99 [160846,93051] 0 2024-06-03T01:20:56.206 INFO:tasks.workunit.client.1.smithi044.stdout:8/592: write d5/d16/d81/f9c [673870,83814] 0 2024-06-03T01:20:56.208 INFO:tasks.workunit.client.1.smithi044.stdout:9/427: mkdir d6/dd/d51/d16/d5f/d8b 0 2024-06-03T01:20:56.208 INFO:tasks.workunit.client.1.smithi044.stdout:9/428: read d6/dd/d51/d16/d27/f53 [1466587,113527] 0 2024-06-03T01:20:56.209 INFO:tasks.workunit.client.1.smithi044.stdout:3/456: mkdir df/d13/d1b/d93 0 2024-06-03T01:20:56.209 INFO:tasks.workunit.client.1.smithi044.stdout:3/457: readlink df/d1e/l5d 0 2024-06-03T01:20:56.209 INFO:tasks.workunit.client.0.smithi032.stdout:2/138: write d5/da/f1e [2695514,90646] 0 2024-06-03T01:20:56.209 INFO:tasks.workunit.client.1.smithi044.stdout:3/458: fsync f9 0 2024-06-03T01:20:56.212 INFO:tasks.workunit.client.1.smithi044.stdout:9/429: mknod d6/dd/d51/d16/d4d/c8c 0 2024-06-03T01:20:56.213 INFO:tasks.workunit.client.1.smithi044.stdout:3/459: getdents df/d13/d39/d3d/d43/d72 0 2024-06-03T01:20:56.215 INFO:tasks.workunit.client.1.smithi044.stdout:9/430: mkdir d6/dd/d51/d16/d5f/d8b/d8d 0 2024-06-03T01:20:56.218 INFO:tasks.workunit.client.1.smithi044.stdout:3/460: unlink df/d13/f2c 0 2024-06-03T01:20:56.218 INFO:tasks.workunit.client.1.smithi044.stdout:3/461: creat df/d13/d39/d3d/d43/d71/f94 x:0 0 0 2024-06-03T01:20:56.219 INFO:tasks.workunit.client.1.smithi044.stdout:1/637: dread d0/d3/d20/d7/d13/f3d [0,4194304] 0 2024-06-03T01:20:56.221 INFO:tasks.workunit.client.1.smithi044.stdout:3/462: creat df/d13/d1b/d4b/d69/f95 x:0 0 0 2024-06-03T01:20:56.221 INFO:tasks.workunit.client.1.smithi044.stdout:3/463: chown df/d13/d39/d3d 147799 1 2024-06-03T01:20:56.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/638: creat d0/d3/d14/fe5 x:0 0 0 2024-06-03T01:20:56.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/639: truncate d0/d3/d20/d40/d45/d4b/d80/f92 4459624 0 2024-06-03T01:20:56.225 INFO:tasks.workunit.client.1.smithi044.stdout:3/464: creat df/d1e/d2a/d70/f96 x:0 0 0 2024-06-03T01:20:56.231 INFO:tasks.workunit.client.0.smithi032.stdout:1/70: dwrite f12 [0,4194304] 0 2024-06-03T01:20:56.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/168: dwrite d1/da/d10/f19 [0,4194304] 0 2024-06-03T01:20:56.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/169: write d1/da/f1f [1462713,88122] 0 2024-06-03T01:20:56.234 INFO:tasks.workunit.client.0.smithi032.stdout:1/71: unlink l16 0 2024-06-03T01:20:56.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/170: rename d1 to d1/da/d10/d2f 22 2024-06-03T01:20:56.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/171: truncate d1/f2b 575648 0 2024-06-03T01:20:56.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/172: truncate d1/da/d10/f2d 308569 0 2024-06-03T01:20:56.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/173: chown d1 173679490 1 2024-06-03T01:20:56.237 INFO:tasks.workunit.client.0.smithi032.stdout:9/229: dwrite d0/d11/f48 [0,4194304] 0 2024-06-03T01:20:56.237 INFO:tasks.workunit.client.0.smithi032.stdout:7/104: dwrite dc/d12/d1b/f1d [0,4194304] 0 2024-06-03T01:20:56.243 INFO:tasks.workunit.client.0.smithi032.stdout:7/105: rename dc/d12/f18 to dc/dd/f20 0 2024-06-03T01:20:56.244 INFO:tasks.workunit.client.0.smithi032.stdout:4/107: dwrite d4/f7 [0,4194304] 0 2024-06-03T01:20:56.248 INFO:tasks.workunit.client.0.smithi032.stdout:4/108: mkdir d4/db/da/d1d/d25/d27 0 2024-06-03T01:20:56.250 INFO:tasks.workunit.client.1.smithi044.stdout:3/465: read df/d13/f1d [3844348,99324] 0 2024-06-03T01:20:56.250 INFO:tasks.workunit.client.0.smithi032.stdout:4/109: mknod d4/db/da/d1d/d25/d27/c28 0 2024-06-03T01:20:56.250 INFO:tasks.workunit.client.0.smithi032.stdout:4/110: creat d4/f29 x:0 0 0 2024-06-03T01:20:56.250 INFO:tasks.workunit.client.0.smithi032.stdout:4/111: chown d4/db/f1a 638819618 1 2024-06-03T01:20:56.251 INFO:tasks.workunit.client.1.smithi044.stdout:3/466: getdents df/d13/d39/d3d/d43 0 2024-06-03T01:20:56.252 INFO:tasks.workunit.client.0.smithi032.stdout:4/112: link d4/f29 d4/db/da/d1d/d25/f2a 0 2024-06-03T01:20:56.252 INFO:tasks.workunit.client.0.smithi032.stdout:4/113: write d4/db/f11 [5003012,18976] 0 2024-06-03T01:20:56.253 INFO:tasks.workunit.client.0.smithi032.stdout:4/114: creat d4/db/da/d1d/d25/d27/f2b x:0 0 0 2024-06-03T01:20:56.255 INFO:tasks.workunit.client.0.smithi032.stdout:2/139: fdatasync d5/da/f1e 0 2024-06-03T01:20:56.261 INFO:tasks.workunit.client.0.smithi032.stdout:7/106: getdents dc/dd 0 2024-06-03T01:20:56.261 INFO:tasks.workunit.client.0.smithi032.stdout:7/107: creat dc/d12/d1b/f21 x:0 0 0 2024-06-03T01:20:56.263 INFO:tasks.workunit.client.1.smithi044.stdout:6/701: sync 2024-06-03T01:20:56.263 INFO:tasks.workunit.client.1.smithi044.stdout:6/702: readlink d2/d23/d30/d1e/d34/d39/d9b/db4/l88 0 2024-06-03T01:20:56.265 INFO:tasks.workunit.client.0.smithi032.stdout:8/90: dwrite dc/de/f11 [0,4194304] 0 2024-06-03T01:20:56.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/108: mkdir dc/d12/d22 0 2024-06-03T01:20:56.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/109: dread - dc/d12/d1b/f21 zero size 2024-06-03T01:20:56.265 INFO:tasks.workunit.client.0.smithi032.stdout:9/230: dwrite d0/d11/d1d/d38/f3e [0,4194304] 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/618: dwrite d0/d3/d8/f1b [0,4194304] 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/619: write d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 [1985628,82971] 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/620: truncate d0/d3/d8/f6d 1510657 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/621: write d0/d3/d8/d95/d63/d7f/f99 [836000,23130] 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/622: write d0/f7c [975970,104974] 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/623: chown d0/d16/d23/f78 31906 1 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/624: fdatasync d0/d3/d8/d32/f38 0 2024-06-03T01:20:56.266 INFO:tasks.workunit.client.1.smithi044.stdout:6/703: mkdir d2/d23/df4 0 2024-06-03T01:20:56.267 INFO:tasks.workunit.client.1.smithi044.stdout:1/640: dwrite d0/d66/f65 [0,4194304] 0 2024-06-03T01:20:56.267 INFO:tasks.workunit.client.0.smithi032.stdout:5/119: dwrite da/f17 [0,4194304] 0 2024-06-03T01:20:56.267 INFO:tasks.workunit.client.0.smithi032.stdout:6/121: dwrite d8/d11/f12 [0,4194304] 0 2024-06-03T01:20:56.267 INFO:tasks.workunit.client.0.smithi032.stdout:4/115: dwrite d4/db/da/f15 [0,4194304] 0 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:4/116: fsync d4/db/da/f1b 0 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:4/117: readlink d4/db/l24 0 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:7/110: mknod dc/d12/d22/c23 0 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:7/111: chown dc/dd/f20 103 1 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:7/112: dread - dc/dd/fe zero size 2024-06-03T01:20:56.268 INFO:tasks.workunit.client.0.smithi032.stdout:7/113: truncate dc/dd/fe 1024888 0 2024-06-03T01:20:56.269 INFO:tasks.workunit.client.1.smithi044.stdout:8/593: rename d5/d16/d1d/d6b to d5/d20/d2b/d58/d89/dbf 0 2024-06-03T01:20:56.269 INFO:tasks.workunit.client.0.smithi032.stdout:1/72: dwrite f15 [0,4194304] 0 2024-06-03T01:20:56.270 INFO:tasks.workunit.client.0.smithi032.stdout:0/174: write d1/f6 [2242401,17420] 0 2024-06-03T01:20:56.270 INFO:tasks.workunit.client.0.smithi032.stdout:0/175: truncate d1/da/f16 900045 0 2024-06-03T01:20:56.271 INFO:tasks.workunit.client.0.smithi032.stdout:9/231: mkdir d0/d11/d1d/d38/d4f 0 2024-06-03T01:20:56.274 INFO:tasks.workunit.client.0.smithi032.stdout:8/91: dread f2 [0,4194304] 0 2024-06-03T01:20:56.274 INFO:tasks.workunit.client.0.smithi032.stdout:5/120: creat da/dd/df/d10/d16/d18/f27 x:0 0 0 2024-06-03T01:20:56.276 INFO:tasks.workunit.client.1.smithi044.stdout:6/704: truncate d2/d23/d55/d5f/db2/d3f/fa1 4144291 0 2024-06-03T01:20:56.276 INFO:tasks.workunit.client.1.smithi044.stdout:6/705: truncate d2/d23/d55/d5f/d99/dcc/dd4/fd7 646464 0 2024-06-03T01:20:56.278 INFO:tasks.workunit.client.0.smithi032.stdout:6/122: symlink d8/d11/d20/d21/l26 0 2024-06-03T01:20:56.279 INFO:tasks.workunit.client.0.smithi032.stdout:2/140: dwrite d5/da/f29 [4194304,4194304] 0 2024-06-03T01:20:56.282 INFO:tasks.workunit.client.1.smithi044.stdout:1/641: mknod d0/d3/ce6 0 2024-06-03T01:20:56.283 INFO:tasks.workunit.client.0.smithi032.stdout:4/118: rename d4/f19 to d4/db/da/f2c 0 2024-06-03T01:20:56.284 INFO:tasks.workunit.client.1.smithi044.stdout:8/594: dread d5/d55/d74/d87/f7b [0,4194304] 0 2024-06-03T01:20:56.286 INFO:tasks.workunit.client.0.smithi032.stdout:5/121: write da/f17 [2752495,44471] 0 2024-06-03T01:20:56.286 INFO:tasks.workunit.client.0.smithi032.stdout:5/122: rename da to da/dd/df/d10/d16/d18/d28 22 2024-06-03T01:20:56.286 INFO:tasks.workunit.client.0.smithi032.stdout:5/123: creat da/dd/f29 x:0 0 0 2024-06-03T01:20:56.286 INFO:tasks.workunit.client.0.smithi032.stdout:5/124: dread - da/dd/df/d10/d16/d18/f1d zero size 2024-06-03T01:20:56.287 INFO:tasks.workunit.client.0.smithi032.stdout:5/125: creat da/f2a x:0 0 0 2024-06-03T01:20:56.288 INFO:tasks.workunit.client.1.smithi044.stdout:3/467: rename df/d1e/f35 to df/d13/d1b/d4b/f97 0 2024-06-03T01:20:56.288 INFO:tasks.workunit.client.1.smithi044.stdout:1/642: truncate d0/d3/d20/d7/fa7 8337351 0 2024-06-03T01:20:56.288 INFO:tasks.workunit.client.1.smithi044.stdout:3/468: stat df/d13/d39/d3d/d6a 0 2024-06-03T01:20:56.288 INFO:tasks.workunit.client.1.smithi044.stdout:3/469: chown df/d13/d1b/d59/l92 55 1 2024-06-03T01:20:56.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/114: mkdir dc/d12/d24 0 2024-06-03T01:20:56.289 INFO:tasks.workunit.client.1.smithi044.stdout:8/595: mknod d5/d20/d96/cc0 0 2024-06-03T01:20:56.290 INFO:tasks.workunit.client.1.smithi044.stdout:1/643: rename d0/c2 to d0/d3/d20/d40/d45/d6b/ce7 0 2024-06-03T01:20:56.291 INFO:tasks.workunit.client.1.smithi044.stdout:8/596: dread d5/d20/d2b/f30 [0,4194304] 0 2024-06-03T01:20:56.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/644: getdents d0/d3/de1 0 2024-06-03T01:20:56.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/645: dread d0/dd2/f60 [0,4194304] 0 2024-06-03T01:20:56.305 INFO:tasks.workunit.client.1.smithi044.stdout:8/597: dread d5/d20/d2b/f48 [0,4194304] 0 2024-06-03T01:20:56.306 INFO:tasks.workunit.client.1.smithi044.stdout:6/706: dwrite d2/f14 [0,4194304] 0 2024-06-03T01:20:56.309 INFO:tasks.workunit.client.1.smithi044.stdout:8/598: write d5/d55/d74/d87/f70 [539352,102816] 0 2024-06-03T01:20:56.309 INFO:tasks.workunit.client.1.smithi044.stdout:6/707: mknod d2/d23/d55/d5f/db2/d3f/cf5 0 2024-06-03T01:20:56.310 INFO:tasks.workunit.client.1.smithi044.stdout:6/708: fsync d2/d23/d30/d1e/d34/d39/d64/d70/fc9 0 2024-06-03T01:20:56.313 INFO:tasks.workunit.client.1.smithi044.stdout:6/709: dread d2/d23/d44/f54 [0,4194304] 0 2024-06-03T01:20:56.314 INFO:tasks.workunit.client.0.smithi032.stdout:5/126: dwrite da/dd/fe [0,4194304] 0 2024-06-03T01:20:56.317 INFO:tasks.workunit.client.1.smithi044.stdout:0/637: sync 2024-06-03T01:20:56.317 INFO:tasks.workunit.client.0.smithi032.stdout:0/176: symlink d1/da/l30 0 2024-06-03T01:20:56.317 INFO:tasks.workunit.client.0.smithi032.stdout:8/92: symlink dc/l18 0 2024-06-03T01:20:56.318 INFO:tasks.workunit.client.0.smithi032.stdout:7/115: dwrite dc/d12/d1b/f1d [4194304,4194304] 0 2024-06-03T01:20:56.318 INFO:tasks.workunit.client.0.smithi032.stdout:7/116: fsync f6 0 2024-06-03T01:20:56.319 INFO:tasks.workunit.client.0.smithi032.stdout:7/117: creat dc/d12/f25 x:0 0 0 2024-06-03T01:20:56.319 INFO:tasks.workunit.client.1.smithi044.stdout:6/710: dread d2/d23/d30/d1e/f49 [0,4194304] 0 2024-06-03T01:20:56.319 INFO:tasks.workunit.client.1.smithi044.stdout:6/711: readlink d2/d23/d30/d1e/d34/d39/da8/le4 0 2024-06-03T01:20:56.319 INFO:tasks.workunit.client.0.smithi032.stdout:4/119: dwrite d4/db/da/d1d/f21 [0,4194304] 0 2024-06-03T01:20:56.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/638: creat d3/d9/d55/d5a/fe9 x:0 0 0 2024-06-03T01:20:56.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/639: write d3/da6/d56/d8e/f92 [271238,76115] 0 2024-06-03T01:20:56.321 INFO:tasks.workunit.client.1.smithi044.stdout:6/712: mknod d2/d23/d30/ddf/cf6 0 2024-06-03T01:20:56.322 INFO:tasks.workunit.client.0.smithi032.stdout:6/123: creat d8/d11/d20/d21/f27 x:0 0 0 2024-06-03T01:20:56.322 INFO:tasks.workunit.client.0.smithi032.stdout:6/124: truncate d8/d25/f1f 1086262 0 2024-06-03T01:20:56.322 INFO:tasks.workunit.client.0.smithi032.stdout:2/141: symlink d5/l2d 0 2024-06-03T01:20:56.322 INFO:tasks.workunit.client.0.smithi032.stdout:2/142: chown d5/da/c28 34290 1 2024-06-03T01:20:56.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/713: stat d2/d23/d55/d5f/db2/d3f/c68 0 2024-06-03T01:20:56.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/714: readlink d2/d23/d55/d5f/db2/l4a 0 2024-06-03T01:20:56.324 INFO:tasks.workunit.client.0.smithi032.stdout:1/73: rename f12 to f17 0 2024-06-03T01:20:56.324 INFO:tasks.workunit.client.1.smithi044.stdout:1/646: dwrite d0/fba [0,4194304] 0 2024-06-03T01:20:56.324 INFO:tasks.workunit.client.1.smithi044.stdout:1/647: dread - d0/d3/d14/d81/f69 zero size 2024-06-03T01:20:56.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/648: symlink d0/d3/d20/le8 0 2024-06-03T01:20:56.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/649: chown d0/d3/d20/d40/d45/d6f/d8c/ca4 9 1 2024-06-03T01:20:56.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/650: chown d0/d3/d20/d7/d13/f4a 555 1 2024-06-03T01:20:56.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/651: chown d0/d3/d20/d7/fa7 3025 1 2024-06-03T01:20:56.329 INFO:tasks.workunit.client.0.smithi032.stdout:5/127: creat da/dd/df/d10/d16/f2b x:0 0 0 2024-06-03T01:20:56.329 INFO:tasks.workunit.client.0.smithi032.stdout:5/128: readlink da/dd/df/d10/l1f 0 2024-06-03T01:20:56.329 INFO:tasks.workunit.client.0.smithi032.stdout:0/177: mkdir d1/da/d10/d29/d31 0 2024-06-03T01:20:56.330 INFO:tasks.workunit.client.1.smithi044.stdout:1/652: symlink d0/d66/dbe/le9 0 2024-06-03T01:20:56.330 INFO:tasks.workunit.client.1.smithi044.stdout:1/653: fdatasync d0/d66/dbe/d9e/fa6 0 2024-06-03T01:20:56.330 INFO:tasks.workunit.client.1.smithi044.stdout:1/654: write d0/d3/d20/d40/d45/d6f/fd3 [457945,51010] 0 2024-06-03T01:20:56.331 INFO:tasks.workunit.client.0.smithi032.stdout:7/118: mknod dc/d12/c26 0 2024-06-03T01:20:56.333 INFO:tasks.workunit.client.1.smithi044.stdout:0/640: dread d3/da6/d20/d25/f40 [0,4194304] 0 2024-06-03T01:20:56.334 INFO:tasks.workunit.client.0.smithi032.stdout:6/125: dread f4 [0,4194304] 0 2024-06-03T01:20:56.334 INFO:tasks.workunit.client.0.smithi032.stdout:6/126: write d8/f9 [3335101,34333] 0 2024-06-03T01:20:56.336 INFO:tasks.workunit.client.0.smithi032.stdout:2/143: mkdir d5/da/d2e 0 2024-06-03T01:20:56.336 INFO:tasks.workunit.client.0.smithi032.stdout:2/144: dread - d5/f23 zero size 2024-06-03T01:20:56.336 INFO:tasks.workunit.client.0.smithi032.stdout:1/74: dread fa [0,4194304] 0 2024-06-03T01:20:56.339 INFO:tasks.workunit.client.0.smithi032.stdout:7/119: rename dc/l1f to dc/l27 0 2024-06-03T01:20:56.340 INFO:tasks.workunit.client.1.smithi044.stdout:8/599: dwrite d5/d20/d2b/f35 [4194304,4194304] 0 2024-06-03T01:20:56.340 INFO:tasks.workunit.client.1.smithi044.stdout:8/600: chown d5/d20/d2b/c3b 410382 1 2024-06-03T01:20:56.340 INFO:tasks.workunit.client.0.smithi032.stdout:6/127: mkdir d8/d11/d28 0 2024-06-03T01:20:56.342 INFO:tasks.workunit.client.1.smithi044.stdout:0/641: mkdir d3/d9/d55/d5a/dde/dea 0 2024-06-03T01:20:56.343 INFO:tasks.workunit.client.0.smithi032.stdout:4/120: rename d4/db/da/d1d/d25 to d4/db/da/d2d 0 2024-06-03T01:20:56.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/178: dread d1/fc [0,4194304] 0 2024-06-03T01:20:56.344 INFO:tasks.workunit.client.0.smithi032.stdout:6/128: dread d8/d25/f1f [0,4194304] 0 2024-06-03T01:20:56.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/642: mknod d3/d9/d55/ceb 0 2024-06-03T01:20:56.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/145: rename d5/da/f10 to d5/da/f2f 0 2024-06-03T01:20:56.348 INFO:tasks.workunit.client.1.smithi044.stdout:0/643: rename d3/da6/d20/c32 to d3/da6/d56/d8e/cec 0 2024-06-03T01:20:56.349 INFO:tasks.workunit.client.1.smithi044.stdout:0/644: mkdir d3/d9/d55/d5a/dde/ded 0 2024-06-03T01:20:56.351 INFO:tasks.workunit.client.1.smithi044.stdout:0/645: rename d3/d6b/d95/ca5 to d3/d9/d55/d5a/dde/dea/cee 0 2024-06-03T01:20:56.353 INFO:tasks.workunit.client.1.smithi044.stdout:0/646: unlink d3/d9/d36/d82/la7 0 2024-06-03T01:20:56.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/179: read d1/f17 [2614194,122887] 0 2024-06-03T01:20:56.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/647: creat d3/da6/dc1/dc2/fef x:0 0 0 2024-06-03T01:20:56.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/648: write d3/d9/d36/f50 [227029,4190] 0 2024-06-03T01:20:56.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/649: stat d3/d6b/d95 0 2024-06-03T01:20:56.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/650: fdatasync d3/d9/d55/d73/f77 0 2024-06-03T01:20:56.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/651: truncate d3/d9/d36/fe0 760265 0 2024-06-03T01:20:56.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/120: unlink f7 0 2024-06-03T01:20:56.356 INFO:tasks.workunit.client.0.smithi032.stdout:4/121: truncate d4/db/ff 264344 0 2024-06-03T01:20:56.356 INFO:tasks.workunit.client.0.smithi032.stdout:4/122: dread - d4/db/f1a zero size 2024-06-03T01:20:56.356 INFO:tasks.workunit.client.0.smithi032.stdout:4/123: fsync d4/db/da/f2c 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:0/652: rename d3/d9/d55/d5a/d29/d58/l89 to d3/da6/dc1/dc2/de4/lf0 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:0/653: dread - d3/f98 zero size 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/715: dwrite d2/f27 [0,4194304] 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/716: write d2/d23/d55/f6f [559908,2081] 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/717: stat d2/d23/d30/d1e/d34/d39/d7d/fac 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/718: stat d2/d23 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/719: chown d2/d23/d30/d1e/d34/d39/d64/lb6 14441266 1 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/720: fdatasync d2/d23/d55/d5f/db2/d3f/fc5 0 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/721: chown d2/fe 6 1 2024-06-03T01:20:56.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/722: write d2/d23/d30/d1e/d38/d6c/fbf [561071,73503] 0 2024-06-03T01:20:56.358 INFO:tasks.workunit.client.0.smithi032.stdout:2/146: mknod d5/c30 0 2024-06-03T01:20:56.358 INFO:tasks.workunit.client.0.smithi032.stdout:2/147: chown d5/f23 11365099 1 2024-06-03T01:20:56.358 INFO:tasks.workunit.client.1.smithi044.stdout:1/655: dwrite d0/d3/d20/d7/d13/d2d/fce [0,4194304] 0 2024-06-03T01:20:56.358 INFO:tasks.workunit.client.1.smithi044.stdout:1/656: chown d0/la3 521242447 1 2024-06-03T01:20:56.358 INFO:tasks.workunit.client.1.smithi044.stdout:1/657: chown d0/d66/dbe/fc5 30910134 1 2024-06-03T01:20:56.360 INFO:tasks.workunit.client.0.smithi032.stdout:1/75: mkdir d18 0 2024-06-03T01:20:56.360 INFO:tasks.workunit.client.1.smithi044.stdout:6/723: symlink d2/d23/d44/d4d/da0/lf7 0 2024-06-03T01:20:56.360 INFO:tasks.workunit.client.1.smithi044.stdout:6/724: write d2/d23/d44/f94 [188982,44705] 0 2024-06-03T01:20:56.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/129: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:56.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/658: creat d0/d3/d20/d40/d45/fea x:0 0 0 2024-06-03T01:20:56.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/659: chown d0/d3/d20/d40/d45/d4b/d80/cad 1066 1 2024-06-03T01:20:56.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/660: stat d0/d3/d20/d40/f88 0 2024-06-03T01:20:56.365 INFO:tasks.workunit.client.0.smithi032.stdout:0/180: truncate d1/f1c 2166306 0 2024-06-03T01:20:56.365 INFO:tasks.workunit.client.0.smithi032.stdout:0/181: chown d1/f21 353982 1 2024-06-03T01:20:56.371 INFO:tasks.workunit.client.1.smithi044.stdout:1/661: stat d0/d3/d20/d40/d45/l7b 0 2024-06-03T01:20:56.371 INFO:tasks.workunit.client.0.smithi032.stdout:5/130: dread da/f17 [0,4194304] 0 2024-06-03T01:20:56.371 INFO:tasks.workunit.client.0.smithi032.stdout:4/124: rename d4/db/f1a to d4/db/f2e 0 2024-06-03T01:20:56.372 INFO:tasks.workunit.client.0.smithi032.stdout:7/121: link dc/dd/lf dc/d12/d1b/l28 0 2024-06-03T01:20:56.372 INFO:tasks.workunit.client.0.smithi032.stdout:4/125: write d4/f6 [980234,99114] 0 2024-06-03T01:20:56.373 INFO:tasks.workunit.client.0.smithi032.stdout:1/76: creat d18/f19 x:0 0 0 2024-06-03T01:20:56.375 INFO:tasks.workunit.client.1.smithi044.stdout:6/725: dread d2/fe [0,4194304] 0 2024-06-03T01:20:56.378 INFO:tasks.workunit.client.1.smithi044.stdout:0/654: dwrite d3/f6d [0,4194304] 0 2024-06-03T01:20:56.379 INFO:tasks.workunit.client.1.smithi044.stdout:6/726: mkdir d2/d23/d30/d1e/d34/d39/d7d/da7/df8 0 2024-06-03T01:20:56.382 INFO:tasks.workunit.client.0.smithi032.stdout:7/122: dread dc/f14 [0,4194304] 0 2024-06-03T01:20:56.393 INFO:tasks.workunit.client.0.smithi032.stdout:0/182: dwrite d1/da/f16 [0,4194304] 0 2024-06-03T01:20:56.393 INFO:tasks.workunit.client.0.smithi032.stdout:0/183: fsync d1/da/d10/f2d 0 2024-06-03T01:20:56.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/131: dwrite da/dd/df/d10/f14 [0,4194304] 0 2024-06-03T01:20:56.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/132: write da/dd/df/d10/d16/f2b [1006193,88149] 0 2024-06-03T01:20:56.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/133: write da/f15 [101247,84069] 0 2024-06-03T01:20:56.397 INFO:tasks.workunit.client.1.smithi044.stdout:1/662: dwrite d0/d3/d14/d81/f56 [0,4194304] 0 2024-06-03T01:20:56.413 INFO:tasks.workunit.client.0.smithi032.stdout:5/134: dwrite f4 [4194304,4194304] 0 2024-06-03T01:20:56.413 INFO:tasks.workunit.client.0.smithi032.stdout:5/135: write da/f15 [23256,114476] 0 2024-06-03T01:20:56.476 INFO:tasks.workunit.client.0.smithi032.stdout:2/148: rmdir d5/da 39 2024-06-03T01:20:56.477 INFO:tasks.workunit.client.0.smithi032.stdout:4/126: truncate d4/db/f1c 411235 0 2024-06-03T01:20:56.484 INFO:tasks.workunit.client.0.smithi032.stdout:7/123: creat dc/d12/d22/f29 x:0 0 0 2024-06-03T01:20:56.491 INFO:tasks.workunit.client.0.smithi032.stdout:0/184: creat d1/da/d10/f32 x:0 0 0 2024-06-03T01:20:56.500 INFO:tasks.workunit.client.0.smithi032.stdout:2/149: creat d5/da/d1a/f31 x:0 0 0 2024-06-03T01:20:56.503 INFO:tasks.workunit.client.0.smithi032.stdout:4/127: unlink d4/db/l18 0 2024-06-03T01:20:56.504 INFO:tasks.workunit.client.0.smithi032.stdout:1/77: rmdir d18 39 2024-06-03T01:20:56.509 INFO:tasks.workunit.client.0.smithi032.stdout:7/124: mkdir dc/d12/d2a 0 2024-06-03T01:20:56.509 INFO:tasks.workunit.client.0.smithi032.stdout:7/125: chown dc/d12/d22/c23 374869 1 2024-06-03T01:20:56.513 INFO:tasks.workunit.client.0.smithi032.stdout:0/185: link d1/da/d10/f2a d1/da/d28/f33 0 2024-06-03T01:20:56.519 INFO:tasks.workunit.client.0.smithi032.stdout:5/136: rmdir da 39 2024-06-03T01:20:56.524 INFO:tasks.workunit.client.0.smithi032.stdout:4/128: truncate d4/db/da/d1d/f23 4767167 0 2024-06-03T01:20:56.524 INFO:tasks.workunit.client.0.smithi032.stdout:4/129: creat d4/db/da/f2f x:0 0 0 2024-06-03T01:20:56.524 INFO:tasks.workunit.client.0.smithi032.stdout:1/78: dwrite d18/f19 [0,4194304] 0 2024-06-03T01:20:56.526 INFO:tasks.workunit.client.1.smithi044.stdout:0/655: rename d3/d9/d36/f57 to d3/da6/d21/d94/da3/dd8/ff1 0 2024-06-03T01:20:56.528 INFO:tasks.workunit.client.0.smithi032.stdout:0/186: unlink d1/da/f1f 0 2024-06-03T01:20:56.529 INFO:tasks.workunit.client.0.smithi032.stdout:5/137: mknod da/dd/df/d10/d16/d25/c2c 0 2024-06-03T01:20:56.530 INFO:tasks.workunit.client.0.smithi032.stdout:2/150: rmdir d5 39 2024-06-03T01:20:56.532 INFO:tasks.workunit.client.0.smithi032.stdout:4/130: creat d4/db/da/d2d/d27/f30 x:0 0 0 2024-06-03T01:20:56.533 INFO:tasks.workunit.client.0.smithi032.stdout:4/131: truncate d4/db/f1c 672859 0 2024-06-03T01:20:56.533 INFO:tasks.workunit.client.0.smithi032.stdout:4/132: read - d4/db/f2e zero size 2024-06-03T01:20:56.535 INFO:tasks.workunit.client.0.smithi032.stdout:1/79: creat d18/f1a x:0 0 0 2024-06-03T01:20:56.535 INFO:tasks.workunit.client.0.smithi032.stdout:1/80: write d18/f1a [103930,58628] 0 2024-06-03T01:20:56.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/663: link d0/d16/d64/c87 d0/d3/d20/d7/d13/d2d/dd9/ceb 0 2024-06-03T01:20:56.539 INFO:tasks.workunit.client.0.smithi032.stdout:2/151: truncate d5/f13 4132138 0 2024-06-03T01:20:56.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/138: creat da/dd/f2d x:0 0 0 2024-06-03T01:20:56.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/139: readlink da/dd/df/l22 0 2024-06-03T01:20:56.539 INFO:tasks.workunit.client.0.smithi032.stdout:2/152: fsync d5/da/f20 0 2024-06-03T01:20:56.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/140: fsync da/dd/df/d10/d16/d18/f27 0 2024-06-03T01:20:56.540 INFO:tasks.workunit.client.0.smithi032.stdout:4/133: unlink d4/db/f11 0 2024-06-03T01:20:56.541 INFO:tasks.workunit.client.0.smithi032.stdout:0/187: symlink d1/l34 0 2024-06-03T01:20:56.542 INFO:tasks.workunit.client.0.smithi032.stdout:5/141: symlink da/dd/df/d10/d16/l2e 0 2024-06-03T01:20:56.542 INFO:tasks.workunit.client.0.smithi032.stdout:5/142: readlink da/dd/df/l22 0 2024-06-03T01:20:56.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/143: dread da/dd/df/d10/d16/f2b [0,4194304] 0 2024-06-03T01:20:56.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/144: fsync da/dd/df/f1c 0 2024-06-03T01:20:56.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/145: readlink l5 0 2024-06-03T01:20:56.552 INFO:tasks.workunit.client.0.smithi032.stdout:4/134: read d4/db/da/f15 [585780,112688] 0 2024-06-03T01:20:56.552 INFO:tasks.workunit.client.0.smithi032.stdout:4/135: dread - d4/db/da/d2d/d27/f2b zero size 2024-06-03T01:20:56.555 INFO:tasks.workunit.client.0.smithi032.stdout:1/81: truncate d18/f19 3988630 0 2024-06-03T01:20:56.555 INFO:tasks.workunit.client.0.smithi032.stdout:3/112: sync 2024-06-03T01:20:56.555 INFO:tasks.workunit.client.0.smithi032.stdout:7/126: dwrite dc/dd/f11 [4194304,4194304] 0 2024-06-03T01:20:56.555 INFO:tasks.workunit.client.0.smithi032.stdout:7/127: creat dc/f2b x:0 0 0 2024-06-03T01:20:56.555 INFO:tasks.workunit.client.0.smithi032.stdout:2/153: write d5/ff [2825399,48779] 0 2024-06-03T01:20:56.557 INFO:tasks.workunit.client.0.smithi032.stdout:0/188: rename d1/f1b to d1/f35 0 2024-06-03T01:20:56.564 INFO:tasks.workunit.client.0.smithi032.stdout:1/82: dread f15 [0,4194304] 0 2024-06-03T01:20:56.565 INFO:tasks.workunit.client.0.smithi032.stdout:2/154: creat d5/da/d2e/f32 x:0 0 0 2024-06-03T01:20:56.565 INFO:tasks.workunit.client.0.smithi032.stdout:2/155: write d5/da/d2e/f32 [166399,78853] 0 2024-06-03T01:20:56.567 INFO:tasks.workunit.client.0.smithi032.stdout:0/189: link d1/l34 d1/da/d10/d29/l36 0 2024-06-03T01:20:56.567 INFO:tasks.workunit.client.0.smithi032.stdout:0/190: chown d1 111824375 1 2024-06-03T01:20:56.568 INFO:tasks.workunit.client.0.smithi032.stdout:3/113: rmdir de 39 2024-06-03T01:20:56.575 INFO:tasks.workunit.client.0.smithi032.stdout:1/83: mkdir d18/d1b 0 2024-06-03T01:20:56.577 INFO:tasks.workunit.client.0.smithi032.stdout:4/136: dwrite d4/db/da/d2d/d27/f2b [0,4194304] 0 2024-06-03T01:20:56.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/191: mknod d1/da/d28/c37 0 2024-06-03T01:20:56.579 INFO:tasks.workunit.client.0.smithi032.stdout:3/114: rmdir de 39 2024-06-03T01:20:56.579 INFO:tasks.workunit.client.0.smithi032.stdout:3/115: truncate de/d17/f1b 391780 0 2024-06-03T01:20:56.579 INFO:tasks.workunit.client.0.smithi032.stdout:3/116: readlink l6 0 2024-06-03T01:20:56.579 INFO:tasks.workunit.client.0.smithi032.stdout:3/117: write de/d17/f18 [783153,14171] 0 2024-06-03T01:20:56.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/118: write de/f1a [4157978,101217] 0 2024-06-03T01:20:56.582 INFO:tasks.workunit.client.0.smithi032.stdout:1/84: rmdir d18/d1b 0 2024-06-03T01:20:56.582 INFO:tasks.workunit.client.0.smithi032.stdout:1/85: creat d18/f1c x:0 0 0 2024-06-03T01:20:56.583 INFO:tasks.workunit.client.0.smithi032.stdout:5/146: dwrite da/f12 [0,4194304] 0 2024-06-03T01:20:56.586 INFO:tasks.workunit.client.0.smithi032.stdout:1/86: dread f15 [0,4194304] 0 2024-06-03T01:20:56.587 INFO:tasks.workunit.client.0.smithi032.stdout:3/119: symlink de/l1f 0 2024-06-03T01:20:56.594 INFO:tasks.workunit.client.0.smithi032.stdout:5/147: stat da/cb 0 2024-06-03T01:20:56.595 INFO:tasks.workunit.client.0.smithi032.stdout:0/192: symlink d1/l38 0 2024-06-03T01:20:56.595 INFO:tasks.workunit.client.0.smithi032.stdout:1/87: symlink d18/l1d 0 2024-06-03T01:20:56.595 INFO:tasks.workunit.client.0.smithi032.stdout:1/88: creat d18/f1e x:0 0 0 2024-06-03T01:20:56.595 INFO:tasks.workunit.client.0.smithi032.stdout:1/89: chown d18/f1a 137136 1 2024-06-03T01:20:56.596 INFO:tasks.workunit.client.0.smithi032.stdout:0/193: chown d1/da/d10/c25 40 1 2024-06-03T01:20:56.597 INFO:tasks.workunit.client.0.smithi032.stdout:0/194: write d1/da/d10/f2d [47534,56355] 0 2024-06-03T01:20:56.597 INFO:tasks.workunit.client.0.smithi032.stdout:0/195: read d1/fc [2618594,14102] 0 2024-06-03T01:20:56.597 INFO:tasks.workunit.client.0.smithi032.stdout:0/196: stat d1/da/l30 0 2024-06-03T01:20:56.606 INFO:tasks.workunit.client.0.smithi032.stdout:0/197: creat d1/da/d10/f39 x:0 0 0 2024-06-03T01:20:56.606 INFO:tasks.workunit.client.0.smithi032.stdout:0/198: chown d1/l3 518 1 2024-06-03T01:20:56.612 INFO:tasks.workunit.client.0.smithi032.stdout:3/120: dwrite fd [0,4194304] 0 2024-06-03T01:20:56.615 INFO:tasks.workunit.client.0.smithi032.stdout:1/90: dwrite f17 [0,4194304] 0 2024-06-03T01:20:56.615 INFO:tasks.workunit.client.0.smithi032.stdout:5/148: truncate da/f12 2344854 0 2024-06-03T01:20:56.617 INFO:tasks.workunit.client.0.smithi032.stdout:3/121: symlink de/l20 0 2024-06-03T01:20:56.618 INFO:tasks.workunit.client.0.smithi032.stdout:3/122: creat de/d17/f21 x:0 0 0 2024-06-03T01:20:56.618 INFO:tasks.workunit.client.0.smithi032.stdout:3/123: chown de/d17/f18 77 1 2024-06-03T01:20:56.619 INFO:tasks.workunit.client.0.smithi032.stdout:3/124: symlink de/d17/l22 0 2024-06-03T01:20:56.623 INFO:tasks.workunit.client.0.smithi032.stdout:3/125: read de/f10 [1266095,130758] 0 2024-06-03T01:20:56.623 INFO:tasks.workunit.client.0.smithi032.stdout:3/126: truncate de/d17/f1b 1429330 0 2024-06-03T01:20:56.625 INFO:tasks.workunit.client.0.smithi032.stdout:3/127: mkdir de/d17/d23 0 2024-06-03T01:20:56.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/199: dwrite d1/f18 [0,4194304] 0 2024-06-03T01:20:56.626 INFO:tasks.workunit.client.1.smithi044.stdout:4/610: sync 2024-06-03T01:20:56.627 INFO:tasks.workunit.client.0.smithi032.stdout:0/200: mkdir d1/da/d10/d29/d31/d3a 0 2024-06-03T01:20:56.628 INFO:tasks.workunit.client.0.smithi032.stdout:0/201: readlink d1/da/d10/l15 0 2024-06-03T01:20:56.630 INFO:tasks.workunit.client.0.smithi032.stdout:5/149: read f4 [244482,89776] 0 2024-06-03T01:20:56.631 INFO:tasks.workunit.client.0.smithi032.stdout:5/150: chown da/dd/f2d 123 1 2024-06-03T01:20:56.631 INFO:tasks.workunit.client.0.smithi032.stdout:5/151: read da/f15 [91692,73520] 0 2024-06-03T01:20:56.631 INFO:tasks.workunit.client.1.smithi044.stdout:4/611: dread d17/f69 [0,4194304] 0 2024-06-03T01:20:56.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/202: mkdir d1/da/d28/d3b 0 2024-06-03T01:20:56.633 INFO:tasks.workunit.client.0.smithi032.stdout:5/152: symlink da/dd/df/d10/l2f 0 2024-06-03T01:20:56.639 INFO:tasks.workunit.client.0.smithi032.stdout:1/91: dwrite d18/f1e [0,4194304] 0 2024-06-03T01:20:56.646 INFO:tasks.workunit.client.0.smithi032.stdout:3/128: truncate de/d17/f1b 331082 0 2024-06-03T01:20:56.646 INFO:tasks.workunit.client.0.smithi032.stdout:5/153: write da/dd/df/d10/f14 [885619,6079] 0 2024-06-03T01:20:56.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/612: dwrite d17/d1c/fb7 [0,4194304] 0 2024-06-03T01:20:56.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/613: chown d17/d63/f65 435396516 1 2024-06-03T01:20:56.648 INFO:tasks.workunit.client.0.smithi032.stdout:3/129: mkdir de/d17/d24 0 2024-06-03T01:20:56.650 INFO:tasks.workunit.client.0.smithi032.stdout:5/154: mkdir da/dd/df/d10/d16/d18/d1a/d30 0 2024-06-03T01:20:56.650 INFO:tasks.workunit.client.0.smithi032.stdout:5/155: fsync da/dd/df/d10/d16/f2b 0 2024-06-03T01:20:56.651 INFO:tasks.workunit.client.0.smithi032.stdout:3/130: creat de/d17/d24/f25 x:0 0 0 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:8/601: sync 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:3/470: sync 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:9/431: sync 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:5/528: sync 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:9/432: chown d6/dd/f21 2 1 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:8/602: truncate d5/d55/d74/d87/f99 1515441 0 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:7/625: sync 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:5/529: write d2/d11/d1e/d21/d24/d93/d9d/d28/f7c [617884,110280] 0 2024-06-03T01:20:56.654 INFO:tasks.workunit.client.1.smithi044.stdout:2/448: sync 2024-06-03T01:20:56.655 INFO:tasks.workunit.client.1.smithi044.stdout:2/449: write db/d1b/d2a/f2b [417877,42183] 0 2024-06-03T01:20:56.655 INFO:tasks.workunit.client.1.smithi044.stdout:2/450: write db/d15/d49/d7b/f8a [119354,94597] 0 2024-06-03T01:20:56.655 INFO:tasks.workunit.client.1.smithi044.stdout:6/727: rename d2/d23/d30/d1e/d38/d6c/f89 to d2/d23/df4/ff9 0 2024-06-03T01:20:56.655 INFO:tasks.workunit.client.1.smithi044.stdout:6/728: fdatasync d2/d23/d44/d4d/da0/fb0 0 2024-06-03T01:20:56.656 INFO:tasks.workunit.client.0.smithi032.stdout:1/92: dwrite f17 [0,4194304] 0 2024-06-03T01:20:56.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/93: rename l9 to d18/l1f 0 2024-06-03T01:20:56.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/94: truncate d18/f1a 821815 0 2024-06-03T01:20:56.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/95: truncate fa 4609934 0 2024-06-03T01:20:56.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/96: fsync d18/f1c 0 2024-06-03T01:20:56.662 INFO:tasks.workunit.client.0.smithi032.stdout:1/97: mknod d18/c20 0 2024-06-03T01:20:56.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/614: symlink d17/d1c/d75/d7a/db8/d6a/d84/ld2 0 2024-06-03T01:20:56.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/615: fsync d17/d1c/d75/d7a/db8/d8d/f9e 0 2024-06-03T01:20:56.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/616: fdatasync f2 0 2024-06-03T01:20:56.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/617: fdatasync d17/d1c/d9c/fbd 0 2024-06-03T01:20:56.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/618: write d17/d1c/d75/d7a/db8/d28/fd1 [737678,83140] 0 2024-06-03T01:20:56.669 INFO:tasks.workunit.client.1.smithi044.stdout:9/433: mknod d6/dd/d45/c8e 0 2024-06-03T01:20:56.669 INFO:tasks.workunit.client.1.smithi044.stdout:7/626: getdents d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96 0 2024-06-03T01:20:56.669 INFO:tasks.workunit.client.1.smithi044.stdout:7/627: chown d0/d16/d23/f78 343105463 1 2024-06-03T01:20:56.670 INFO:tasks.workunit.client.1.smithi044.stdout:5/530: creat d2/d11/d12/d2b/fa9 x:0 0 0 2024-06-03T01:20:56.671 INFO:tasks.workunit.client.1.smithi044.stdout:5/531: chown d2/d11/d12/d2b/d44/c41 17475759 1 2024-06-03T01:20:56.671 INFO:tasks.workunit.client.1.smithi044.stdout:5/532: creat d2/d11/d1e/d21/d24/faa x:0 0 0 2024-06-03T01:20:56.671 INFO:tasks.workunit.client.1.smithi044.stdout:5/533: chown d2/d11/d1e/d21/d24/d93/d9d/f8c 173983419 1 2024-06-03T01:20:56.671 INFO:tasks.workunit.client.1.smithi044.stdout:5/534: readlink d2/d11/d12/l1f 0 2024-06-03T01:20:56.676 INFO:tasks.workunit.client.0.smithi032.stdout:1/98: dread d18/f19 [0,4194304] 0 2024-06-03T01:20:56.676 INFO:tasks.workunit.client.0.smithi032.stdout:5/156: dwrite da/f2a [0,4194304] 0 2024-06-03T01:20:56.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/157: stat da/dd/f29 0 2024-06-03T01:20:56.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/158: write da/dd/f29 [440195,50812] 0 2024-06-03T01:20:56.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/159: fdatasync da/f17 0 2024-06-03T01:20:56.678 INFO:tasks.workunit.client.0.smithi032.stdout:3/131: dwrite de/f1a [0,4194304] 0 2024-06-03T01:20:56.681 INFO:tasks.workunit.client.1.smithi044.stdout:0/656: rename d3/d9/d55/d7b/lcb to d3/da6/d21/d94/da3/dad/lf2 0 2024-06-03T01:20:56.684 INFO:tasks.workunit.client.1.smithi044.stdout:4/619: creat d17/d1c/d75/d7a/db8/d6a/fd3 x:0 0 0 2024-06-03T01:20:56.686 INFO:tasks.workunit.client.1.smithi044.stdout:9/434: creat d6/dd/d51/d16/d5f/d8b/d8d/f8f x:0 0 0 2024-06-03T01:20:56.686 INFO:tasks.workunit.client.1.smithi044.stdout:9/435: chown d6/dd/l2a 8543 1 2024-06-03T01:20:56.688 INFO:tasks.workunit.client.1.smithi044.stdout:7/628: creat d0/d3/fca x:0 0 0 2024-06-03T01:20:56.692 INFO:tasks.workunit.client.1.smithi044.stdout:5/535: mknod d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/cab 0 2024-06-03T01:20:56.695 INFO:tasks.workunit.client.1.smithi044.stdout:0/657: mkdir d3/da6/d56/d62/de7/df3 0 2024-06-03T01:20:56.696 INFO:tasks.workunit.client.1.smithi044.stdout:8/603: dwrite d5/d16/d1d/d32/f7a [0,4194304] 0 2024-06-03T01:20:56.697 INFO:tasks.workunit.client.1.smithi044.stdout:3/471: dwrite f9 [4194304,4194304] 0 2024-06-03T01:20:56.697 INFO:tasks.workunit.client.1.smithi044.stdout:4/620: mknod d17/d63/dae/cd4 0 2024-06-03T01:20:56.697 INFO:tasks.workunit.client.1.smithi044.stdout:4/621: chown d17/d1c/fb7 1591 1 2024-06-03T01:20:56.700 INFO:tasks.workunit.client.1.smithi044.stdout:0/658: creat d3/da6/dc1/dc2/de4/ff4 x:0 0 0 2024-06-03T01:20:56.701 INFO:tasks.workunit.client.1.smithi044.stdout:4/622: write d17/d1c/d75/d7a/db8/d8d/f9e [165904,33212] 0 2024-06-03T01:20:56.703 INFO:tasks.workunit.client.1.smithi044.stdout:5/536: write d2/d11/d1e/d21/d24/d93/d9d/d28/d66/f8e [4040610,113862] 0 2024-06-03T01:20:56.704 INFO:tasks.workunit.client.0.smithi032.stdout:3/132: dwrite de/d17/f18 [0,4194304] 0 2024-06-03T01:20:56.705 INFO:tasks.workunit.client.1.smithi044.stdout:8/604: creat d5/d16/d81/d9b/fc1 x:0 0 0 2024-06-03T01:20:56.705 INFO:tasks.workunit.client.1.smithi044.stdout:8/605: chown d5/d55/d56/d5c 30964676 1 2024-06-03T01:20:56.705 INFO:tasks.workunit.client.1.smithi044.stdout:7/629: dwrite d0/d3/d8/f1b [0,4194304] 0 2024-06-03T01:20:56.705 INFO:tasks.workunit.client.1.smithi044.stdout:8/606: truncate d5/d20/d2b/d58/fa3 1024681 0 2024-06-03T01:20:56.712 INFO:tasks.workunit.client.1.smithi044.stdout:0/659: mknod d3/d9/d55/d5a/dde/dea/cf5 0 2024-06-03T01:20:56.712 INFO:tasks.workunit.client.1.smithi044.stdout:0/660: readlink d3/d6b/d95/l96 0 2024-06-03T01:20:56.712 INFO:tasks.workunit.client.1.smithi044.stdout:0/661: fdatasync d3/d9/d36/f50 0 2024-06-03T01:20:56.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/623: symlink d17/d1c/d75/d7a/db8/d6a/ld5 0 2024-06-03T01:20:56.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/624: chown d17/d1c/d75/d7a/db8/d35/f41 1529424 1 2024-06-03T01:20:56.717 INFO:tasks.workunit.client.1.smithi044.stdout:5/537: mknod d2/d11/d1e/d3b/da6/cac 0 2024-06-03T01:20:56.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/607: creat d5/d55/d56/d5c/dbb/d79/d88/fc2 x:0 0 0 2024-06-03T01:20:56.721 INFO:tasks.workunit.client.1.smithi044.stdout:0/662: unlink d3/d6b/fe6 0 2024-06-03T01:20:56.721 INFO:tasks.workunit.client.1.smithi044.stdout:0/663: chown d3/d9/d55/d5a/cab 58 1 2024-06-03T01:20:56.724 INFO:tasks.workunit.client.1.smithi044.stdout:4/625: mknod d17/d1c/d75/d7a/db8/d6a/d84/cd6 0 2024-06-03T01:20:56.726 INFO:tasks.workunit.client.1.smithi044.stdout:4/626: dread d17/d1c/d75/f76 [0,4194304] 0 2024-06-03T01:20:56.726 INFO:tasks.workunit.client.1.smithi044.stdout:4/627: chown d17/d1c/d75/d7a/db8/d28/d2b/l37 105 1 2024-06-03T01:20:56.726 INFO:tasks.workunit.client.1.smithi044.stdout:4/628: creat d17/d1c/d75/d7a/db8/d35/fd7 x:0 0 0 2024-06-03T01:20:56.729 INFO:tasks.workunit.client.1.smithi044.stdout:4/629: creat d17/fd8 x:0 0 0 2024-06-03T01:20:56.730 INFO:tasks.workunit.client.1.smithi044.stdout:7/630: dwrite d0/d3/d30/fc2 [0,4194304] 0 2024-06-03T01:20:56.733 INFO:tasks.workunit.client.1.smithi044.stdout:0/664: write d3/f91 [890016,111970] 0 2024-06-03T01:20:56.733 INFO:tasks.workunit.client.1.smithi044.stdout:5/538: write d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f4e [875344,2201] 0 2024-06-03T01:20:56.733 INFO:tasks.workunit.client.1.smithi044.stdout:5/539: fdatasync d2/d11/d1e/d21/d24/d93/d9d/d28/f74 0 2024-06-03T01:20:56.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/133: dwrite de/f15 [0,4194304] 0 2024-06-03T01:20:56.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/631: unlink d0/d3/d8/d95/d63/d89/fc8 0 2024-06-03T01:20:56.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/632: chown d0/d3/f7 63 1 2024-06-03T01:20:56.741 INFO:tasks.workunit.client.1.smithi044.stdout:8/608: dwrite d5/d20/d2b/d58/fa3 [0,4194304] 0 2024-06-03T01:20:56.742 INFO:tasks.workunit.client.1.smithi044.stdout:7/633: rmdir d0/d3/d8/d32/d33 39 2024-06-03T01:20:56.743 INFO:tasks.workunit.client.0.smithi032.stdout:3/134: write de/f11 [613494,127301] 0 2024-06-03T01:20:56.744 INFO:tasks.workunit.client.0.smithi032.stdout:9/232: sync 2024-06-03T01:20:56.744 INFO:tasks.workunit.client.0.smithi032.stdout:9/233: getdents d0/d23/d24/d26/d41 0 2024-06-03T01:20:56.744 INFO:tasks.workunit.client.0.smithi032.stdout:9/234: write d0/d23/d24/d26/f39 [382632,78665] 0 2024-06-03T01:20:56.746 INFO:tasks.workunit.client.1.smithi044.stdout:0/665: dread d3/f14 [0,4194304] 0 2024-06-03T01:20:56.748 INFO:tasks.workunit.client.1.smithi044.stdout:0/666: dread d3/d9/d36/f50 [0,4194304] 0 2024-06-03T01:20:56.749 INFO:tasks.workunit.client.1.smithi044.stdout:7/634: creat d0/d3/d8/d32/fcb x:0 0 0 2024-06-03T01:20:56.753 INFO:tasks.workunit.client.0.smithi032.stdout:1/99: readlink lf 0 2024-06-03T01:20:56.756 INFO:tasks.workunit.client.1.smithi044.stdout:2/451: rmdir db/dd 39 2024-06-03T01:20:56.758 INFO:tasks.workunit.client.0.smithi032.stdout:3/135: symlink de/l26 0 2024-06-03T01:20:56.760 INFO:tasks.workunit.client.0.smithi032.stdout:3/136: mknod de/d17/c27 0 2024-06-03T01:20:56.764 INFO:tasks.workunit.client.0.smithi032.stdout:3/137: getdents de/d17/d24 0 2024-06-03T01:20:56.764 INFO:tasks.workunit.client.0.smithi032.stdout:3/138: truncate f9 290651 0 2024-06-03T01:20:56.764 INFO:tasks.workunit.client.1.smithi044.stdout:7/635: dread d0/d3/d8/d32/d33/f5b [0,4194304] 0 2024-06-03T01:20:56.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/139: symlink de/d17/l28 0 2024-06-03T01:20:56.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/140: write f9 [496443,22989] 0 2024-06-03T01:20:56.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/141: mkdir de/d29 0 2024-06-03T01:20:56.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/142: write de/f11 [3463104,128926] 0 2024-06-03T01:20:56.769 INFO:tasks.workunit.client.0.smithi032.stdout:3/143: dread de/d17/f1b [0,4194304] 0 2024-06-03T01:20:56.769 INFO:tasks.workunit.client.0.smithi032.stdout:3/144: dread - de/d17/d24/f25 zero size 2024-06-03T01:20:56.769 INFO:tasks.workunit.client.0.smithi032.stdout:3/145: chown de/f10 60274452 1 2024-06-03T01:20:56.770 INFO:tasks.workunit.client.0.smithi032.stdout:3/146: fsync de/f15 0 2024-06-03T01:20:56.770 INFO:tasks.workunit.client.0.smithi032.stdout:3/147: write de/d17/d24/f25 [744673,3765] 0 2024-06-03T01:20:56.770 INFO:tasks.workunit.client.0.smithi032.stdout:3/148: stat de/d17/d24 0 2024-06-03T01:20:56.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/452: dread db/f26 [0,4194304] 0 2024-06-03T01:20:56.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/453: chown db/d1b/d3f/f61 5048 1 2024-06-03T01:20:56.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/454: stat db/d1b/l23 0 2024-06-03T01:20:56.772 INFO:tasks.workunit.client.0.smithi032.stdout:3/149: symlink de/d17/l2a 0 2024-06-03T01:20:56.775 INFO:tasks.workunit.client.1.smithi044.stdout:7/636: write d0/d3/d8/d32/f38 [4033814,98965] 0 2024-06-03T01:20:56.777 INFO:tasks.workunit.client.1.smithi044.stdout:6/729: rename d2/d23/d30/d1e/d34 to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa 0 2024-06-03T01:20:56.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/540: mkdir d2/d11/d12/dad 0 2024-06-03T01:20:56.779 INFO:tasks.workunit.client.1.smithi044.stdout:5/541: chown d2/d11/d1e/d3b 19256738 1 2024-06-03T01:20:56.780 INFO:tasks.workunit.client.1.smithi044.stdout:7/637: symlink d0/d3/d8/d95/d63/d89/d8b/lcc 0 2024-06-03T01:20:56.780 INFO:tasks.workunit.client.1.smithi044.stdout:7/638: dread - d0/d3/d8/d95/d63/d89/d8b/f97 zero size 2024-06-03T01:20:56.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/639: write d0/d3/d8/d95/d63/d89/da3/f66 [1071061,56279] 0 2024-06-03T01:20:56.783 INFO:tasks.workunit.client.1.smithi044.stdout:3/472: rename df/d1e/d2a/d70/d81 to df/d13/d1b/d56/d5c/d98 0 2024-06-03T01:20:56.783 INFO:tasks.workunit.client.1.smithi044.stdout:3/473: fsync df/d13/d39/d3d/d43/d72/f91 0 2024-06-03T01:20:56.783 INFO:tasks.workunit.client.1.smithi044.stdout:8/609: dwrite d5/d16/f5e [0,4194304] 0 2024-06-03T01:20:56.784 INFO:tasks.workunit.client.1.smithi044.stdout:6/730: mkdir d2/d23/d30/ddf/dfb 0 2024-06-03T01:20:56.786 INFO:tasks.workunit.client.1.smithi044.stdout:5/542: mknod d2/d11/d3e/cae 0 2024-06-03T01:20:56.788 INFO:tasks.workunit.client.1.smithi044.stdout:3/474: stat df/d13/d1b/d4b/c60 0 2024-06-03T01:20:56.788 INFO:tasks.workunit.client.1.smithi044.stdout:3/475: dread - df/d13/d39/d3d/d43/d71/f83 zero size 2024-06-03T01:20:56.794 INFO:tasks.workunit.client.1.smithi044.stdout:8/610: truncate d5/d20/d33/f4c 2976461 0 2024-06-03T01:20:56.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/476: mknod df/d13/d1b/c99 0 2024-06-03T01:20:56.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/477: chown df/d13/d39/d3d/d43/d72 7 1 2024-06-03T01:20:56.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/478: chown lc 5336 1 2024-06-03T01:20:56.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/479: fsync df/f58 0 2024-06-03T01:20:56.795 INFO:tasks.workunit.client.1.smithi044.stdout:8/611: creat d5/d55/d74/db4/dba/fc3 x:0 0 0 2024-06-03T01:20:56.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/667: dwrite d3/da6/dc1/dc2/fef [0,4194304] 0 2024-06-03T01:20:56.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/668: chown d3/d9/d55/d5a/f34 31 1 2024-06-03T01:20:56.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/669: chown d3/f81 6 1 2024-06-03T01:20:56.796 INFO:tasks.workunit.client.1.smithi044.stdout:6/731: write d2/d23/d30/f1c [1367255,69098] 0 2024-06-03T01:20:56.796 INFO:tasks.workunit.client.1.smithi044.stdout:0/670: fsync d3/d9/d55/f79 0 2024-06-03T01:20:56.796 INFO:tasks.workunit.client.1.smithi044.stdout:6/732: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dfc 22 2024-06-03T01:20:56.796 INFO:tasks.workunit.client.0.smithi032.stdout:5/160: truncate da/dd/df/d10/f14 7644943 0 2024-06-03T01:20:56.796 INFO:tasks.workunit.client.0.smithi032.stdout:9/235: dwrite d0/d23/d24/d26/f39 [0,4194304] 0 2024-06-03T01:20:56.797 INFO:tasks.workunit.client.1.smithi044.stdout:0/671: dread d3/d9/d36/fe0 [0,4194304] 0 2024-06-03T01:20:56.797 INFO:tasks.workunit.client.1.smithi044.stdout:3/480: mkdir df/d13/d6d/d9a 0 2024-06-03T01:20:56.798 INFO:tasks.workunit.client.0.smithi032.stdout:5/161: mknod da/dd/df/d10/d16/d18/d1a/d30/c31 0 2024-06-03T01:20:56.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/455: dwrite db/d15/d49/d6c/d72/d7a/f68 [0,4194304] 0 2024-06-03T01:20:56.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/456: dread - db/f73 zero size 2024-06-03T01:20:56.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/457: truncate db/d15/d49/d6c/f8c 259938 0 2024-06-03T01:20:56.800 INFO:tasks.workunit.client.1.smithi044.stdout:1/664: symlink d0/d3/d20/d40/d45/lec 0 2024-06-03T01:20:56.801 INFO:tasks.workunit.client.1.smithi044.stdout:1/665: write d0/d3/d20/d40/d45/d4b/f90 [442922,115688] 0 2024-06-03T01:20:56.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/733: creat d2/d23/d30/ddf/ffd x:0 0 0 2024-06-03T01:20:56.807 INFO:tasks.workunit.client.0.smithi032.stdout:9/236: symlink d0/d11/d1d/d4a/l50 0 2024-06-03T01:20:56.808 INFO:tasks.workunit.client.1.smithi044.stdout:0/672: rename d3/da6/d20/faa to d3/d9/d55/d5a/dde/dea/ff6 0 2024-06-03T01:20:56.809 INFO:tasks.workunit.client.0.smithi032.stdout:9/237: mknod d0/d23/d24/d26/d41/c51 0 2024-06-03T01:20:56.809 INFO:tasks.workunit.client.0.smithi032.stdout:3/150: dwrite de/f10 [0,4194304] 0 2024-06-03T01:20:56.810 INFO:tasks.workunit.client.1.smithi044.stdout:3/481: rmdir df/d1e/d78 39 2024-06-03T01:20:56.812 INFO:tasks.workunit.client.0.smithi032.stdout:3/151: creat de/d29/f2b x:0 0 0 2024-06-03T01:20:56.812 INFO:tasks.workunit.client.0.smithi032.stdout:3/152: readlink de/d17/l28 0 2024-06-03T01:20:56.812 INFO:tasks.workunit.client.0.smithi032.stdout:3/153: readlink l6 0 2024-06-03T01:20:56.812 INFO:tasks.workunit.client.1.smithi044.stdout:5/543: dwrite d2/d11/d12/d2b/d4a/f5d [0,4194304] 0 2024-06-03T01:20:56.813 INFO:tasks.workunit.client.0.smithi032.stdout:3/154: symlink de/d17/d23/l2c 0 2024-06-03T01:20:56.814 INFO:tasks.workunit.client.0.smithi032.stdout:3/155: creat de/d17/d23/f2d x:0 0 0 2024-06-03T01:20:56.818 INFO:tasks.workunit.client.1.smithi044.stdout:5/544: dread d2/d11/d1e/d2f/f42 [0,4194304] 0 2024-06-03T01:20:56.818 INFO:tasks.workunit.client.1.smithi044.stdout:1/666: rmdir d0/d3/d20/d7/d13 39 2024-06-03T01:20:56.818 INFO:tasks.workunit.client.0.smithi032.stdout:5/162: write da/dd/df/f1c [2331107,113545] 0 2024-06-03T01:20:56.818 INFO:tasks.workunit.client.0.smithi032.stdout:5/163: truncate da/dd/df/d10/d16/f2b 2047025 0 2024-06-03T01:20:56.818 INFO:tasks.workunit.client.0.smithi032.stdout:5/164: rmdir da/dd/df/d10/d16/d18 39 2024-06-03T01:20:56.820 INFO:tasks.workunit.client.1.smithi044.stdout:8/612: dwrite d5/d20/d47/d64/fa0 [0,4194304] 0 2024-06-03T01:20:56.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/734: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/f9c [0,4194304] 0 2024-06-03T01:20:56.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/735: truncate d2/ff0 1042803 0 2024-06-03T01:20:56.821 INFO:tasks.workunit.client.0.smithi032.stdout:5/165: creat da/dd/df/f32 x:0 0 0 2024-06-03T01:20:56.821 INFO:tasks.workunit.client.0.smithi032.stdout:5/166: fsync da/dd/fe 0 2024-06-03T01:20:56.821 INFO:tasks.workunit.client.0.smithi032.stdout:5/167: truncate da/dd/df/d10/d16/d18/f1d 719589 0 2024-06-03T01:20:56.821 INFO:tasks.workunit.client.0.smithi032.stdout:5/168: creat da/dd/df/f33 x:0 0 0 2024-06-03T01:20:56.821 INFO:tasks.workunit.client.1.smithi044.stdout:5/545: dread d2/d11/d12/f6c [0,4194304] 0 2024-06-03T01:20:56.822 INFO:tasks.workunit.client.1.smithi044.stdout:5/546: chown d2/l5 5518 1 2024-06-03T01:20:56.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/169: mknod da/dd/df/d10/d16/d25/c34 0 2024-06-03T01:20:56.822 INFO:tasks.workunit.client.1.smithi044.stdout:0/673: creat d3/d9/d55/d5a/dde/dea/ff7 x:0 0 0 2024-06-03T01:20:56.822 INFO:tasks.workunit.client.1.smithi044.stdout:0/674: stat d3/da6/d21/d94/da3/dd8 0 2024-06-03T01:20:56.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/675: stat d3/da6/d20/d25/f65 0 2024-06-03T01:20:56.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/676: chown d3/d9/d55/d5a/d29/c7c 17 1 2024-06-03T01:20:56.823 INFO:tasks.workunit.client.1.smithi044.stdout:0/677: fsync d3/da6/d20/fdc 0 2024-06-03T01:20:56.823 INFO:tasks.workunit.client.0.smithi032.stdout:5/170: mkdir da/dd/df/d10/d16/d18/d35 0 2024-06-03T01:20:56.825 INFO:tasks.workunit.client.1.smithi044.stdout:3/482: mkdir df/d1e/d2a/d9b 0 2024-06-03T01:20:56.825 INFO:tasks.workunit.client.1.smithi044.stdout:2/458: getdents db/d15/d49/d6c 0 2024-06-03T01:20:56.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/667: mkdir d0/d3/d20/d7/ded 0 2024-06-03T01:20:56.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/668: creat d0/d3/d20/d40/d45/d4b/d80/fee x:0 0 0 2024-06-03T01:20:56.829 INFO:tasks.workunit.client.1.smithi044.stdout:8/613: mknod d5/d20/d96/cc4 0 2024-06-03T01:20:56.832 INFO:tasks.workunit.client.1.smithi044.stdout:6/736: mkdir d2/d23/d30/dfe 0 2024-06-03T01:20:56.834 INFO:tasks.workunit.client.1.smithi044.stdout:0/678: stat d3/d9/d55/d5a/dde/dea/cee 0 2024-06-03T01:20:56.839 INFO:tasks.workunit.client.1.smithi044.stdout:1/669: mkdir d0/d3/d20/def 0 2024-06-03T01:20:56.839 INFO:tasks.workunit.client.1.smithi044.stdout:1/670: truncate d0/d3/d20/d7/d13/d2d/fce 4384006 0 2024-06-03T01:20:56.839 INFO:tasks.workunit.client.1.smithi044.stdout:1/671: fsync d0/d3/d20/d40/d45/d4b/d5c/d94/fb3 0 2024-06-03T01:20:56.839 INFO:tasks.workunit.client.1.smithi044.stdout:1/672: fsync d0/d3/d20/d7/d13/fde 0 2024-06-03T01:20:56.841 INFO:tasks.workunit.client.1.smithi044.stdout:6/737: mkdir d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/dc0/dff 0 2024-06-03T01:20:56.841 INFO:tasks.workunit.client.1.smithi044.stdout:6/738: chown d2/f27 57995595 1 2024-06-03T01:20:56.843 INFO:tasks.workunit.client.1.smithi044.stdout:0/679: unlink d3/da6/f24 0 2024-06-03T01:20:56.843 INFO:tasks.workunit.client.1.smithi044.stdout:8/614: write d5/d20/d2b/f48 [7725051,62898] 0 2024-06-03T01:20:56.846 INFO:tasks.workunit.client.1.smithi044.stdout:2/459: rmdir db/d15/d49 39 2024-06-03T01:20:56.847 INFO:tasks.workunit.client.1.smithi044.stdout:3/483: write df/d13/d1b/f64 [1710091,92617] 0 2024-06-03T01:20:56.848 INFO:tasks.workunit.client.1.smithi044.stdout:1/673: rename d0/d66/dbe/d9e/fd0 to d0/d66/ff0 0 2024-06-03T01:20:56.848 INFO:tasks.workunit.client.1.smithi044.stdout:6/739: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/f100 x:0 0 0 2024-06-03T01:20:56.849 INFO:tasks.workunit.client.1.smithi044.stdout:6/740: write d2/d23/d30/f1f [2677199,103368] 0 2024-06-03T01:20:56.849 INFO:tasks.workunit.client.1.smithi044.stdout:8/615: symlink d5/d55/db5/lc5 0 2024-06-03T01:20:56.849 INFO:tasks.workunit.client.1.smithi044.stdout:2/460: creat db/d1b/d2a/d67/d53/f93 x:0 0 0 2024-06-03T01:20:56.852 INFO:tasks.workunit.client.1.smithi044.stdout:3/484: mkdir df/d13/d39/d3d/d6a/d9c 0 2024-06-03T01:20:56.853 INFO:tasks.workunit.client.1.smithi044.stdout:1/674: readlink d0/l2a 0 2024-06-03T01:20:56.853 INFO:tasks.workunit.client.1.smithi044.stdout:1/675: read d0/d3/d20/f6d [742788,65716] 0 2024-06-03T01:20:56.853 INFO:tasks.workunit.client.1.smithi044.stdout:8/616: symlink d5/d55/d74/db4/dba/lc6 0 2024-06-03T01:20:56.853 INFO:tasks.workunit.client.1.smithi044.stdout:8/617: truncate d5/d20/d2b/d58/d89/fb9 626890 0 2024-06-03T01:20:56.854 INFO:tasks.workunit.client.1.smithi044.stdout:2/461: mkdir db/d1b/d27/d79/d91/d94 0 2024-06-03T01:20:56.854 INFO:tasks.workunit.client.1.smithi044.stdout:2/462: fdatasync db/d1b/d2a/d2f/d4c/f77 0 2024-06-03T01:20:56.855 INFO:tasks.workunit.client.1.smithi044.stdout:3/485: unlink df/d13/d1b/d59/l77 0 2024-06-03T01:20:56.855 INFO:tasks.workunit.client.1.smithi044.stdout:3/486: chown df/d13/d39/d3d/d43/d72/l7a 473073 1 2024-06-03T01:20:56.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/676: mknod d0/d3/d20/d40/d45/d4b/d80/cf1 0 2024-06-03T01:20:56.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/677: getdents d0/d3/d14/dae 0 2024-06-03T01:20:56.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/678: readlink d0/d3/d20/d40/d45/d4b/d80/lcc 0 2024-06-03T01:20:56.858 INFO:tasks.workunit.client.1.smithi044.stdout:1/679: dread - d0/d66/dc2/fd1 zero size 2024-06-03T01:20:56.859 INFO:tasks.workunit.client.1.smithi044.stdout:8/618: symlink d5/d16/d1d/lc7 0 2024-06-03T01:20:56.860 INFO:tasks.workunit.client.1.smithi044.stdout:5/547: dwrite d2/d11/d3e/f59 [0,4194304] 0 2024-06-03T01:20:56.860 INFO:tasks.workunit.client.1.smithi044.stdout:5/548: rename d2/d11/d1e/d21/d24/d93 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/daf 22 2024-06-03T01:20:56.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/156: dwrite f8 [0,4194304] 0 2024-06-03T01:20:56.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/157: dread - de/d17/f1c zero size 2024-06-03T01:20:56.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/158: chown c3 62380 1 2024-06-03T01:20:56.861 INFO:tasks.workunit.client.0.smithi032.stdout:9/238: dwrite d0/fe [0,4194304] 0 2024-06-03T01:20:56.863 INFO:tasks.workunit.client.1.smithi044.stdout:3/487: link c0 df/d13/d6d/d9a/c9d 0 2024-06-03T01:20:56.863 INFO:tasks.workunit.client.1.smithi044.stdout:3/488: truncate df/d13/d1b/d56/d5c/f5f 482674 0 2024-06-03T01:20:56.864 INFO:tasks.workunit.client.0.smithi032.stdout:3/159: mkdir de/d29/d2e 0 2024-06-03T01:20:56.865 INFO:tasks.workunit.client.1.smithi044.stdout:1/680: dread d0/dd2/f60 [0,4194304] 0 2024-06-03T01:20:56.867 INFO:tasks.workunit.client.1.smithi044.stdout:6/741: read d2/d23/d30/d1e/d38/d77/fe5 [4741716,65878] 0 2024-06-03T01:20:56.874 INFO:tasks.workunit.client.1.smithi044.stdout:6/742: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/f101 x:0 0 0 2024-06-03T01:20:56.877 INFO:tasks.workunit.client.0.smithi032.stdout:3/160: dread de/f15 [0,4194304] 0 2024-06-03T01:20:56.878 INFO:tasks.workunit.client.1.smithi044.stdout:6/743: write d2/ff0 [813073,21249] 0 2024-06-03T01:20:56.878 INFO:tasks.workunit.client.1.smithi044.stdout:6/744: fdatasync d2/d23/d30/d1e/d38/d6c/f92 0 2024-06-03T01:20:56.878 INFO:tasks.workunit.client.1.smithi044.stdout:3/489: dread df/d13/f3b [0,4194304] 0 2024-06-03T01:20:56.882 INFO:tasks.workunit.client.0.smithi032.stdout:3/161: unlink de/l20 0 2024-06-03T01:20:56.882 INFO:tasks.workunit.client.0.smithi032.stdout:3/162: write de/d17/d23/f2d [398152,92025] 0 2024-06-03T01:20:56.884 INFO:tasks.workunit.client.0.smithi032.stdout:5/171: dread da/dd/df/f1c [4194304,4194304] 0 2024-06-03T01:20:56.885 INFO:tasks.workunit.client.0.smithi032.stdout:5/172: stat da/dd/df 0 2024-06-03T01:20:56.886 INFO:tasks.workunit.client.0.smithi032.stdout:3/163: link de/d17/f1c de/d17/d24/f2f 0 2024-06-03T01:20:56.887 INFO:tasks.workunit.client.1.smithi044.stdout:3/490: symlink df/d1e/d2a/d9b/l9e 0 2024-06-03T01:20:56.887 INFO:tasks.workunit.client.1.smithi044.stdout:0/680: dwrite d3/da6/d20/fdc [0,4194304] 0 2024-06-03T01:20:56.887 INFO:tasks.workunit.client.1.smithi044.stdout:0/681: write d3/d5c/fd6 [934899,126690] 0 2024-06-03T01:20:56.888 INFO:tasks.workunit.client.0.smithi032.stdout:5/173: creat da/dd/df/d10/d16/f36 x:0 0 0 2024-06-03T01:20:56.889 INFO:tasks.workunit.client.0.smithi032.stdout:5/174: chown da/f2a 3 1 2024-06-03T01:20:56.889 INFO:tasks.workunit.client.0.smithi032.stdout:5/175: mknod da/c37 0 2024-06-03T01:20:56.889 INFO:tasks.workunit.client.0.smithi032.stdout:5/176: stat da/dd/df/d10/l13 0 2024-06-03T01:20:56.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/745: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/dc7/f5d [0,4194304] 0 2024-06-03T01:20:56.890 INFO:tasks.workunit.client.0.smithi032.stdout:5/177: unlink da/dd/df/f1c 0 2024-06-03T01:20:56.891 INFO:tasks.workunit.client.1.smithi044.stdout:8/619: dwrite d5/d55/f86 [4194304,4194304] 0 2024-06-03T01:20:56.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/239: dwrite d0/d23/d24/f3b [0,4194304] 0 2024-06-03T01:20:56.893 INFO:tasks.workunit.client.1.smithi044.stdout:3/491: symlink df/d1e/d3a/l9f 0 2024-06-03T01:20:56.896 INFO:tasks.workunit.client.0.smithi032.stdout:9/240: creat d0/d23/d25/f52 x:0 0 0 2024-06-03T01:20:56.897 INFO:tasks.workunit.client.1.smithi044.stdout:0/682: mkdir d3/da6/dc1/dc2/df8 0 2024-06-03T01:20:56.898 INFO:tasks.workunit.client.1.smithi044.stdout:0/683: write d3/f85 [782084,48890] 0 2024-06-03T01:20:56.898 INFO:tasks.workunit.client.1.smithi044.stdout:0/684: creat d3/da6/dc1/dc2/ff9 x:0 0 0 2024-06-03T01:20:56.900 INFO:tasks.workunit.client.0.smithi032.stdout:9/241: unlink d0/d23/d24/f27 0 2024-06-03T01:20:56.901 INFO:tasks.workunit.client.1.smithi044.stdout:6/746: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/dee to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/d102 0 2024-06-03T01:20:56.902 INFO:tasks.workunit.client.0.smithi032.stdout:9/242: symlink d0/d23/d24/d26/d41/l53 0 2024-06-03T01:20:56.905 INFO:tasks.workunit.client.1.smithi044.stdout:3/492: symlink df/d13/d6d/la0 0 2024-06-03T01:20:56.905 INFO:tasks.workunit.client.0.smithi032.stdout:5/178: write da/f2a [463577,3582] 0 2024-06-03T01:20:56.906 INFO:tasks.workunit.client.1.smithi044.stdout:8/620: dread d5/d16/f6e [0,4194304] 0 2024-06-03T01:20:56.908 INFO:tasks.workunit.client.0.smithi032.stdout:5/179: symlink da/dd/df/d10/d16/d18/d1a/l38 0 2024-06-03T01:20:56.908 INFO:tasks.workunit.client.0.smithi032.stdout:5/180: chown da/dd/df/d10/d16 32 1 2024-06-03T01:20:56.908 INFO:tasks.workunit.client.0.smithi032.stdout:5/181: write da/dd/df/f32 [805742,22163] 0 2024-06-03T01:20:56.908 INFO:tasks.workunit.client.0.smithi032.stdout:5/182: stat da/dd/fe 0 2024-06-03T01:20:56.909 INFO:tasks.workunit.client.1.smithi044.stdout:6/747: symlink d2/d23/d55/l103 0 2024-06-03T01:20:56.911 INFO:tasks.workunit.client.0.smithi032.stdout:3/164: dwrite f5 [4194304,4194304] 0 2024-06-03T01:20:56.911 INFO:tasks.workunit.client.0.smithi032.stdout:5/183: creat da/dd/df/d10/f39 x:0 0 0 2024-06-03T01:20:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:3/493: symlink df/d13/d1b/d56/d5c/d98/la1 0 2024-06-03T01:20:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:3/494: creat df/d13/d39/d3d/d43/d71/fa2 x:0 0 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:0/685: dread d3/da6/d20/fdc [0,4194304] 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:0/686: stat d3/da6/d20/f5b 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:1/681: dwrite d0/d3/d20/fd7 [0,4194304] 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/748: mkdir d2/d23/d30/d1e/d38/d6c/dc2/d104 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/749: write d2/d23/d55/d5f/fa9 [1716194,3235] 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/750: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/f105 x:0 0 0 2024-06-03T01:20:56.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/751: truncate d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fe3 456956 0 2024-06-03T01:20:56.915 INFO:tasks.workunit.client.0.smithi032.stdout:3/165: mknod de/d17/c30 0 2024-06-03T01:20:56.915 INFO:tasks.workunit.client.1.smithi044.stdout:6/752: read d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/f9f [109302,54217] 0 2024-06-03T01:20:56.915 INFO:tasks.workunit.client.1.smithi044.stdout:8/621: dread d5/d16/f24 [0,4194304] 0 2024-06-03T01:20:56.919 INFO:tasks.workunit.client.0.smithi032.stdout:9/243: dwrite d0/d23/f3d [0,4194304] 0 2024-06-03T01:20:56.920 INFO:tasks.workunit.client.0.smithi032.stdout:9/244: read d0/f6 [1048000,68322] 0 2024-06-03T01:20:56.921 INFO:tasks.workunit.client.1.smithi044.stdout:0/687: rename d3/d9/d36/f50 to d3/da6/ffa 0 2024-06-03T01:20:56.923 INFO:tasks.workunit.client.0.smithi032.stdout:9/245: truncate d0/d11/d1d/d38/f3e 1407049 0 2024-06-03T01:20:56.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/246: rmdir d0/d23 39 2024-06-03T01:20:56.928 INFO:tasks.workunit.client.0.smithi032.stdout:3/166: write f8 [2941462,69851] 0 2024-06-03T01:20:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:1/682: write d0/d3/d20/d7/fa7 [1927298,115584] 0 2024-06-03T01:20:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:1/683: fsync d0/d3/d20/d7/d2f/f33 0 2024-06-03T01:20:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/495: dread fd [0,4194304] 0 2024-06-03T01:20:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/496: dread - df/d1e/d2a/f89 zero size 2024-06-03T01:20:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/497: creat df/d1e/d2a/d70/fa3 x:0 0 0 2024-06-03T01:20:56.930 INFO:tasks.workunit.client.0.smithi032.stdout:9/247: truncate d0/d11/d1d/f49 1320474 0 2024-06-03T01:20:56.930 INFO:tasks.workunit.client.0.smithi032.stdout:9/248: fdatasync d0/d11/d1d/f49 0 2024-06-03T01:20:56.930 INFO:tasks.workunit.client.0.smithi032.stdout:5/184: dwrite da/dd/df/d10/f39 [0,4194304] 0 2024-06-03T01:20:56.931 INFO:tasks.workunit.client.0.smithi032.stdout:5/185: write da/dd/df/f33 [702673,78948] 0 2024-06-03T01:20:56.931 INFO:tasks.workunit.client.0.smithi032.stdout:5/186: truncate da/dd/df/f32 1697788 0 2024-06-03T01:20:56.931 INFO:tasks.workunit.client.0.smithi032.stdout:5/187: read - da/dd/df/d10/d16/d18/f27 zero size 2024-06-03T01:20:56.932 INFO:tasks.workunit.client.1.smithi044.stdout:6/753: mkdir d2/d23/d55/d5f/db2/d3f/d106 0 2024-06-03T01:20:56.932 INFO:tasks.workunit.client.1.smithi044.stdout:6/754: creat d2/d23/d44/d4d/f107 x:0 0 0 2024-06-03T01:20:56.933 INFO:tasks.workunit.client.0.smithi032.stdout:9/249: dread d0/fa [0,4194304] 0 2024-06-03T01:20:56.935 INFO:tasks.workunit.client.1.smithi044.stdout:0/688: mknod d3/da6/d20/d25/dbf/cfb 0 2024-06-03T01:20:56.935 INFO:tasks.workunit.client.1.smithi044.stdout:0/689: write d3/da6/f3e [2524510,120744] 0 2024-06-03T01:20:56.938 INFO:tasks.workunit.client.0.smithi032.stdout:5/188: fdatasync da/f2a 0 2024-06-03T01:20:56.939 INFO:tasks.workunit.client.0.smithi032.stdout:3/167: rename de/d17 to de/d29/d2e/d31 0 2024-06-03T01:20:56.939 INFO:tasks.workunit.client.0.smithi032.stdout:3/168: dread - de/d29/f2b zero size 2024-06-03T01:20:56.943 INFO:tasks.workunit.client.1.smithi044.stdout:8/622: dwrite d5/f6a [0,4194304] 0 2024-06-03T01:20:56.944 INFO:tasks.workunit.client.0.smithi032.stdout:9/250: dread d0/d23/d24/d26/f39 [0,4194304] 0 2024-06-03T01:20:56.944 INFO:tasks.workunit.client.0.smithi032.stdout:9/251: read d0/d11/d1d/f47 [713249,30500] 0 2024-06-03T01:20:56.944 INFO:tasks.workunit.client.0.smithi032.stdout:9/252: fdatasync d0/fd 0 2024-06-03T01:20:56.946 INFO:tasks.workunit.client.0.smithi032.stdout:5/189: creat da/dd/df/d10/d16/d18/d35/f3a x:0 0 0 2024-06-03T01:20:56.946 INFO:tasks.workunit.client.0.smithi032.stdout:3/169: creat de/d29/d2e/d31/f32 x:0 0 0 2024-06-03T01:20:56.946 INFO:tasks.workunit.client.1.smithi044.stdout:1/684: link d0/d3/d20/d40/d45/d6f/fd3 d0/d3/d20/d40/d45/d4b/d5a/ff2 0 2024-06-03T01:20:56.946 INFO:tasks.workunit.client.1.smithi044.stdout:1/685: chown d0/d3/d20/def 432 1 2024-06-03T01:20:56.946 INFO:tasks.workunit.client.1.smithi044.stdout:1/686: write d0/d3/d20/d40/d45/d4b/fbd [1033958,83338] 0 2024-06-03T01:20:56.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/170: getdents de/d29 0 2024-06-03T01:20:56.950 INFO:tasks.workunit.client.1.smithi044.stdout:6/755: creat d2/d23/d55/d5f/db2/f108 x:0 0 0 2024-06-03T01:20:56.950 INFO:tasks.workunit.client.0.smithi032.stdout:3/171: symlink de/l33 0 2024-06-03T01:20:56.951 INFO:tasks.workunit.client.0.smithi032.stdout:3/172: write de/d29/d2e/d31/f32 [501514,106028] 0 2024-06-03T01:20:56.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/690: dread d3/d9/f28 [0,4194304] 0 2024-06-03T01:20:56.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/691: write d3/d9/d55/d5a/f34 [524996,16603] 0 2024-06-03T01:20:56.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/692: fsync d3/da6/d21/d94/da3/dd8/ff1 0 2024-06-03T01:20:56.955 INFO:tasks.workunit.client.1.smithi044.stdout:0/693: read - d3/da6/d21/fc4 zero size 2024-06-03T01:20:56.955 INFO:tasks.workunit.client.1.smithi044.stdout:8/623: rename d5/d16/d1d/d59/l67 to d5/d20/d2b/lc8 0 2024-06-03T01:20:56.959 INFO:tasks.workunit.client.1.smithi044.stdout:1/687: creat d0/d3/d20/d7/d13/d2d/dcd/ff3 x:0 0 0 2024-06-03T01:20:56.961 INFO:tasks.workunit.client.1.smithi044.stdout:0/694: mkdir d3/d9/d55/d7b/dfc 0 2024-06-03T01:20:56.964 INFO:tasks.workunit.client.1.smithi044.stdout:0/695: mknod d3/d6b/de8/cfd 0 2024-06-03T01:20:56.964 INFO:tasks.workunit.client.1.smithi044.stdout:0/696: chown d3/da6/dc1/cdb 5 1 2024-06-03T01:20:56.964 INFO:tasks.workunit.client.1.smithi044.stdout:0/697: stat d3/d9/d55/d73/da0 0 2024-06-03T01:20:56.966 INFO:tasks.workunit.client.1.smithi044.stdout:8/624: write d5/d16/f5e [4097198,26771] 0 2024-06-03T01:20:56.969 INFO:tasks.workunit.client.1.smithi044.stdout:8/625: rename d5/d20/d96 to d5/d55/d74/dc9 0 2024-06-03T01:20:56.969 INFO:tasks.workunit.client.1.smithi044.stdout:8/626: write d5/d20/d2b/d58/d89/fab [188542,111076] 0 2024-06-03T01:20:56.972 INFO:tasks.workunit.client.0.smithi032.stdout:5/190: write da/dd/df/d10/f14 [5818764,111535] 0 2024-06-03T01:20:56.975 INFO:tasks.workunit.client.0.smithi032.stdout:5/191: symlink da/l3b 0 2024-06-03T01:20:56.975 INFO:tasks.workunit.client.0.smithi032.stdout:5/192: chown da/dd/df/d10/l2f 2683485 1 2024-06-03T01:20:56.976 INFO:tasks.workunit.client.0.smithi032.stdout:5/193: symlink da/dd/df/d10/d16/d18/d35/l3c 0 2024-06-03T01:20:56.977 INFO:tasks.workunit.client.1.smithi044.stdout:8/627: unlink d5/d20/d47/d64/l93 0 2024-06-03T01:20:56.977 INFO:tasks.workunit.client.1.smithi044.stdout:0/698: write d3/d9/f22 [3578321,67245] 0 2024-06-03T01:20:56.977 INFO:tasks.workunit.client.0.smithi032.stdout:5/194: creat da/dd/df/d10/d16/d25/f3d x:0 0 0 2024-06-03T01:20:56.977 INFO:tasks.workunit.client.0.smithi032.stdout:5/195: readlink da/dd/df/l22 0 2024-06-03T01:20:56.978 INFO:tasks.workunit.client.1.smithi044.stdout:3/498: dwrite df/d13/f24 [0,4194304] 0 2024-06-03T01:20:56.978 INFO:tasks.workunit.client.1.smithi044.stdout:8/628: dread d5/d20/d2b/d98/fa7 [0,4194304] 0 2024-06-03T01:20:56.979 INFO:tasks.workunit.client.0.smithi032.stdout:9/253: dwrite d0/d11/d1d/f49 [0,4194304] 0 2024-06-03T01:20:56.979 INFO:tasks.workunit.client.0.smithi032.stdout:5/196: rename da/dd/df/d10/d16/f2b to da/f3e 0 2024-06-03T01:20:56.980 INFO:tasks.workunit.client.0.smithi032.stdout:9/254: rmdir d0/d23/d25 39 2024-06-03T01:20:56.980 INFO:tasks.workunit.client.0.smithi032.stdout:9/255: truncate d0/fd 2474161 0 2024-06-03T01:20:56.980 INFO:tasks.workunit.client.1.smithi044.stdout:0/699: link d3/d9/d55/d5a/fe9 d3/d6b/d95/db3/ffe 0 2024-06-03T01:20:56.982 INFO:tasks.workunit.client.0.smithi032.stdout:5/197: mknod da/c3f 0 2024-06-03T01:20:56.983 INFO:tasks.workunit.client.1.smithi044.stdout:1/688: dwrite d0/d16/f3a [0,4194304] 0 2024-06-03T01:20:56.983 INFO:tasks.workunit.client.1.smithi044.stdout:1/689: write d0/d3/d20/d40/fdd [1586505,43451] 0 2024-06-03T01:20:56.983 INFO:tasks.workunit.client.0.smithi032.stdout:9/256: creat d0/d11/d1d/f54 x:0 0 0 2024-06-03T01:20:56.983 INFO:tasks.workunit.client.0.smithi032.stdout:9/257: chown d0/c4 620544843 1 2024-06-03T01:20:56.984 INFO:tasks.workunit.client.1.smithi044.stdout:8/629: mkdir d5/d10/dca 0 2024-06-03T01:20:56.986 INFO:tasks.workunit.client.1.smithi044.stdout:0/700: truncate d3/f14 2277591 0 2024-06-03T01:20:56.986 INFO:tasks.workunit.client.1.smithi044.stdout:0/701: write d3/d9/d55/d5a/fe9 [491344,58379] 0 2024-06-03T01:20:56.989 INFO:tasks.workunit.client.1.smithi044.stdout:3/499: read df/d1e/d2a/f36 [7459332,67678] 0 2024-06-03T01:20:56.989 INFO:tasks.workunit.client.1.smithi044.stdout:6/756: dwrite d2/d23/d55/d5f/d99/dcc/dd4/fd7 [0,4194304] 0 2024-06-03T01:20:56.989 INFO:tasks.workunit.client.0.smithi032.stdout:5/198: dread da/f17 [0,4194304] 0 2024-06-03T01:20:56.990 INFO:tasks.workunit.client.1.smithi044.stdout:0/702: mknod d3/da6/d56/d8e/cff 0 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/199: getdents da/dd/df 0 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/200: rename da to da/dd/df/d40 22 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/201: fdatasync f4 0 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/202: chown da/f12 331 1 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/203: readlink da/dd/df/d10/l1f 0 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/204: truncate da/dd/df/d10/d16/d18/d35/f3a 375808 0 2024-06-03T01:20:56.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/205: write da/dd/df/d10/d16/d18/f1d [1176568,16181] 0 2024-06-03T01:20:56.994 INFO:tasks.workunit.client.1.smithi044.stdout:0/703: symlink d3/d5c/de3/l100 0 2024-06-03T01:20:56.995 INFO:tasks.workunit.client.1.smithi044.stdout:3/500: mkdir df/d1e/d78/da4 0 2024-06-03T01:20:56.995 INFO:tasks.workunit.client.1.smithi044.stdout:8/630: dread d5/d20/d2b/f35 [4194304,4194304] 0 2024-06-03T01:20:56.997 INFO:tasks.workunit.client.1.smithi044.stdout:0/704: link d3/da6/lba d3/da6/d21/d94/da3/dad/l101 0 2024-06-03T01:20:57.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/501: getdents df/d13/d6d/d9a 0 2024-06-03T01:20:57.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/502: read df/d13/d1b/f38 [249229,75507] 0 2024-06-03T01:20:57.000 INFO:tasks.workunit.client.0.smithi032.stdout:5/206: mknod da/dd/df/c41 0 2024-06-03T01:20:57.002 INFO:tasks.workunit.client.0.smithi032.stdout:9/258: rmdir d0/d11 39 2024-06-03T01:20:57.003 INFO:tasks.workunit.client.0.smithi032.stdout:6/129: sync 2024-06-03T01:20:57.003 INFO:tasks.workunit.client.0.smithi032.stdout:8/93: sync 2024-06-03T01:20:57.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/207: dread da/dd/df/f32 [0,4194304] 0 2024-06-03T01:20:57.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/208: creat da/dd/df/d10/f42 x:0 0 0 2024-06-03T01:20:57.005 INFO:tasks.workunit.client.1.smithi044.stdout:8/631: creat d5/d55/d74/db4/fcb x:0 0 0 2024-06-03T01:20:57.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/705: mkdir d3/d9/d102 0 2024-06-03T01:20:57.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/706: chown d3/da6/d56/fd0 982754 1 2024-06-03T01:20:57.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/707: write d3/da6/d20/d25/fc3 [443676,58528] 0 2024-06-03T01:20:57.007 INFO:tasks.workunit.client.0.smithi032.stdout:9/259: creat d0/d23/d24/f55 x:0 0 0 2024-06-03T01:20:57.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/690: dwrite d0/d3/d20/d7/f34 [0,4194304] 0 2024-06-03T01:20:57.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/691: read d0/d3/d20/d7/f36 [2020478,94362] 0 2024-06-03T01:20:57.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/692: chown d0/d3 2540106 1 2024-06-03T01:20:57.009 INFO:tasks.workunit.client.1.smithi044.stdout:3/503: unlink df/d13/d1b/d56/f6c 0 2024-06-03T01:20:57.009 INFO:tasks.workunit.client.0.smithi032.stdout:6/130: link d8/d25/d19/l1a d8/d25/l29 0 2024-06-03T01:20:57.013 INFO:tasks.workunit.client.0.smithi032.stdout:5/209: unlink da/dd/df/l23 0 2024-06-03T01:20:57.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/260: creat d0/d11/d1d/f56 x:0 0 0 2024-06-03T01:20:57.015 INFO:tasks.workunit.client.1.smithi044.stdout:6/757: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/da8/fbb [0,4194304] 0 2024-06-03T01:20:57.019 INFO:tasks.workunit.client.0.smithi032.stdout:9/261: dread d0/d11/d1d/d38/f3e [0,4194304] 0 2024-06-03T01:20:57.019 INFO:tasks.workunit.client.1.smithi044.stdout:0/708: symlink d3/d9/d55/d5a/l103 0 2024-06-03T01:20:57.025 INFO:tasks.workunit.client.1.smithi044.stdout:1/693: mknod d0/d66/dbe/d9e/cf4 0 2024-06-03T01:20:57.029 INFO:tasks.workunit.client.1.smithi044.stdout:1/694: dread d0/d3/d20/d7/d13/fb6 [0,4194304] 0 2024-06-03T01:20:57.031 INFO:tasks.workunit.client.1.smithi044.stdout:8/632: dwrite d5/d55/d56/d5c/fae [0,4194304] 0 2024-06-03T01:20:57.033 INFO:tasks.workunit.client.0.smithi032.stdout:6/131: write d8/d11/f12 [196263,78111] 0 2024-06-03T01:20:57.033 INFO:tasks.workunit.client.0.smithi032.stdout:6/132: fsync d8/ff 0 2024-06-03T01:20:57.033 INFO:tasks.workunit.client.0.smithi032.stdout:9/262: dread d0/fd [0,4194304] 0 2024-06-03T01:20:57.034 INFO:tasks.workunit.client.1.smithi044.stdout:1/695: dread d0/d3/d20/d40/d45/d4b/fbd [0,4194304] 0 2024-06-03T01:20:57.034 INFO:tasks.workunit.client.1.smithi044.stdout:6/758: truncate d2/d23/d30/f1f 1516497 0 2024-06-03T01:20:57.035 INFO:tasks.workunit.client.0.smithi032.stdout:6/133: link d8/d11/c16 d8/d11/d20/c2a 0 2024-06-03T01:20:57.036 INFO:tasks.workunit.client.0.smithi032.stdout:9/263: write d0/d11/f31 [81790,113507] 0 2024-06-03T01:20:57.036 INFO:tasks.workunit.client.1.smithi044.stdout:8/633: truncate d5/d55/d74/d87/f7b 4615494 0 2024-06-03T01:20:57.036 INFO:tasks.workunit.client.1.smithi044.stdout:8/634: chown d5/d16/d81/f9c 110 1 2024-06-03T01:20:57.038 INFO:tasks.workunit.client.0.smithi032.stdout:6/134: truncate f4 3487458 0 2024-06-03T01:20:57.038 INFO:tasks.workunit.client.0.smithi032.stdout:9/264: dread d0/fd [0,4194304] 0 2024-06-03T01:20:57.039 INFO:tasks.workunit.client.1.smithi044.stdout:6/759: rmdir d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7 39 2024-06-03T01:20:57.039 INFO:tasks.workunit.client.0.smithi032.stdout:6/135: truncate d8/fd 757068 0 2024-06-03T01:20:57.040 INFO:tasks.workunit.client.1.smithi044.stdout:8/635: truncate d5/d20/d2b/d98/fa7 422560 0 2024-06-03T01:20:57.040 INFO:tasks.workunit.client.1.smithi044.stdout:8/636: readlink d5/d55/d74/d82/l61 0 2024-06-03T01:20:57.041 INFO:tasks.workunit.client.0.smithi032.stdout:9/265: creat d0/f57 x:0 0 0 2024-06-03T01:20:57.041 INFO:tasks.workunit.client.1.smithi044.stdout:3/504: dwrite f9 [8388608,4194304] 0 2024-06-03T01:20:57.042 INFO:tasks.workunit.client.1.smithi044.stdout:3/505: write df/d13/d39/d3d/d43/d71/fa2 [954584,47534] 0 2024-06-03T01:20:57.042 INFO:tasks.workunit.client.1.smithi044.stdout:3/506: creat df/d13/d39/d3d/fa5 x:0 0 0 2024-06-03T01:20:57.042 INFO:tasks.workunit.client.0.smithi032.stdout:6/136: mkdir d8/d11/d20/d2b 0 2024-06-03T01:20:57.043 INFO:tasks.workunit.client.1.smithi044.stdout:8/637: rename d5/d55/d74/d87/f70 to d5/d20/db3/fcc 0 2024-06-03T01:20:57.043 INFO:tasks.workunit.client.0.smithi032.stdout:6/137: mknod d8/d11/c2c 0 2024-06-03T01:20:57.044 INFO:tasks.workunit.client.1.smithi044.stdout:6/760: write d2/d23/d30/d1e/f49 [2684379,2799] 0 2024-06-03T01:20:57.044 INFO:tasks.workunit.client.0.smithi032.stdout:8/94: dwrite dc/f15 [0,4194304] 0 2024-06-03T01:20:57.045 INFO:tasks.workunit.client.1.smithi044.stdout:0/709: dwrite d3/d9/d36/fb4 [0,4194304] 0 2024-06-03T01:20:57.046 INFO:tasks.workunit.client.0.smithi032.stdout:5/210: dwrite da/dd/df/d10/d16/d25/f3d [0,4194304] 0 2024-06-03T01:20:57.046 INFO:tasks.workunit.client.0.smithi032.stdout:5/211: readlink da/l3b 0 2024-06-03T01:20:57.046 INFO:tasks.workunit.client.0.smithi032.stdout:8/95: symlink dc/de/l19 0 2024-06-03T01:20:57.046 INFO:tasks.workunit.client.0.smithi032.stdout:8/96: write f2 [6316983,57970] 0 2024-06-03T01:20:57.046 INFO:tasks.workunit.client.0.smithi032.stdout:8/97: readlink dc/l17 0 2024-06-03T01:20:57.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/507: creat df/d13/d1b/d56/d5c/d98/fa6 x:0 0 0 2024-06-03T01:20:57.048 INFO:tasks.workunit.client.0.smithi032.stdout:8/98: creat dc/d14/f1a x:0 0 0 2024-06-03T01:20:57.049 INFO:tasks.workunit.client.1.smithi044.stdout:8/638: creat d5/d20/d2b/d58/d89/dbf/fcd x:0 0 0 2024-06-03T01:20:57.049 INFO:tasks.workunit.client.1.smithi044.stdout:8/639: chown d5/d20/d2b/d98 181 1 2024-06-03T01:20:57.051 INFO:tasks.workunit.client.1.smithi044.stdout:6/761: mknod d2/d23/d30/d1e/d38/d6c/c109 0 2024-06-03T01:20:57.052 INFO:tasks.workunit.client.0.smithi032.stdout:8/99: mknod dc/de/c1b 0 2024-06-03T01:20:57.052 INFO:tasks.workunit.client.0.smithi032.stdout:8/100: readlink lb 0 2024-06-03T01:20:57.052 INFO:tasks.workunit.client.0.smithi032.stdout:8/101: truncate dc/f13 717224 0 2024-06-03T01:20:57.052 INFO:tasks.workunit.client.0.smithi032.stdout:8/102: mknod dc/d14/c1c 0 2024-06-03T01:20:57.056 INFO:tasks.workunit.client.1.smithi044.stdout:0/710: rename d3/da6/d56/c5e to d3/da6/dc1/dc2/de4/c104 0 2024-06-03T01:20:57.062 INFO:tasks.workunit.client.1.smithi044.stdout:3/508: creat df/d13/d1b/d93/fa7 x:0 0 0 2024-06-03T01:20:57.062 INFO:tasks.workunit.client.1.smithi044.stdout:3/509: fdatasync df/d1e/d2a/d70/f96 0 2024-06-03T01:20:57.065 INFO:tasks.workunit.client.0.smithi032.stdout:8/103: getdents dc/de 0 2024-06-03T01:20:57.065 INFO:tasks.workunit.client.0.smithi032.stdout:6/138: dwrite d8/d11/d20/d21/f24 [0,4194304] 0 2024-06-03T01:20:57.066 INFO:tasks.workunit.client.1.smithi044.stdout:6/762: unlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/f93 0 2024-06-03T01:20:57.066 INFO:tasks.workunit.client.1.smithi044.stdout:6/763: getdents d2/d23/d55/d5f/db2/dca 0 2024-06-03T01:20:57.067 INFO:tasks.workunit.client.0.smithi032.stdout:8/104: rename dc/de/c16 to dc/d14/c1d 0 2024-06-03T01:20:57.067 INFO:tasks.workunit.client.0.smithi032.stdout:8/105: chown f1 0 1 2024-06-03T01:20:57.067 INFO:tasks.workunit.client.1.smithi044.stdout:0/711: creat d3/d9/f105 x:0 0 0 2024-06-03T01:20:57.068 INFO:tasks.workunit.client.1.smithi044.stdout:3/510: creat df/d13/d39/d3d/d6a/fa8 x:0 0 0 2024-06-03T01:20:57.070 INFO:tasks.workunit.client.0.smithi032.stdout:6/139: mknod d8/d25/c2d 0 2024-06-03T01:20:57.070 INFO:tasks.workunit.client.1.smithi044.stdout:8/640: getdents d5/d55/d56/d5c/dbb/d79/d88 0 2024-06-03T01:20:57.071 INFO:tasks.workunit.client.0.smithi032.stdout:6/140: rename d8/d11/c2c to d8/d11/d20/d2b/c2e 0 2024-06-03T01:20:57.072 INFO:tasks.workunit.client.1.smithi044.stdout:0/712: write d3/d9/f1d [2161922,91880] 0 2024-06-03T01:20:57.072 INFO:tasks.workunit.client.1.smithi044.stdout:8/641: mkdir d5/d20/d2b/d58/d89/dbf/dce 0 2024-06-03T01:20:57.073 INFO:tasks.workunit.client.1.smithi044.stdout:0/713: symlink d3/d9/d55/d5a/dde/dea/l106 0 2024-06-03T01:20:57.075 INFO:tasks.workunit.client.1.smithi044.stdout:8/642: creat d5/d55/d56/d5c/d68/fcf x:0 0 0 2024-06-03T01:20:57.077 INFO:tasks.workunit.client.0.smithi032.stdout:8/106: write dc/f15 [3986617,38193] 0 2024-06-03T01:20:57.079 INFO:tasks.workunit.client.0.smithi032.stdout:8/107: fsync dc/de/f10 0 2024-06-03T01:20:57.080 INFO:tasks.workunit.client.1.smithi044.stdout:0/714: link d3/d9/d55/d5a/d29/f4f d3/da6/d21/d94/da3/dd8/f107 0 2024-06-03T01:20:57.081 INFO:tasks.workunit.client.1.smithi044.stdout:6/764: read d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/dc7/f5d [1019175,70641] 0 2024-06-03T01:20:57.081 INFO:tasks.workunit.client.0.smithi032.stdout:6/141: dread d8/fb [0,4194304] 0 2024-06-03T01:20:57.084 INFO:tasks.workunit.client.0.smithi032.stdout:6/142: unlink d8/d11/f12 0 2024-06-03T01:20:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr fail", "who": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: osdmap e47: 6 total, 6 up, 6 in 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "mgr fail", "who": "smithi032.kfcfaa"}]': finished 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: mgrmap e21: smithi044.wszjlw(active, starting, since 0.00801018s) 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:20:57.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: Manager daemon smithi044.wszjlw is now available 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:57.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:20:57.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:20:57.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/trash_purge_schedule"}]: dispatch 2024-06-03T01:20:57.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/trash_purge_schedule"}]: dispatch 2024-06-03T01:20:57.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: [03/Jun/2024:01:20:56] ENGINE Bus STARTING 2024-06-03T01:20:57.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:56 smithi032 ceph-mon[29360]: [03/Jun/2024:01:20:56] ENGINE Serving on http://172.21.15.44:8765 2024-06-03T01:20:57.090 INFO:tasks.workunit.client.1.smithi044.stdout:8/643: rename d5/d10 to d5/d20/d47/d64/dd0 0 2024-06-03T01:20:57.091 INFO:tasks.workunit.client.1.smithi044.stdout:8/644: symlink d5/d20/d47/d64/ld1 0 2024-06-03T01:20:57.091 INFO:tasks.workunit.client.0.smithi032.stdout:8/108: dread dc/de/f11 [0,4194304] 0 2024-06-03T01:20:57.093 INFO:tasks.workunit.client.1.smithi044.stdout:6/765: dread d2/d23/d30/f45 [0,4194304] 0 2024-06-03T01:20:57.093 INFO:tasks.workunit.client.1.smithi044.stdout:8/645: mkdir d5/d20/d2b/d58/d89/dbf/dd2 0 2024-06-03T01:20:57.093 INFO:tasks.workunit.client.1.smithi044.stdout:8/646: stat d5/d20/db3/fb7 0 2024-06-03T01:20:57.093 INFO:tasks.workunit.client.1.smithi044.stdout:8/647: stat d5/d55/d56/d5c/la2 0 2024-06-03T01:20:57.093 INFO:tasks.workunit.client.1.smithi044.stdout:8/648: chown f3 10 1 2024-06-03T01:20:57.095 INFO:tasks.workunit.client.1.smithi044.stdout:6/766: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/da8/f10a x:0 0 0 2024-06-03T01:20:57.100 INFO:tasks.workunit.client.1.smithi044.stdout:3/511: dwrite df/d13/d39/d3d/d43/d71/f83 [0,4194304] 0 2024-06-03T01:20:57.100 INFO:tasks.workunit.client.1.smithi044.stdout:3/512: dread - df/d1e/d2a/d70/fa3 zero size 2024-06-03T01:20:57.101 INFO:tasks.workunit.client.1.smithi044.stdout:3/513: creat df/d13/d39/d3d/fa9 x:0 0 0 2024-06-03T01:20:57.103 INFO:tasks.workunit.client.1.smithi044.stdout:0/715: dwrite d3/d6b/f6f [0,4194304] 0 2024-06-03T01:20:57.103 INFO:tasks.workunit.client.1.smithi044.stdout:0/716: readlink d3/da6/d21/d94/da3/dad/l101 0 2024-06-03T01:20:57.103 INFO:tasks.workunit.client.1.smithi044.stdout:0/717: chown d3/d6b/f6f 508144 1 2024-06-03T01:20:57.105 INFO:tasks.workunit.client.1.smithi044.stdout:3/514: rename df/d13/d1b/d4b/f97 to df/d13/d6d/d9a/faa 0 2024-06-03T01:20:57.106 INFO:tasks.workunit.client.0.smithi032.stdout:6/143: rmdir d8/d11 39 2024-06-03T01:20:57.108 INFO:tasks.workunit.client.0.smithi032.stdout:6/144: getdents d8/d11 0 2024-06-03T01:20:57.109 INFO:tasks.workunit.client.1.smithi044.stdout:3/515: symlink df/d1e/d3a/d84/lab 0 2024-06-03T01:20:57.109 INFO:tasks.workunit.client.1.smithi044.stdout:3/516: stat df/d13/f3f 0 2024-06-03T01:20:57.109 INFO:tasks.workunit.client.1.smithi044.stdout:8/649: dread d5/d20/d47/f71 [0,4194304] 0 2024-06-03T01:20:57.111 INFO:tasks.workunit.client.1.smithi044.stdout:8/650: unlink d5/d20/db3/fcc 0 2024-06-03T01:20:57.111 INFO:tasks.workunit.client.1.smithi044.stdout:8/651: write d5/f25 [4358254,115213] 0 2024-06-03T01:20:57.114 INFO:tasks.workunit.client.1.smithi044.stdout:8/652: creat d5/d55/d56/d5c/dbb/d79/fd3 x:0 0 0 2024-06-03T01:20:57.116 INFO:tasks.workunit.client.0.smithi032.stdout:8/109: dwrite dc/de/f10 [0,4194304] 0 2024-06-03T01:20:57.119 INFO:tasks.workunit.client.1.smithi044.stdout:3/517: write df/f17 [9288611,92289] 0 2024-06-03T01:20:57.119 INFO:tasks.workunit.client.1.smithi044.stdout:8/653: mkdir d5/dd4 0 2024-06-03T01:20:57.120 INFO:tasks.workunit.client.1.smithi044.stdout:9/436: sync 2024-06-03T01:20:57.121 INFO:tasks.workunit.client.1.smithi044.stdout:9/437: read - d6/dd/d51/d16/d27/d5e/d71/f78 zero size 2024-06-03T01:20:57.122 INFO:tasks.workunit.client.1.smithi044.stdout:3/518: symlink df/d1e/d3a/lac 0 2024-06-03T01:20:57.122 INFO:tasks.workunit.client.1.smithi044.stdout:3/519: read - df/d1e/d3a/f87 zero size 2024-06-03T01:20:57.122 INFO:tasks.workunit.client.1.smithi044.stdout:3/520: stat df/d13/d6d/la0 0 2024-06-03T01:20:57.123 INFO:tasks.workunit.client.0.smithi032.stdout:8/110: mkdir dc/de/d1e 0 2024-06-03T01:20:57.124 INFO:tasks.workunit.client.0.smithi032.stdout:8/111: chown dc/de 356702542 1 2024-06-03T01:20:57.124 INFO:tasks.workunit.client.0.smithi032.stdout:8/112: truncate f1 4663354 0 2024-06-03T01:20:57.124 INFO:tasks.workunit.client.1.smithi044.stdout:9/438: getdents d6/dd/d51/d16/d4d/d55/d68 0 2024-06-03T01:20:57.127 INFO:tasks.workunit.client.1.smithi044.stdout:3/521: mkdir df/d13/d6d/d9a/dad 0 2024-06-03T01:20:57.127 INFO:tasks.workunit.client.0.smithi032.stdout:8/113: link dc/de/f10 dc/f1f 0 2024-06-03T01:20:57.128 INFO:tasks.workunit.client.1.smithi044.stdout:6/767: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fac [0,4194304] 0 2024-06-03T01:20:57.129 INFO:tasks.workunit.client.1.smithi044.stdout:9/439: creat d6/dd/d51/d16/d5f/d79/f90 x:0 0 0 2024-06-03T01:20:57.131 INFO:tasks.workunit.client.1.smithi044.stdout:3/522: rename df/d13/f26 to df/d13/d1b/fae 0 2024-06-03T01:20:57.131 INFO:tasks.workunit.client.1.smithi044.stdout:3/523: fdatasync df/f65 0 2024-06-03T01:20:57.131 INFO:tasks.workunit.client.1.smithi044.stdout:3/524: fdatasync df/f49 0 2024-06-03T01:20:57.137 INFO:tasks.workunit.client.1.smithi044.stdout:9/440: rmdir d6/dd/d51/d46 39 2024-06-03T01:20:57.138 INFO:tasks.workunit.client.1.smithi044.stdout:9/441: rmdir d6/dd/d51 39 2024-06-03T01:20:57.139 INFO:tasks.workunit.client.1.smithi044.stdout:9/442: symlink d6/dd/d51/d16/d27/d5e/l91 0 2024-06-03T01:20:57.139 INFO:tasks.workunit.client.1.smithi044.stdout:9/443: write d6/f65 [200162,100043] 0 2024-06-03T01:20:57.139 INFO:tasks.workunit.client.1.smithi044.stdout:9/444: stat d6/dd/d51/d46/f76 0 2024-06-03T01:20:57.142 INFO:tasks.workunit.client.1.smithi044.stdout:6/768: dread d2/d23/d55/d5f/d53/f61 [0,4194304] 0 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:6/769: creat d2/d23/d30/dfe/f10b x:0 0 0 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/718: dwrite d3/d9/f86 [0,4194304] 0 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/719: creat d3/d9/d36/f108 x:0 0 0 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/720: chown d3/d5c 79468783 1 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/721: readlink d3/d9/l8a 0 2024-06-03T01:20:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/722: fdatasync d3/da6/d20/d25/f40 0 2024-06-03T01:20:57.145 INFO:tasks.workunit.client.1.smithi044.stdout:6/770: mkdir d2/d23/d30/d1e/d38/d77/d10c 0 2024-06-03T01:20:57.145 INFO:tasks.workunit.client.1.smithi044.stdout:6/771: read d2/d23/d30/d1e/d38/d77/f8e [344507,57211] 0 2024-06-03T01:20:57.148 INFO:tasks.workunit.client.1.smithi044.stdout:0/723: creat d3/da6/d20/d25/dbf/f109 x:0 0 0 2024-06-03T01:20:57.148 INFO:tasks.workunit.client.1.smithi044.stdout:0/724: read - d3/da6/d21/d94/da3/dd8/ff1 zero size 2024-06-03T01:20:57.149 INFO:tasks.workunit.client.1.smithi044.stdout:0/725: write d3/da6/d21/d94/da3/dd8/f107 [145923,8848] 0 2024-06-03T01:20:57.149 INFO:tasks.workunit.client.1.smithi044.stdout:6/772: creat d2/d23/d44/dcb/f10d x:0 0 0 2024-06-03T01:20:57.151 INFO:tasks.workunit.client.1.smithi044.stdout:0/726: link d3/da6/d20/d25/f40 d3/da6/d21/f10a 0 2024-06-03T01:20:57.152 INFO:tasks.workunit.client.1.smithi044.stdout:6/773: mknod d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/dc0/c10e 0 2024-06-03T01:20:57.153 INFO:tasks.workunit.client.1.smithi044.stdout:0/727: link d3/d9/d55/d5a/d29/f4f d3/d9/d55/d73/f10b 0 2024-06-03T01:20:57.154 INFO:tasks.workunit.client.1.smithi044.stdout:6/774: link d2/d23/d30/d1e/d38/d6c/d5c/d79/f81 d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/dc0/f10f 0 2024-06-03T01:20:57.155 INFO:tasks.workunit.client.1.smithi044.stdout:8/654: dwrite d5/d55/fad [0,4194304] 0 2024-06-03T01:20:57.155 INFO:tasks.workunit.client.1.smithi044.stdout:0/728: mkdir d3/d9/d55/d7b/dfc/d10c 0 2024-06-03T01:20:57.156 INFO:tasks.workunit.client.1.smithi044.stdout:0/729: read d3/d9/f1d [2162417,46192] 0 2024-06-03T01:20:57.156 INFO:tasks.workunit.client.0.smithi032.stdout:8/114: dwrite f1 [0,4194304] 0 2024-06-03T01:20:57.156 INFO:tasks.workunit.client.0.smithi032.stdout:6/145: dwrite f3 [4194304,4194304] 0 2024-06-03T01:20:57.157 INFO:tasks.workunit.client.1.smithi044.stdout:0/730: dread d3/d6b/d95/db3/ffe [0,4194304] 0 2024-06-03T01:20:57.157 INFO:tasks.workunit.client.1.smithi044.stdout:0/731: chown d3/da6/dc1/dc2/de4/c104 900 1 2024-06-03T01:20:57.158 INFO:tasks.workunit.client.1.smithi044.stdout:6/775: unlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/c98 0 2024-06-03T01:20:57.158 INFO:tasks.workunit.client.0.smithi032.stdout:8/115: rename dc/d14/f1a to dc/d14/f20 0 2024-06-03T01:20:57.158 INFO:tasks.workunit.client.1.smithi044.stdout:3/525: dwrite df/d13/d39/d3d/f8b [0,4194304] 0 2024-06-03T01:20:57.158 INFO:tasks.workunit.client.1.smithi044.stdout:3/526: readlink df/d1e/d3a/l9f 0 2024-06-03T01:20:57.159 INFO:tasks.workunit.client.0.smithi032.stdout:6/146: mknod d8/d25/c2f 0 2024-06-03T01:20:57.160 INFO:tasks.workunit.client.0.smithi032.stdout:8/116: truncate dc/fd 3607984 0 2024-06-03T01:20:57.160 INFO:tasks.workunit.client.1.smithi044.stdout:9/445: dwrite d6/dd/f3d [0,4194304] 0 2024-06-03T01:20:57.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/147: getdents d8/d11/d20 0 2024-06-03T01:20:57.161 INFO:tasks.workunit.client.0.smithi032.stdout:6/148: read d8/fd [241043,19723] 0 2024-06-03T01:20:57.161 INFO:tasks.workunit.client.0.smithi032.stdout:6/149: fdatasync d8/f9 0 2024-06-03T01:20:57.162 INFO:tasks.workunit.client.0.smithi032.stdout:6/150: symlink d8/d11/d28/l30 0 2024-06-03T01:20:57.162 INFO:tasks.workunit.client.1.smithi044.stdout:8/655: mkdir d5/d20/d2b/d58/dd5 0 2024-06-03T01:20:57.165 INFO:tasks.workunit.client.1.smithi044.stdout:0/732: rename d3/da6/d20/d25/fc3 to d3/d9/d55/d5a/d29/f10d 0 2024-06-03T01:20:57.165 INFO:tasks.workunit.client.1.smithi044.stdout:3/527: rename df/d1e to df/d1e/d2a/d70/daf 22 2024-06-03T01:20:57.165 INFO:tasks.workunit.client.1.smithi044.stdout:0/733: readlink d3/d9/d55/d5a/l3d 0 2024-06-03T01:20:57.166 INFO:tasks.workunit.client.1.smithi044.stdout:6/776: mknod d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/dc0/c110 0 2024-06-03T01:20:57.167 INFO:tasks.workunit.client.1.smithi044.stdout:6/777: write d2/d23/d30/d1e/d38/d6c/fbf [723276,8036] 0 2024-06-03T01:20:57.168 INFO:tasks.workunit.client.0.smithi032.stdout:6/151: dread d8/ff [0,4194304] 0 2024-06-03T01:20:57.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/734: fdatasync d3/da6/d21/d94/da3/dd8/f107 0 2024-06-03T01:20:57.170 INFO:tasks.workunit.client.0.smithi032.stdout:6/152: mkdir d8/d11/d20/d31 0 2024-06-03T01:20:57.171 INFO:tasks.workunit.client.0.smithi032.stdout:6/153: write d8/d25/f1f [875738,104838] 0 2024-06-03T01:20:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:9/446: creat d6/f92 x:0 0 0 2024-06-03T01:20:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:8/656: read d5/d16/d1d/f31 [2265378,64539] 0 2024-06-03T01:20:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:8/657: dread - d5/d55/d74/db4/dba/fc3 zero size 2024-06-03T01:20:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:3/528: mknod df/cb0 0 2024-06-03T01:20:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:3/529: write df/d13/d39/d3d/d6a/fa8 [644882,16678] 0 2024-06-03T01:20:57.176 INFO:tasks.workunit.client.0.smithi032.stdout:8/117: dread dc/f1f [0,4194304] 0 2024-06-03T01:20:57.178 INFO:tasks.workunit.client.1.smithi044.stdout:9/447: symlink d6/dd/l93 0 2024-06-03T01:20:57.181 INFO:tasks.workunit.client.1.smithi044.stdout:8/658: link d5/d20/d2b/d58/d89/dbf/l95 d5/d16/d81/d9b/ld6 0 2024-06-03T01:20:57.183 INFO:tasks.workunit.client.1.smithi044.stdout:3/530: unlink c0 0 2024-06-03T01:20:57.203 INFO:tasks.workunit.client.0.smithi032.stdout:6/154: dwrite d8/fb [4194304,4194304] 0 2024-06-03T01:20:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:6/778: rmdir d2/d23/d30 39 2024-06-03T01:20:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:6/779: write d2/d23/d44/d4d/da0/fce [437927,80088] 0 2024-06-03T01:20:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:6/780: getdents d2/d23/d30/d1e/d38/d6c/dc2/d104 0 2024-06-03T01:20:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:6/781: write d2/d23/d30/f7f [584421,46705] 0 2024-06-03T01:20:57.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/782: mknod d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/d102/c111 0 2024-06-03T01:20:57.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/783: chown d2/d23/d30/ce7 90003 1 2024-06-03T01:20:57.212 INFO:tasks.workunit.client.1.smithi044.stdout:0/735: dwrite d3/d9/d55/d5a/fe9 [0,4194304] 0 2024-06-03T01:20:57.214 INFO:tasks.workunit.client.0.smithi032.stdout:6/155: dread f4 [0,4194304] 0 2024-06-03T01:20:57.220 INFO:tasks.workunit.client.1.smithi044.stdout:0/736: dread d3/f91 [0,4194304] 0 2024-06-03T01:20:57.221 INFO:tasks.workunit.client.1.smithi044.stdout:0/737: fsync d3/d9/f53 0 2024-06-03T01:20:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr fail", "who": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:20:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: osdmap e47: 6 total, 6 up, 6 in 2024-06-03T01:20:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.14219 172.21.15.32:0/1592650844' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "mgr fail", "who": "smithi032.kfcfaa"}]': finished 2024-06-03T01:20:57.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: mgrmap e21: smithi044.wszjlw(active, starting, since 0.00801018s) 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:20:57.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: Manager daemon smithi044.wszjlw is now available 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/trash_purge_schedule"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.wszjlw/trash_purge_schedule"}]: dispatch 2024-06-03T01:20:57.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: [03/Jun/2024:01:20:56] ENGINE Bus STARTING 2024-06-03T01:20:57.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:56 smithi044 ceph-mon[36956]: [03/Jun/2024:01:20:56] ENGINE Serving on http://172.21.15.44:8765 2024-06-03T01:20:57.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/738: mknod d3/d9/d55/d5a/dde/c10e 0 2024-06-03T01:20:57.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/739: dread - d3/da6/dc1/dc2/ff9 zero size 2024-06-03T01:20:57.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/740: symlink d3/d9/d36/l10f 0 2024-06-03T01:20:57.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/659: dwrite d5/d20/d47/d64/fa0 [0,4194304] 0 2024-06-03T01:20:57.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/660: truncate d5/d20/f69 1361864 0 2024-06-03T01:20:57.233 INFO:tasks.workunit.client.1.smithi044.stdout:8/661: dread d5/d20/d47/f71 [0,4194304] 0 2024-06-03T01:20:57.234 INFO:tasks.workunit.client.1.smithi044.stdout:8/662: truncate d5/d55/f86 1265597 0 2024-06-03T01:20:57.238 INFO:tasks.workunit.client.1.smithi044.stdout:0/741: write d3/f6d [3911695,7331] 0 2024-06-03T01:20:57.238 INFO:tasks.workunit.client.1.smithi044.stdout:0/742: chown d3/d9/d55/d5a/d29/f10d 3488323 1 2024-06-03T01:20:57.240 INFO:tasks.workunit.client.1.smithi044.stdout:0/743: link d3/d9/d36/fe0 d3/da6/dc1/dc2/df8/f110 0 2024-06-03T01:20:57.241 INFO:tasks.workunit.client.1.smithi044.stdout:0/744: mkdir d3/d6b/d111 0 2024-06-03T01:20:57.242 INFO:tasks.workunit.client.1.smithi044.stdout:0/745: rename d3/d9/d55/f79 to d3/da6/d20/d25/f112 0 2024-06-03T01:20:57.243 INFO:tasks.workunit.client.1.smithi044.stdout:9/448: dwrite d6/dd/d51/d16/d27/f53 [0,4194304] 0 2024-06-03T01:20:57.245 INFO:tasks.workunit.client.0.smithi032.stdout:8/118: dwrite dc/f1f [4194304,4194304] 0 2024-06-03T01:20:57.247 INFO:tasks.workunit.client.1.smithi044.stdout:0/746: read d3/d9/d55/d7b/fb0 [3923211,40952] 0 2024-06-03T01:20:57.247 INFO:tasks.workunit.client.1.smithi044.stdout:6/784: dwrite d2/d23/d44/d4d/da0/fae [0,4194304] 0 2024-06-03T01:20:57.247 INFO:tasks.workunit.client.1.smithi044.stdout:3/531: dwrite df/d13/d1b/f64 [0,4194304] 0 2024-06-03T01:20:57.247 INFO:tasks.workunit.client.1.smithi044.stdout:3/532: chown df/d13/d39/d3d/d43/d71/fa2 265677652 1 2024-06-03T01:20:57.248 INFO:tasks.workunit.client.0.smithi032.stdout:6/156: dwrite d8/fd [0,4194304] 0 2024-06-03T01:20:57.248 INFO:tasks.workunit.client.0.smithi032.stdout:8/119: symlink dc/de/l21 0 2024-06-03T01:20:57.248 INFO:tasks.workunit.client.0.smithi032.stdout:8/120: creat dc/d14/f22 x:0 0 0 2024-06-03T01:20:57.248 INFO:tasks.workunit.client.1.smithi044.stdout:0/747: dread d3/da6/dc1/dc2/df8/f110 [0,4194304] 0 2024-06-03T01:20:57.249 INFO:tasks.workunit.client.1.smithi044.stdout:8/663: dread d5/d16/d1d/fa5 [0,4194304] 0 2024-06-03T01:20:57.249 INFO:tasks.workunit.client.1.smithi044.stdout:8/664: creat d5/d55/d74/db4/fd7 x:0 0 0 2024-06-03T01:20:57.252 INFO:tasks.workunit.client.0.smithi032.stdout:7/128: sync 2024-06-03T01:20:57.252 INFO:tasks.workunit.client.0.smithi032.stdout:7/129: creat dc/d12/d1b/f2c x:0 0 0 2024-06-03T01:20:57.253 INFO:tasks.workunit.client.0.smithi032.stdout:7/130: creat dc/f2d x:0 0 0 2024-06-03T01:20:57.253 INFO:tasks.workunit.client.0.smithi032.stdout:7/131: creat dc/d12/d22/f2e x:0 0 0 2024-06-03T01:20:57.253 INFO:tasks.workunit.client.1.smithi044.stdout:9/449: dread d6/dd/d51/d16/d5f/f7c [0,4194304] 0 2024-06-03T01:20:57.259 INFO:tasks.workunit.client.1.smithi044.stdout:8/665: dread d5/d55/d74/f94 [0,4194304] 0 2024-06-03T01:20:57.270 INFO:tasks.workunit.client.1.smithi044.stdout:0/748: dwrite d3/d9/fc7 [0,4194304] 0 2024-06-03T01:20:57.271 INFO:tasks.workunit.client.0.smithi032.stdout:6/157: mknod d8/d11/d28/c32 0 2024-06-03T01:20:57.273 INFO:tasks.workunit.client.0.smithi032.stdout:8/121: mknod dc/de/d1e/c23 0 2024-06-03T01:20:57.277 INFO:tasks.workunit.client.0.smithi032.stdout:7/132: dread - dc/dd/f20 zero size 2024-06-03T01:20:57.277 INFO:tasks.workunit.client.1.smithi044.stdout:3/533: rename df/d13/d1b/d56/d5c/c62 to df/d13/cb1 0 2024-06-03T01:20:57.278 INFO:tasks.workunit.client.0.smithi032.stdout:6/158: symlink d8/d11/d20/d31/l33 0 2024-06-03T01:20:57.278 INFO:tasks.workunit.client.0.smithi032.stdout:7/133: mkdir dc/d12/d2a/d2f 0 2024-06-03T01:20:57.279 INFO:tasks.workunit.client.0.smithi032.stdout:6/159: getdents d8 0 2024-06-03T01:20:57.280 INFO:tasks.workunit.client.0.smithi032.stdout:7/134: chown l0 12642 1 2024-06-03T01:20:57.280 INFO:tasks.workunit.client.1.smithi044.stdout:6/785: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/l112 0 2024-06-03T01:20:57.282 INFO:tasks.workunit.client.0.smithi032.stdout:6/160: creat d8/d11/f34 x:0 0 0 2024-06-03T01:20:57.282 INFO:tasks.workunit.client.0.smithi032.stdout:6/161: readlink d8/le 0 2024-06-03T01:20:57.282 INFO:tasks.workunit.client.0.smithi032.stdout:8/122: write dc/de/f10 [7228600,130350] 0 2024-06-03T01:20:57.283 INFO:tasks.workunit.client.0.smithi032.stdout:7/135: link dc/d12/f13 dc/f30 0 2024-06-03T01:20:57.285 INFO:tasks.workunit.client.0.smithi032.stdout:7/136: symlink dc/d12/l31 0 2024-06-03T01:20:57.285 INFO:tasks.workunit.client.0.smithi032.stdout:7/137: rename dc to dc/d12/d22/d32 22 2024-06-03T01:20:57.285 INFO:tasks.workunit.client.0.smithi032.stdout:7/138: mkdir dc/d12/d24/d33 0 2024-06-03T01:20:57.286 INFO:tasks.workunit.client.1.smithi044.stdout:3/534: write df/d13/d6d/d9a/faa [2841245,72230] 0 2024-06-03T01:20:57.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/666: mknod d5/d55/d74/d82/cd8 0 2024-06-03T01:20:57.288 INFO:tasks.workunit.client.0.smithi032.stdout:8/123: read dc/f1f [3589629,23773] 0 2024-06-03T01:20:57.288 INFO:tasks.workunit.client.0.smithi032.stdout:8/124: creat dc/d14/f24 x:0 0 0 2024-06-03T01:20:57.288 INFO:tasks.workunit.client.0.smithi032.stdout:8/125: dread - dc/d14/f24 zero size 2024-06-03T01:20:57.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/667: read d5/d55/f5d [176567,38738] 0 2024-06-03T01:20:57.293 INFO:tasks.workunit.client.0.smithi032.stdout:8/126: mkdir dc/d25 0 2024-06-03T01:20:57.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/749: link d3/d9/d36/fb4 d3/d9/d36/d82/f113 0 2024-06-03T01:20:57.295 INFO:tasks.workunit.client.0.smithi032.stdout:8/127: truncate f8 172575 0 2024-06-03T01:20:57.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/139: read dc/d12/f13 [7589704,108498] 0 2024-06-03T01:20:57.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/140: chown dc/f30 917 1 2024-06-03T01:20:57.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/141: dread - dc/d12/f25 zero size 2024-06-03T01:20:57.297 INFO:tasks.workunit.client.1.smithi044.stdout:9/450: rename d6/d38/c50 to d6/dd/d51/d16/d5f/d8b/d8d/c94 0 2024-06-03T01:20:57.297 INFO:tasks.workunit.client.1.smithi044.stdout:9/451: fsync f2 0 2024-06-03T01:20:57.300 INFO:tasks.workunit.client.1.smithi044.stdout:8/668: unlink d5/d20/d33/f4c 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:1/696: sync 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:7/640: sync 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:4/630: sync 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:2/463: sync 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:5/549: sync 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:7/641: truncate d0/d3/d8/d95/d63/d89/da3/f66 1937104 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:2/464: creat db/d1b/d5d/f95 x:0 0 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:1/697: write d0/d3/d20/d40/d45/d6f/f98 [101543,69022] 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:4/631: write d17/d1c/d75/d7a/db8/d28/fd1 [995364,88825] 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:7/642: fdatasync d0/d3/d8/d95/f53 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:2/465: creat db/d1b/d2a/d67/f96 x:0 0 0 2024-06-03T01:20:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:1/698: creat d0/d3/d20/d40/d45/d6f/ff5 x:0 0 0 2024-06-03T01:20:57.302 INFO:tasks.workunit.client.1.smithi044.stdout:7/643: write d0/d3/d8/f6d [245211,65062] 0 2024-06-03T01:20:57.302 INFO:tasks.workunit.client.1.smithi044.stdout:0/750: symlink d3/d9/d55/d5a/dde/ded/l114 0 2024-06-03T01:20:57.306 INFO:tasks.workunit.client.1.smithi044.stdout:6/786: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/c3d to d2/d23/d30/d1e/d38/d77/d10c/c113 0 2024-06-03T01:20:57.309 INFO:tasks.workunit.client.1.smithi044.stdout:9/452: write d6/dd/d51/d16/f34 [104212,41959] 0 2024-06-03T01:20:57.309 INFO:tasks.workunit.client.1.smithi044.stdout:9/453: write d6/f84 [379013,119169] 0 2024-06-03T01:20:57.310 INFO:tasks.workunit.client.1.smithi044.stdout:5/550: mknod d2/d11/d1e/d21/d24/d93/d9d/d28/d66/cb0 0 2024-06-03T01:20:57.310 INFO:tasks.workunit.client.1.smithi044.stdout:5/551: dread - d2/d11/d3e/f99 zero size 2024-06-03T01:20:57.312 INFO:tasks.workunit.client.1.smithi044.stdout:9/454: dread d6/dd/d45/f56 [0,4194304] 0 2024-06-03T01:20:57.312 INFO:tasks.workunit.client.1.smithi044.stdout:9/455: chown d6/dd/d51/d46/f48 32126 1 2024-06-03T01:20:57.316 INFO:tasks.workunit.client.1.smithi044.stdout:4/632: creat d17/d1c/d75/d7a/db3/fd9 x:0 0 0 2024-06-03T01:20:57.317 INFO:tasks.workunit.client.1.smithi044.stdout:2/466: truncate db/d1b/d27/f39 4484147 0 2024-06-03T01:20:57.317 INFO:tasks.workunit.client.1.smithi044.stdout:4/633: read d17/d1c/d7e/fa2 [508486,63276] 0 2024-06-03T01:20:57.319 INFO:tasks.workunit.client.1.smithi044.stdout:1/699: creat d0/d16/d64/ff6 x:0 0 0 2024-06-03T01:20:57.319 INFO:tasks.workunit.client.1.smithi044.stdout:1/700: truncate d0/d3/d14/fe5 276976 0 2024-06-03T01:20:57.319 INFO:tasks.workunit.client.1.smithi044.stdout:7/644: dwrite d0/d3/d8/d32/d33/f57 [0,4194304] 0 2024-06-03T01:20:57.319 INFO:tasks.workunit.client.1.smithi044.stdout:0/751: mkdir d3/d6b/de8/d115 0 2024-06-03T01:20:57.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/787: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/f114 x:0 0 0 2024-06-03T01:20:57.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/788: creat d2/d23/d55/d5f/db2/f115 x:0 0 0 2024-06-03T01:20:57.325 INFO:tasks.workunit.client.0.smithi032.stdout:7/142: dwrite dc/f2b [0,4194304] 0 2024-06-03T01:20:57.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/467: symlink db/d1b/d27/d79/d91/d94/l97 0 2024-06-03T01:20:57.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/468: stat db/d1b/d2a/c46 0 2024-06-03T01:20:57.330 INFO:tasks.workunit.client.1.smithi044.stdout:4/634: rename d17/d1c/d75/d7a/db8/f99 to d17/d1c/d75/d7a/db8/d59/fda 0 2024-06-03T01:20:57.331 INFO:tasks.workunit.client.1.smithi044.stdout:4/635: write d17/d1c/d75/d7a/db8/fa5 [1469103,116881] 0 2024-06-03T01:20:57.331 INFO:tasks.workunit.client.1.smithi044.stdout:4/636: write d17/d1c/d75/d7a/db8/d28/d2b/f2e [3860803,112835] 0 2024-06-03T01:20:57.331 INFO:tasks.workunit.client.1.smithi044.stdout:5/552: dwrite d2/d11/d1e/f51 [8388608,4194304] 0 2024-06-03T01:20:57.332 INFO:tasks.workunit.client.1.smithi044.stdout:1/701: symlink d0/d3/d20/d40/d45/d4b/d5c/lf7 0 2024-06-03T01:20:57.333 INFO:tasks.workunit.client.1.smithi044.stdout:9/456: dwrite d6/f12 [0,4194304] 0 2024-06-03T01:20:57.337 INFO:tasks.workunit.client.1.smithi044.stdout:7/645: symlink d0/d3/d8/d95/d84/lcd 0 2024-06-03T01:20:57.341 INFO:tasks.workunit.client.0.smithi032.stdout:7/143: dread f6 [0,4194304] 0 2024-06-03T01:20:57.347 INFO:tasks.workunit.client.1.smithi044.stdout:4/637: creat d17/d63/fdb x:0 0 0 2024-06-03T01:20:57.347 INFO:tasks.workunit.client.1.smithi044.stdout:4/638: chown d17/d1c/d75/d7a/db8/d6a/d84 213608010 1 2024-06-03T01:20:57.347 INFO:tasks.workunit.client.1.smithi044.stdout:4/639: chown d17/d1c/d75/d7a/db8/l1f 3 1 2024-06-03T01:20:57.351 INFO:tasks.workunit.client.1.smithi044.stdout:1/702: truncate d0/d3/d20/d7/d13/d2d/fce 4307070 0 2024-06-03T01:20:57.354 INFO:tasks.workunit.client.1.smithi044.stdout:9/457: link d6/d26/c7a d6/dd/d51/d16/c95 0 2024-06-03T01:20:57.355 INFO:tasks.workunit.client.1.smithi044.stdout:2/469: dwrite db/d1b/d27/f63 [0,4194304] 0 2024-06-03T01:20:57.356 INFO:tasks.workunit.client.1.smithi044.stdout:1/703: mknod d0/d66/dc2/cf8 0 2024-06-03T01:20:57.356 INFO:tasks.workunit.client.1.smithi044.stdout:1/704: write d0/d3/d20/d40/f88 [681193,105863] 0 2024-06-03T01:20:57.356 INFO:tasks.workunit.client.1.smithi044.stdout:1/705: chown d0/f5 2597103 1 2024-06-03T01:20:57.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/458: rmdir d6/dd/d6e 0 2024-06-03T01:20:57.360 INFO:tasks.workunit.client.1.smithi044.stdout:2/470: mknod db/d15/d49/d6c/d72/c98 0 2024-06-03T01:20:57.361 INFO:tasks.workunit.client.1.smithi044.stdout:6/789: rename d2/d23/d44/d4d/da0 to d2/d23/d55/d5f/db2/d3f/d106/d116 0 2024-06-03T01:20:57.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/706: link d0/d3/d20/d7/d2f/db7/fc6 d0/d3/d14/dae/ff9 0 2024-06-03T01:20:57.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/707: fsync d0/d66/dbe/fc5 0 2024-06-03T01:20:57.368 INFO:tasks.workunit.client.1.smithi044.stdout:5/553: dwrite d2/d11/d1e/d21/d24/f38 [0,4194304] 0 2024-06-03T01:20:57.370 INFO:tasks.workunit.client.1.smithi044.stdout:4/640: rename d17/d1c/d75/f43 to d17/d1c/d75/d7a/db3/fdc 0 2024-06-03T01:20:57.371 INFO:tasks.workunit.client.1.smithi044.stdout:6/790: creat d2/d23/d30/f117 x:0 0 0 2024-06-03T01:20:57.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/641: dread d17/f69 [0,4194304] 0 2024-06-03T01:20:57.373 INFO:tasks.workunit.client.1.smithi044.stdout:7/646: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-06-03T01:20:57.374 INFO:tasks.workunit.client.1.smithi044.stdout:5/554: mkdir d2/d11/d1e/d21/d24/d9e/db1 0 2024-06-03T01:20:57.374 INFO:tasks.workunit.client.1.smithi044.stdout:5/555: stat d2/d11/d12/d2b/f34 0 2024-06-03T01:20:57.374 INFO:tasks.workunit.client.1.smithi044.stdout:5/556: chown d2/f9f 547310372 1 2024-06-03T01:20:57.376 INFO:tasks.workunit.client.1.smithi044.stdout:6/791: unlink d2/f2c 0 2024-06-03T01:20:57.378 INFO:tasks.workunit.client.1.smithi044.stdout:9/459: write d6/dd/d51/db/f32 [8194610,51989] 0 2024-06-03T01:20:57.380 INFO:tasks.workunit.client.1.smithi044.stdout:7/647: unlink d0/d3/d8/d95/d84/l85 0 2024-06-03T01:20:57.382 INFO:tasks.workunit.client.1.smithi044.stdout:5/557: creat d2/d11/fb2 x:0 0 0 2024-06-03T01:20:57.385 INFO:tasks.workunit.client.0.smithi032.stdout:4/137: sync 2024-06-03T01:20:57.385 INFO:tasks.workunit.client.0.smithi032.stdout:2/156: sync 2024-06-03T01:20:57.385 INFO:tasks.workunit.client.0.smithi032.stdout:1/100: sync 2024-06-03T01:20:57.385 INFO:tasks.workunit.client.0.smithi032.stdout:2/157: truncate d5/fe 408428 0 2024-06-03T01:20:57.385 INFO:tasks.workunit.client.0.smithi032.stdout:4/138: fsync d4/db/da/f2f 0 2024-06-03T01:20:57.386 INFO:tasks.workunit.client.0.smithi032.stdout:0/203: sync 2024-06-03T01:20:57.386 INFO:tasks.workunit.client.0.smithi032.stdout:0/204: fsync d1/fc 0 2024-06-03T01:20:57.387 INFO:tasks.workunit.client.0.smithi032.stdout:1/101: symlink d18/l21 0 2024-06-03T01:20:57.388 INFO:tasks.workunit.client.0.smithi032.stdout:1/102: creat d18/f22 x:0 0 0 2024-06-03T01:20:57.388 INFO:tasks.workunit.client.0.smithi032.stdout:1/103: fdatasync d18/f1c 0 2024-06-03T01:20:57.390 INFO:tasks.workunit.client.1.smithi044.stdout:4/642: dread d17/d1c/d75/d7a/db8/d6a/f9a [0,4194304] 0 2024-06-03T01:20:57.391 INFO:tasks.workunit.client.0.smithi032.stdout:2/158: symlink d5/l33 0 2024-06-03T01:20:57.392 INFO:tasks.workunit.client.0.smithi032.stdout:0/205: rename d1/da/d10/f27 to d1/da/d28/d3b/f3c 0 2024-06-03T01:20:57.394 INFO:tasks.workunit.client.1.smithi044.stdout:6/792: dread d2/d23/d30/d1e/f49 [0,4194304] 0 2024-06-03T01:20:57.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/708: dwrite d0/d3/d20/d40/fdd [0,4194304] 0 2024-06-03T01:20:57.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/709: truncate d0/d3/d20/d40/d45/d4b/d5c/f63 4621324 0 2024-06-03T01:20:57.396 INFO:tasks.workunit.client.0.smithi032.stdout:2/159: truncate d5/ff 3840965 0 2024-06-03T01:20:57.396 INFO:tasks.workunit.client.0.smithi032.stdout:2/160: fsync d5/da/f29 0 2024-06-03T01:20:57.398 INFO:tasks.workunit.client.0.smithi032.stdout:2/161: link d5/l14 d5/l34 0 2024-06-03T01:20:57.398 INFO:tasks.workunit.client.0.smithi032.stdout:2/162: truncate d5/da/d1a/f26 7448 0 2024-06-03T01:20:57.398 INFO:tasks.workunit.client.0.smithi032.stdout:2/163: write d5/fe [252025,74265] 0 2024-06-03T01:20:57.399 INFO:tasks.workunit.client.1.smithi044.stdout:4/643: symlink d17/d1c/d70/ldd 0 2024-06-03T01:20:57.400 INFO:tasks.workunit.client.0.smithi032.stdout:2/164: creat d5/f35 x:0 0 0 2024-06-03T01:20:57.400 INFO:tasks.workunit.client.0.smithi032.stdout:2/165: fdatasync d5/f13 0 2024-06-03T01:20:57.401 INFO:tasks.workunit.client.1.smithi044.stdout:2/471: dwrite db/d15/d49/d6c/f8c [0,4194304] 0 2024-06-03T01:20:57.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/166: mkdir d5/da/d1a/d2a/d36 0 2024-06-03T01:20:57.405 INFO:tasks.workunit.client.0.smithi032.stdout:2/167: unlink d5/da/c28 0 2024-06-03T01:20:57.405 INFO:tasks.workunit.client.0.smithi032.stdout:2/168: rename d5 to d5/da/d1a/d2a/d37 22 2024-06-03T01:20:57.405 INFO:tasks.workunit.client.0.smithi032.stdout:2/169: fdatasync d5/f19 0 2024-06-03T01:20:57.407 INFO:tasks.workunit.client.1.smithi044.stdout:6/793: write d2/d23/d55/d5f/d53/f7e [3990023,114789] 0 2024-06-03T01:20:57.407 INFO:tasks.workunit.client.1.smithi044.stdout:6/794: chown d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/cd3 200113202 1 2024-06-03T01:20:57.408 INFO:tasks.workunit.client.1.smithi044.stdout:4/644: symlink d17/d1c/d75/d7a/db3/lde 0 2024-06-03T01:20:57.410 INFO:tasks.workunit.client.0.smithi032.stdout:0/206: dread d1/da/d10/f19 [0,4194304] 0 2024-06-03T01:20:57.412 INFO:tasks.workunit.client.1.smithi044.stdout:1/710: write d0/d66/dbe/fc5 [2478963,71453] 0 2024-06-03T01:20:57.412 INFO:tasks.workunit.client.0.smithi032.stdout:0/207: mknod d1/da/d10/d29/c3d 0 2024-06-03T01:20:57.415 INFO:tasks.workunit.client.1.smithi044.stdout:7/648: dwrite d0/d3/d8/d95/d63/d7f/fa1 [0,4194304] 0 2024-06-03T01:20:57.415 INFO:tasks.workunit.client.1.smithi044.stdout:6/795: link d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/f9d d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/f118 0 2024-06-03T01:20:57.421 INFO:tasks.workunit.client.1.smithi044.stdout:4/645: unlink d17/d1c/d75/d7a/db8/d28/d2b/d8e/l90 0 2024-06-03T01:20:57.422 INFO:tasks.workunit.client.1.smithi044.stdout:2/472: write db/d15/d49/d6c/f8c [817027,8989] 0 2024-06-03T01:20:57.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/460: dwrite d6/dd/d51/f62 [0,4194304] 0 2024-06-03T01:20:57.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/461: stat d6/la 0 2024-06-03T01:20:57.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/462: readlink d6/dd/d51/d46/l58 0 2024-06-03T01:20:57.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/463: truncate d6/dd/f14 5175728 0 2024-06-03T01:20:57.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/464: write d6/f92 [609363,48262] 0 2024-06-03T01:20:57.424 INFO:tasks.workunit.client.0.smithi032.stdout:4/139: dwrite d4/db/da/d2d/d27/f30 [0,4194304] 0 2024-06-03T01:20:57.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/104: dwrite fa [0,4194304] 0 2024-06-03T01:20:57.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/105: rename d18 to d18/d23 22 2024-06-03T01:20:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:1/711: write d0/d3/d20/d40/d45/d4b/d80/f92 [3538203,15588] 0 2024-06-03T01:20:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:1/712: write d0/dd2/f60 [2320252,76322] 0 2024-06-03T01:20:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:1/713: chown d0/d3/d20/d40/cda 0 1 2024-06-03T01:20:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:6/796: creat d2/d23/d55/d5f/f119 x:0 0 0 2024-06-03T01:20:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:5/558: dwrite d2/d11/d12/f6c [0,4194304] 0 2024-06-03T01:20:57.428 INFO:tasks.workunit.client.1.smithi044.stdout:4/646: link d17/d1c/d75/d7a/db8/lc7 d17/d1c/d75/d7a/db8/d28/d2b/d8e/ldf 0 2024-06-03T01:20:57.430 INFO:tasks.workunit.client.0.smithi032.stdout:4/140: dread d4/db/da/f15 [0,4194304] 0 2024-06-03T01:20:57.430 INFO:tasks.workunit.client.0.smithi032.stdout:4/141: creat d4/db/da/f31 x:0 0 0 2024-06-03T01:20:57.430 INFO:tasks.workunit.client.0.smithi032.stdout:4/142: fsync d4/f29 0 2024-06-03T01:20:57.430 INFO:tasks.workunit.client.0.smithi032.stdout:4/143: creat d4/db/da/d2d/d27/f32 x:0 0 0 2024-06-03T01:20:57.431 INFO:tasks.workunit.client.0.smithi032.stdout:4/144: chown d4/db 1428886 1 2024-06-03T01:20:57.434 INFO:tasks.workunit.client.1.smithi044.stdout:2/473: dread db/d1b/d3f/f41 [0,4194304] 0 2024-06-03T01:20:57.435 INFO:tasks.workunit.client.1.smithi044.stdout:6/797: link d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/lb6 d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/l11a 0 2024-06-03T01:20:57.435 INFO:tasks.workunit.client.1.smithi044.stdout:9/465: dread d6/dd/d51/d16/d27/f64 [0,4194304] 0 2024-06-03T01:20:57.435 INFO:tasks.workunit.client.1.smithi044.stdout:6/798: dread - d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/f100 zero size 2024-06-03T01:20:57.437 INFO:tasks.workunit.client.0.smithi032.stdout:4/145: creat d4/db/da/d1d/f33 x:0 0 0 2024-06-03T01:20:57.439 INFO:tasks.workunit.client.0.smithi032.stdout:4/146: write d4/db/f1c [601781,127376] 0 2024-06-03T01:20:57.439 INFO:tasks.workunit.client.0.smithi032.stdout:1/106: write fa [1345109,60067] 0 2024-06-03T01:20:57.440 INFO:tasks.workunit.client.1.smithi044.stdout:5/559: unlink d2/d11/d12/d2b/c96 0 2024-06-03T01:20:57.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/647: mkdir d17/d1c/de0 0 2024-06-03T01:20:57.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/648: write d17/d1c/d75/d7a/db8/d28/d2b/f4a [221088,36875] 0 2024-06-03T01:20:57.445 INFO:tasks.workunit.client.1.smithi044.stdout:5/560: mkdir d2/d11/d12/dad/db3 0 2024-06-03T01:20:57.445 INFO:tasks.workunit.client.1.smithi044.stdout:2/474: dread db/d1b/d3f/f61 [0,4194304] 0 2024-06-03T01:20:57.449 INFO:tasks.workunit.client.1.smithi044.stdout:4/649: rename ff to d17/d1c/d75/fe1 0 2024-06-03T01:20:57.449 INFO:tasks.workunit.client.1.smithi044.stdout:4/650: write d17/d1c/d75/d7a/db8/d28/d2b/f4a [4071571,67404] 0 2024-06-03T01:20:57.449 INFO:tasks.workunit.client.1.smithi044.stdout:4/651: write d17/d1c/d75/d7a/db8/d28/f33 [1531733,77319] 0 2024-06-03T01:20:57.449 INFO:tasks.workunit.client.1.smithi044.stdout:4/652: chown d17/d1c/d75/d7a/db3/l79 59165855 1 2024-06-03T01:20:57.450 INFO:tasks.workunit.client.1.smithi044.stdout:4/653: write d17/d1c/d75/d7a/db8/d28/d2b/f40 [414276,37239] 0 2024-06-03T01:20:57.450 INFO:tasks.workunit.client.1.smithi044.stdout:4/654: write d17/f7c [744521,40610] 0 2024-06-03T01:20:57.451 INFO:tasks.workunit.client.1.smithi044.stdout:4/655: truncate d17/fd8 196005 0 2024-06-03T01:20:57.451 INFO:tasks.workunit.client.1.smithi044.stdout:2/475: creat db/d1b/f99 x:0 0 0 2024-06-03T01:20:57.451 INFO:tasks.workunit.client.1.smithi044.stdout:2/476: chown db/d1b/d27/l3b 3981 1 2024-06-03T01:20:57.451 INFO:tasks.workunit.client.1.smithi044.stdout:2/477: fdatasync db/d1b/d27/d79/d91/f92 0 2024-06-03T01:20:57.452 INFO:tasks.workunit.client.1.smithi044.stdout:4/656: mknod d17/ce2 0 2024-06-03T01:20:57.454 INFO:tasks.workunit.client.1.smithi044.stdout:2/478: getdents db 0 2024-06-03T01:20:57.454 INFO:tasks.workunit.client.1.smithi044.stdout:2/479: read - db/d1b/d3f/f59 zero size 2024-06-03T01:20:57.458 INFO:tasks.workunit.client.1.smithi044.stdout:2/480: rename db/d15/d49/d6c/d72/d7a/d4e/d7e/f89 to db/dd/f9a 0 2024-06-03T01:20:57.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/208: dwrite d1/f35 [0,4194304] 0 2024-06-03T01:20:57.462 INFO:tasks.workunit.client.0.smithi032.stdout:0/209: mknod d1/da/d10/d29/d31/c3e 0 2024-06-03T01:20:57.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/657: dread d17/d1c/d75/f4b [0,4194304] 0 2024-06-03T01:20:57.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/658: stat d17/d1c/d75/d7a/db8/c56 0 2024-06-03T01:20:57.463 INFO:tasks.workunit.client.1.smithi044.stdout:1/714: dwrite d0/d3/d20/d7/d13/f24 [4194304,4194304] 0 2024-06-03T01:20:57.465 INFO:tasks.workunit.client.1.smithi044.stdout:4/659: mkdir d17/d63/de3 0 2024-06-03T01:20:57.465 INFO:tasks.workunit.client.1.smithi044.stdout:4/660: dread - d17/d1c/d75/d7a/db8/db9/fc8 zero size 2024-06-03T01:20:57.466 INFO:tasks.workunit.client.0.smithi032.stdout:0/210: rename d1/da/d10/d29/d31/c3e to d1/da/d28/c3f 0 2024-06-03T01:20:57.466 INFO:tasks.workunit.client.0.smithi032.stdout:0/211: chown d1/da/d28/c3f 18 1 2024-06-03T01:20:57.466 INFO:tasks.workunit.client.1.smithi044.stdout:1/715: truncate d0/d66/dbe/f75 4088447 0 2024-06-03T01:20:57.466 INFO:tasks.workunit.client.1.smithi044.stdout:1/716: chown d0/d3/d20/d40/d45/d4b/d80/lc1 26251 1 2024-06-03T01:20:57.467 INFO:tasks.workunit.client.0.smithi032.stdout:0/212: creat d1/f40 x:0 0 0 2024-06-03T01:20:57.468 INFO:tasks.workunit.client.1.smithi044.stdout:4/661: link d17/d1c/d7e/fa2 d17/d1c/d70/fe4 0 2024-06-03T01:20:57.470 INFO:tasks.workunit.client.1.smithi044.stdout:1/717: creat d0/d3/d14/d81/ffa x:0 0 0 2024-06-03T01:20:57.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/718: getdents d0/d3/d20/d40/d45/d6f 0 2024-06-03T01:20:57.476 INFO:tasks.workunit.client.1.smithi044.stdout:1/719: mknod d0/d3/d20/d40/d45/d6f/d8c/cfb 0 2024-06-03T01:20:57.478 INFO:tasks.workunit.client.1.smithi044.stdout:1/720: mknod d0/d3/d20/d7/d13/d2d/cfc 0 2024-06-03T01:20:57.479 INFO:tasks.workunit.client.1.smithi044.stdout:4/662: write d17/d1c/d75/fe1 [3484862,67331] 0 2024-06-03T01:20:57.479 INFO:tasks.workunit.client.1.smithi044.stdout:4/663: write d17/d1c/d75/d7a/db8/d28/f33 [2479624,37062] 0 2024-06-03T01:20:57.481 INFO:tasks.workunit.client.1.smithi044.stdout:9/466: dwrite d6/dd/d51/d16/d61/f7e [0,4194304] 0 2024-06-03T01:20:57.481 INFO:tasks.workunit.client.0.smithi032.stdout:2/170: dwrite d5/ff [0,4194304] 0 2024-06-03T01:20:57.482 INFO:tasks.workunit.client.0.smithi032.stdout:2/171: chown d5/l34 17001 1 2024-06-03T01:20:57.482 INFO:tasks.workunit.client.0.smithi032.stdout:4/147: dwrite d4/db/da/f2f [0,4194304] 0 2024-06-03T01:20:57.482 INFO:tasks.workunit.client.1.smithi044.stdout:2/481: dwrite db/dd/f11 [0,4194304] 0 2024-06-03T01:20:57.482 INFO:tasks.workunit.client.1.smithi044.stdout:2/482: fdatasync db/d15/d49/d7b/f8a 0 2024-06-03T01:20:57.488 INFO:tasks.workunit.client.0.smithi032.stdout:4/148: mknod d4/c34 0 2024-06-03T01:20:57.489 INFO:tasks.workunit.client.0.smithi032.stdout:4/149: dread d4/db/ff [0,4194304] 0 2024-06-03T01:20:57.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/721: creat d0/d3/d14/dae/ffd x:0 0 0 2024-06-03T01:20:57.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/722: readlink d0/d3/d20/d7/lf 0 2024-06-03T01:20:57.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/723: dread - d0/d3/d20/d7/d13/fbc zero size 2024-06-03T01:20:57.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/172: rmdir d5 39 2024-06-03T01:20:57.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/173: fdatasync d5/da/f2c 0 2024-06-03T01:20:57.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/174: chown d5 171543 1 2024-06-03T01:20:57.491 INFO:tasks.workunit.client.1.smithi044.stdout:4/664: mkdir d17/d1c/d75/d7a/db8/d59/de5 0 2024-06-03T01:20:57.493 INFO:tasks.workunit.client.0.smithi032.stdout:3/173: sync 2024-06-03T01:20:57.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/150: link d4/db/da/d2d/d27/c28 d4/db/da/d2d/c35 0 2024-06-03T01:20:57.494 INFO:tasks.workunit.client.0.smithi032.stdout:2/175: truncate d5/da/f1e 1335872 0 2024-06-03T01:20:57.503 INFO:tasks.workunit.client.1.smithi044.stdout:9/467: symlink d6/dd/d51/d16/d5f/l96 0 2024-06-03T01:20:57.503 INFO:tasks.workunit.client.1.smithi044.stdout:9/468: creat d6/dd/d51/d16/d5f/d79/f97 x:0 0 0 2024-06-03T01:20:57.503 INFO:tasks.workunit.client.1.smithi044.stdout:9/469: chown d6/dd/d51/d16/d5f/d79/f97 3873189 1 2024-06-03T01:20:57.505 INFO:tasks.workunit.client.0.smithi032.stdout:4/151: read d4/db/da/f2f [1908280,75751] 0 2024-06-03T01:20:57.506 INFO:tasks.workunit.client.0.smithi032.stdout:4/152: write d4/db/da/d2d/d27/f2b [5225180,45470] 0 2024-06-03T01:20:57.506 INFO:tasks.workunit.client.0.smithi032.stdout:4/153: creat d4/db/f36 x:0 0 0 2024-06-03T01:20:57.507 INFO:tasks.workunit.client.1.smithi044.stdout:2/483: rename db/d1b/d27/d79/d91/d94 to db/d1b/d2a/d67/d53/d9b 0 2024-06-03T01:20:57.507 INFO:tasks.workunit.client.1.smithi044.stdout:2/484: dread - db/dd/f9a zero size 2024-06-03T01:20:57.511 INFO:tasks.workunit.client.1.smithi044.stdout:4/665: getdents d17/d1c/d9c 0 2024-06-03T01:20:57.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/176: dwrite d5/f1c [0,4194304] 0 2024-06-03T01:20:57.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/177: write d5/da/f2c [1269804,73220] 0 2024-06-03T01:20:57.514 INFO:tasks.workunit.client.0.smithi032.stdout:3/174: dwrite de/d29/d2e/d31/d24/f25 [0,4194304] 0 2024-06-03T01:20:57.514 INFO:tasks.workunit.client.0.smithi032.stdout:3/175: read f8 [1640684,102842] 0 2024-06-03T01:20:57.517 INFO:tasks.workunit.client.1.smithi044.stdout:9/470: mknod d6/dd/d51/d16/d4d/d67/c98 0 2024-06-03T01:20:57.518 INFO:tasks.workunit.client.0.smithi032.stdout:3/176: mknod de/d29/c34 0 2024-06-03T01:20:57.520 INFO:tasks.workunit.client.0.smithi032.stdout:3/177: link de/d29/f2b de/d29/f35 0 2024-06-03T01:20:57.520 INFO:tasks.workunit.client.1.smithi044.stdout:2/485: mknod db/d1b/d2a/d67/c9c 0 2024-06-03T01:20:57.521 INFO:tasks.workunit.client.0.smithi032.stdout:3/178: rmdir de/d29/d2e 39 2024-06-03T01:20:57.522 INFO:tasks.workunit.client.1.smithi044.stdout:4/666: creat d17/d1c/d75/fe6 x:0 0 0 2024-06-03T01:20:57.522 INFO:tasks.workunit.client.1.smithi044.stdout:4/667: readlink d17/d1c/d75/d7a/db8/d28/d2b/lc0 0 2024-06-03T01:20:57.522 INFO:tasks.workunit.client.0.smithi032.stdout:3/179: mknod de/d29/c36 0 2024-06-03T01:20:57.524 INFO:tasks.workunit.client.1.smithi044.stdout:9/471: truncate d6/f44 214786 0 2024-06-03T01:20:57.524 INFO:tasks.workunit.client.1.smithi044.stdout:9/472: chown d6/dd/d51/d16/d4d/d55/l6b 403035 1 2024-06-03T01:20:57.527 INFO:tasks.workunit.client.1.smithi044.stdout:2/486: truncate db/d15/d49/f5c 3617649 0 2024-06-03T01:20:57.527 INFO:tasks.workunit.client.1.smithi044.stdout:2/487: write db/d1b/d2a/d67/f96 [826003,71039] 0 2024-06-03T01:20:57.528 INFO:tasks.workunit.client.0.smithi032.stdout:4/154: dwrite d4/db/f1c [0,4194304] 0 2024-06-03T01:20:57.528 INFO:tasks.workunit.client.0.smithi032.stdout:4/155: truncate d4/db/da/d2d/f2a 195832 0 2024-06-03T01:20:57.531 INFO:tasks.workunit.client.1.smithi044.stdout:9/473: link d6/dd/d51/db/l25 d6/l99 0 2024-06-03T01:20:57.533 INFO:tasks.workunit.client.1.smithi044.stdout:4/668: dread d17/d1c/d75/d7a/db8/d6a/f73 [0,4194304] 0 2024-06-03T01:20:57.534 INFO:tasks.workunit.client.0.smithi032.stdout:4/156: fsync d4/db/da/d2d/d27/f2b 0 2024-06-03T01:20:57.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/474: symlink d6/d26/l9a 0 2024-06-03T01:20:57.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/475: write d6/dd/d45/f81 [473389,109390] 0 2024-06-03T01:20:57.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/724: rename d0/d66 to d0/dd2/dfe 0 2024-06-03T01:20:57.537 INFO:tasks.workunit.client.1.smithi044.stdout:4/669: symlink d17/d1c/d75/d7a/db3/le7 0 2024-06-03T01:20:57.537 INFO:tasks.workunit.client.0.smithi032.stdout:2/178: dwrite d5/da/f2f [0,4194304] 0 2024-06-03T01:20:57.538 INFO:tasks.workunit.client.1.smithi044.stdout:9/476: rmdir d6/dd/d51/d16/d27/d5e 39 2024-06-03T01:20:57.538 INFO:tasks.workunit.client.0.smithi032.stdout:2/179: unlink d5/l34 0 2024-06-03T01:20:57.538 INFO:tasks.workunit.client.0.smithi032.stdout:2/180: write d5/da/d2e/f32 [764379,97664] 0 2024-06-03T01:20:57.538 INFO:tasks.workunit.client.0.smithi032.stdout:2/181: readlink d5/l7 0 2024-06-03T01:20:57.539 INFO:tasks.workunit.client.0.smithi032.stdout:2/182: mkdir d5/d38 0 2024-06-03T01:20:57.541 INFO:tasks.workunit.client.0.smithi032.stdout:3/180: dwrite de/f15 [0,4194304] 0 2024-06-03T01:20:57.542 INFO:tasks.workunit.client.1.smithi044.stdout:4/670: rename d17/d1c/d75/d7a/db3/f97 to d17/d1c/d75/d7a/db8/d28/d2b/fe8 0 2024-06-03T01:20:57.542 INFO:tasks.workunit.client.1.smithi044.stdout:4/671: stat d17/d1c/d70/d7d 0 2024-06-03T01:20:57.542 INFO:tasks.workunit.client.1.smithi044.stdout:4/672: stat d17/d1c/d70 0 2024-06-03T01:20:57.542 INFO:tasks.workunit.client.1.smithi044.stdout:4/673: dread - d17/d1c/d75/d7a/db8/d59/f82 zero size 2024-06-03T01:20:57.542 INFO:tasks.workunit.client.0.smithi032.stdout:2/183: link d5/ff d5/da/f39 0 2024-06-03T01:20:57.543 INFO:tasks.workunit.client.0.smithi032.stdout:2/184: fsync d5/f19 0 2024-06-03T01:20:57.544 INFO:tasks.workunit.client.1.smithi044.stdout:1/725: getdents d0/d3/d20/d7/d13/d2d/dcd 0 2024-06-03T01:20:57.544 INFO:tasks.workunit.client.1.smithi044.stdout:1/726: dread - d0/d3/d20/d40/d45/d6f/ff5 zero size 2024-06-03T01:20:57.544 INFO:tasks.workunit.client.1.smithi044.stdout:1/727: truncate d0/dd2/dfe/ff0 113927 0 2024-06-03T01:20:57.552 INFO:tasks.workunit.client.0.smithi032.stdout:4/157: write d4/f7 [2015338,12085] 0 2024-06-03T01:20:57.552 INFO:tasks.workunit.client.1.smithi044.stdout:3/535: sync 2024-06-03T01:20:57.552 INFO:tasks.workunit.client.1.smithi044.stdout:3/536: creat df/d1e/d2a/d70/fb2 x:0 0 0 2024-06-03T01:20:57.552 INFO:tasks.workunit.client.1.smithi044.stdout:3/537: read df/d13/f16 [424308,86920] 0 2024-06-03T01:20:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:7/649: sync 2024-06-03T01:20:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:0/752: sync 2024-06-03T01:20:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:5/561: sync 2024-06-03T01:20:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:6/799: sync 2024-06-03T01:20:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/669: sync 2024-06-03T01:20:57.561 INFO:tasks.workunit.client.1.smithi044.stdout:0/753: symlink d3/d6b/d95/l116 0 2024-06-03T01:20:57.561 INFO:tasks.workunit.client.1.smithi044.stdout:1/728: read d0/d3/d20/d7/d13/d2d/f79 [441242,32712] 0 2024-06-03T01:20:57.561 INFO:tasks.workunit.client.1.smithi044.stdout:1/729: creat d0/d3/d20/fff x:0 0 0 2024-06-03T01:20:57.564 INFO:tasks.workunit.client.1.smithi044.stdout:5/562: unlink d2/d11/d12/c71 0 2024-06-03T01:20:57.564 INFO:tasks.workunit.client.1.smithi044.stdout:5/563: chown d2/d11/d1e/d21/d24/d93/d9d/d28/d66 15 1 2024-06-03T01:20:57.567 INFO:tasks.workunit.client.1.smithi044.stdout:6/800: rmdir d2/d23/d55/d5f/d99/dcc/ddb 0 2024-06-03T01:20:57.573 INFO:tasks.workunit.client.1.smithi044.stdout:3/538: rmdir df/d13/d1b/d56/d8e 0 2024-06-03T01:20:57.574 INFO:tasks.workunit.client.1.smithi044.stdout:0/754: getdents d3/d6b/de8 0 2024-06-03T01:20:57.574 INFO:tasks.workunit.client.1.smithi044.stdout:0/755: dread - d3/da6/d20/d25/dbf/f109 zero size 2024-06-03T01:20:57.577 INFO:tasks.workunit.client.1.smithi044.stdout:8/670: dread d5/d20/d2b/f48 [4194304,4194304] 0 2024-06-03T01:20:57.580 INFO:tasks.workunit.client.1.smithi044.stdout:5/564: unlink d2/d11/d1e/d21/d24/d93/d9d/d28/c55 0 2024-06-03T01:20:57.581 INFO:tasks.workunit.client.1.smithi044.stdout:2/488: write db/d15/d49/f5c [873191,103302] 0 2024-06-03T01:20:57.583 INFO:tasks.workunit.client.1.smithi044.stdout:6/801: link d2/d23/d30/d1e/d38/d6c/d5c/d79/fb8 d2/d23/d30/d1e/d38/d6c/dc2/d104/f11b 0 2024-06-03T01:20:57.583 INFO:tasks.workunit.client.0.smithi032.stdout:3/181: dwrite de/d29/d2e/d31/f18 [0,4194304] 0 2024-06-03T01:20:57.584 INFO:tasks.workunit.client.1.smithi044.stdout:0/756: symlink d3/d5c/de3/l117 0 2024-06-03T01:20:57.585 INFO:tasks.workunit.client.0.smithi032.stdout:3/182: getdents de/d29/d2e/d31/d23 0 2024-06-03T01:20:57.586 INFO:tasks.workunit.client.1.smithi044.stdout:8/671: symlink d5/d20/d47/d64/dd0/ld9 0 2024-06-03T01:20:57.587 INFO:tasks.workunit.client.1.smithi044.stdout:5/565: rename d2/d11/d12/d2b/fa9 to d2/d11/d12/d2b/d4a/d57/fb4 0 2024-06-03T01:20:57.587 INFO:tasks.workunit.client.1.smithi044.stdout:5/566: write d2/d11/d12/f1b [5011906,35095] 0 2024-06-03T01:20:57.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/489: truncate db/f26 3526069 0 2024-06-03T01:20:57.590 INFO:tasks.workunit.client.1.smithi044.stdout:6/802: symlink d2/d23/d55/d5f/d99/l11c 0 2024-06-03T01:20:57.592 INFO:tasks.workunit.client.1.smithi044.stdout:8/672: rename d5/d20/d47/c92 to d5/d55/d74/db4/cda 0 2024-06-03T01:20:57.592 INFO:tasks.workunit.client.1.smithi044.stdout:8/673: creat d5/d55/d74/d87/fdb x:0 0 0 2024-06-03T01:20:57.595 INFO:tasks.workunit.client.1.smithi044.stdout:5/567: dread d2/d11/d1e/d2f/f6e [0,4194304] 0 2024-06-03T01:20:57.596 INFO:tasks.workunit.client.1.smithi044.stdout:5/568: readlink d2/d11/d12/d2b/d44/l46 0 2024-06-03T01:20:57.598 INFO:tasks.workunit.client.1.smithi044.stdout:5/569: creat d2/d11/d12/fb5 x:0 0 0 2024-06-03T01:20:57.604 INFO:tasks.workunit.client.1.smithi044.stdout:6/803: dread d2/d23/d30/f1c [0,4194304] 0 2024-06-03T01:20:57.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/477: dwrite d6/dd/f3d [0,4194304] 0 2024-06-03T01:20:57.623 INFO:tasks.workunit.client.1.smithi044.stdout:4/674: dwrite d17/d1c/d75/f4b [4194304,4194304] 0 2024-06-03T01:20:57.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/478: write d6/dd/d51/d46/f76 [151426,107910] 0 2024-06-03T01:20:57.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/479: truncate d6/dd/d51/d16/d5f/d79/f97 506534 0 2024-06-03T01:20:57.637 INFO:tasks.workunit.client.1.smithi044.stdout:4/675: dread d17/d1c/d75/d7a/db8/d28/d2b/f2e [0,4194304] 0 2024-06-03T01:20:57.639 INFO:tasks.workunit.client.1.smithi044.stdout:4/676: mkdir d17/d1c/d75/d7a/db3/de9 0 2024-06-03T01:20:57.639 INFO:tasks.workunit.client.0.smithi032.stdout:2/185: dwrite d5/f19 [0,4194304] 0 2024-06-03T01:20:57.640 INFO:tasks.workunit.client.1.smithi044.stdout:7/650: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [4194304,4194304] 0 2024-06-03T01:20:57.643 INFO:tasks.workunit.client.1.smithi044.stdout:7/651: creat d0/d3/d8/d95/d63/fce x:0 0 0 2024-06-03T01:20:57.644 INFO:tasks.workunit.client.1.smithi044.stdout:4/677: dread d17/d1c/d75/d7a/db8/d28/fd1 [0,4194304] 0 2024-06-03T01:20:57.646 INFO:tasks.workunit.client.0.smithi032.stdout:2/186: dread d5/da/f1e [0,4194304] 0 2024-06-03T01:20:57.646 INFO:tasks.workunit.client.0.smithi032.stdout:2/187: write d5/da/d1a/f26 [405346,128117] 0 2024-06-03T01:20:57.647 INFO:tasks.workunit.client.1.smithi044.stdout:3/539: dwrite fd [0,4194304] 0 2024-06-03T01:20:57.647 INFO:tasks.workunit.client.1.smithi044.stdout:2/490: dwrite db/dd/f11 [4194304,4194304] 0 2024-06-03T01:20:57.647 INFO:tasks.workunit.client.1.smithi044.stdout:2/491: write db/d1b/d2a/d67/f96 [1161526,37368] 0 2024-06-03T01:20:57.648 INFO:tasks.workunit.client.0.smithi032.stdout:2/188: mknod d5/da/d1a/c3a 0 2024-06-03T01:20:57.648 INFO:tasks.workunit.client.0.smithi032.stdout:2/189: chown d5/ff 55 1 2024-06-03T01:20:57.648 INFO:tasks.workunit.client.0.smithi032.stdout:3/183: dwrite de/d29/d2e/d31/f21 [0,4194304] 0 2024-06-03T01:20:57.648 INFO:tasks.workunit.client.0.smithi032.stdout:3/184: chown de/l26 126 1 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/480: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.1.smithi044.stdout:5/570: dwrite d2/d11/d12/f6c [4194304,4194304] 0 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/481: dread - d6/dd/f87 zero size 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/482: truncate d6/dd/d45/f81 1343343 0 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/757: dwrite d3/d9/d55/d5a/d29/f4f [0,4194304] 0 2024-06-03T01:20:57.649 INFO:tasks.workunit.client.0.smithi032.stdout:4/158: dwrite d4/db/da/f2f [0,4194304] 0 2024-06-03T01:20:57.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/159: chown d4/db/da/d1d/f23 210714 1 2024-06-03T01:20:57.650 INFO:tasks.workunit.client.0.smithi032.stdout:2/190: rename d5/da/f18 to d5/d38/f3b 0 2024-06-03T01:20:57.650 INFO:tasks.workunit.client.0.smithi032.stdout:2/191: dread - d5/f23 zero size 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:8/674: dwrite d5/d20/f9e [0,4194304] 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:8/675: read d5/f25 [171581,6771] 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:8/676: fsync d5/d16/d1d/f38 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:1/730: dwrite d0/d3/d20/f6d [0,4194304] 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:1/731: write d0/f5 [2159894,130702] 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:1/732: fsync d0/d3/d20/d40/d45/d4b/d5c/d94/fc0 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:6/804: getdents d2/d23/d55/d5f/d99/dcc 0 2024-06-03T01:20:57.651 INFO:tasks.workunit.client.1.smithi044.stdout:6/805: write d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/fd9 [1342194,124768] 0 2024-06-03T01:20:57.655 INFO:tasks.workunit.client.1.smithi044.stdout:7/652: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:57.656 INFO:tasks.workunit.client.0.smithi032.stdout:3/185: truncate de/d29/d2e/d31/d24/f25 649593 0 2024-06-03T01:20:57.661 INFO:tasks.workunit.client.1.smithi044.stdout:5/571: creat d2/d11/d1e/d21/d24/d93/d9d/fb6 x:0 0 0 2024-06-03T01:20:57.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/483: mknod d6/dd/d51/d16/d4d/d55/d68/c9b 0 2024-06-03T01:20:57.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/484: readlink d6/dd/d51/l1d 0 2024-06-03T01:20:57.669 INFO:tasks.workunit.client.1.smithi044.stdout:2/492: write db/d1b/d2a/d67/f7c [2803177,106433] 0 2024-06-03T01:20:57.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/677: link d5/d55/d56/d5c/dbb/l8f d5/d20/ldc 0 2024-06-03T01:20:57.681 INFO:tasks.workunit.client.1.smithi044.stdout:6/806: mkdir d2/d23/d44/d4d/d11d 0 2024-06-03T01:20:57.681 INFO:tasks.workunit.client.1.smithi044.stdout:6/807: fsync d2/d23/d30/d1e/d38/d6c/dc2/d104/f11b 0 2024-06-03T01:20:57.681 INFO:tasks.workunit.client.1.smithi044.stdout:4/678: dwrite d17/d1c/d75/d7a/db8/d28/d2b/d8e/fc9 [0,4194304] 0 2024-06-03T01:20:57.683 INFO:tasks.workunit.client.0.smithi032.stdout:4/160: dwrite d4/db/da/d2d/d27/f2b [0,4194304] 0 2024-06-03T01:20:57.683 INFO:tasks.workunit.client.1.smithi044.stdout:5/572: mkdir d2/d11/d1e/d21/d24/d9e/db1/db7 0 2024-06-03T01:20:57.683 INFO:tasks.workunit.client.1.smithi044.stdout:5/573: chown d2/d11/d1e/l36 76719981 1 2024-06-03T01:20:57.684 INFO:tasks.workunit.client.1.smithi044.stdout:9/485: truncate d6/dd/d51/d16/d61/f6c 2178818 0 2024-06-03T01:20:57.684 INFO:tasks.workunit.client.1.smithi044.stdout:9/486: write d6/dd/d51/d46/f76 [109643,31953] 0 2024-06-03T01:20:57.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/161: mkdir d4/db/da/d37 0 2024-06-03T01:20:57.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/162: chown d4/db/da/d2d/d27/f2b 244684 1 2024-06-03T01:20:57.685 INFO:tasks.workunit.client.1.smithi044.stdout:8/678: mkdir d5/d20/d2b/d98/ddd 0 2024-06-03T01:20:57.687 INFO:tasks.workunit.client.1.smithi044.stdout:1/733: getdents d0/d3/d20/d40/dcb 0 2024-06-03T01:20:57.687 INFO:tasks.workunit.client.1.smithi044.stdout:1/734: dread - d0/f7d zero size 2024-06-03T01:20:57.689 INFO:tasks.workunit.client.1.smithi044.stdout:6/808: rename d2/d23/d30/d1e/d38/cd0 to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/dd1/c11e 0 2024-06-03T01:20:57.694 INFO:tasks.workunit.client.1.smithi044.stdout:3/540: dwrite df/d13/d39/d3d/d50/f6f [0,4194304] 0 2024-06-03T01:20:57.695 INFO:tasks.workunit.client.1.smithi044.stdout:5/574: creat d2/d11/d12/d2b/d44/fb8 x:0 0 0 2024-06-03T01:20:57.695 INFO:tasks.workunit.client.1.smithi044.stdout:5/575: truncate d2/d11/d12/fb5 7338 0 2024-06-03T01:20:57.698 INFO:tasks.workunit.client.1.smithi044.stdout:9/487: symlink d6/dd/d51/d16/d27/d5e/l9c 0 2024-06-03T01:20:57.700 INFO:tasks.workunit.client.1.smithi044.stdout:3/541: fdatasync df/d13/d1b/f64 0 2024-06-03T01:20:57.700 INFO:tasks.workunit.client.1.smithi044.stdout:1/735: write d0/dd2/dfe/dbe/f74 [2743301,20747] 0 2024-06-03T01:20:57.700 INFO:tasks.workunit.client.1.smithi044.stdout:5/576: getdents d2/d11/d1e 0 2024-06-03T01:20:57.701 INFO:tasks.workunit.client.1.smithi044.stdout:6/809: write d2/d23/d30/d1e/fbc [1933323,62256] 0 2024-06-03T01:20:57.701 INFO:tasks.workunit.client.1.smithi044.stdout:9/488: mknod d6/dd/d51/d16/d4d/d55/c9d 0 2024-06-03T01:20:57.702 INFO:tasks.workunit.client.1.smithi044.stdout:3/542: symlink df/d1e/d3a/d84/lb3 0 2024-06-03T01:20:57.705 INFO:tasks.workunit.client.1.smithi044.stdout:1/736: truncate d0/d3/d20/d40/d45/d4b/d5c/d94/fb3 2722733 0 2024-06-03T01:20:57.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/758: dwrite d3/da6/d21/f76 [0,4194304] 0 2024-06-03T01:20:57.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/759: write d3/d9/d36/d82/f9c [468162,126471] 0 2024-06-03T01:20:57.707 INFO:tasks.workunit.client.0.smithi032.stdout:3/186: dwrite f5 [4194304,4194304] 0 2024-06-03T01:20:57.709 INFO:tasks.workunit.client.1.smithi044.stdout:4/679: dwrite d17/d1c/d75/d7a/db8/db9/fc3 [0,4194304] 0 2024-06-03T01:20:57.710 INFO:tasks.workunit.client.1.smithi044.stdout:4/680: chown d17/d1c/d7e/fa2 0 1 2024-06-03T01:20:57.710 INFO:tasks.workunit.client.1.smithi044.stdout:8/679: dwrite d5/d55/d56/f8a [0,4194304] 0 2024-06-03T01:20:57.710 INFO:tasks.workunit.client.1.smithi044.stdout:8/680: creat d5/d55/d56/fde x:0 0 0 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:4/681: read d17/d1c/d75/d7a/db8/f68 [3176658,15436] 0 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:4/682: write d17/d1c/d75/d7a/db8/d28/f3e [5064864,58826] 0 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/810: rename d2/d23/f33 to d2/d23/d55/d5f/d99/f11f 0 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/811: fdatasync d2/d23/d55/d5f/d53/f61 0 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/812: dread - d2/d23/d55/d5f/f119 zero size 2024-06-03T01:20:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/813: write d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/fdc [800917,78942] 0 2024-06-03T01:20:57.723 INFO:tasks.workunit.client.1.smithi044.stdout:3/543: rmdir df 39 2024-06-03T01:20:57.724 INFO:tasks.workunit.client.1.smithi044.stdout:1/737: mknod d0/d3/d20/d40/d45/d6b/c100 0 2024-06-03T01:20:57.724 INFO:tasks.workunit.client.1.smithi044.stdout:4/683: dread d17/d1c/d75/d7a/db8/d59/f5f [0,4194304] 0 2024-06-03T01:20:57.727 INFO:tasks.workunit.client.0.smithi032.stdout:3/187: dwrite de/d29/d2e/d31/f32 [0,4194304] 0 2024-06-03T01:20:57.731 INFO:tasks.workunit.client.1.smithi044.stdout:6/814: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/f118 [0,4194304] 0 2024-06-03T01:20:57.732 INFO:tasks.workunit.client.1.smithi044.stdout:6/815: dread - d2/d23/d30/d1e/d38/d6c/f92 zero size 2024-06-03T01:20:57.732 INFO:tasks.workunit.client.1.smithi044.stdout:6/816: fsync d2/d23/d30/d1e/d38/d77/fe5 0 2024-06-03T01:20:57.732 INFO:tasks.workunit.client.0.smithi032.stdout:3/188: link f9 de/d29/d2e/f37 0 2024-06-03T01:20:57.732 INFO:tasks.workunit.client.0.smithi032.stdout:3/189: fdatasync f5 0 2024-06-03T01:20:57.733 INFO:tasks.workunit.client.0.smithi032.stdout:3/190: creat de/d29/d2e/d31/f38 x:0 0 0 2024-06-03T01:20:57.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/191: creat de/d29/d2e/d31/d24/f39 x:0 0 0 2024-06-03T01:20:57.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/192: write de/f11 [3569045,73106] 0 2024-06-03T01:20:57.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/193: write f8 [2530821,91761] 0 2024-06-03T01:20:57.736 INFO:tasks.workunit.client.1.smithi044.stdout:5/577: rmdir d2/d11 39 2024-06-03T01:20:57.737 INFO:tasks.workunit.client.0.smithi032.stdout:3/194: rename fb to de/d29/f3a 0 2024-06-03T01:20:57.739 INFO:tasks.workunit.client.1.smithi044.stdout:0/760: mknod d3/da6/d21/d94/da3/dd8/c118 0 2024-06-03T01:20:57.744 INFO:tasks.workunit.client.1.smithi044.stdout:8/681: rmdir d5/d20/d47/d64/dd0 39 2024-06-03T01:20:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/489: mknod d6/dd/d51/db/c9e 0 2024-06-03T01:20:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/490: fsync d6/dd/d51/d16/d27/d5e/d71/f78 0 2024-06-03T01:20:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/491: stat d6/dd/d51/d16/d27/c2e 0 2024-06-03T01:20:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/492: write d6/dd/d51/d16/d27/f39 [1043315,106679] 0 2024-06-03T01:20:57.747 INFO:tasks.workunit.client.1.smithi044.stdout:1/738: mkdir d0/d3/d20/d7/d13/d2d/d101 0 2024-06-03T01:20:57.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/684: symlink d17/d1c/d75/d7a/db8/d28/lea 0 2024-06-03T01:20:57.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/685: chown d17/d1c/d75/d7a/db8/l1f 515444192 1 2024-06-03T01:20:57.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/686: write d17/d1c/d75/d7a/db8/d35/fce [294083,30058] 0 2024-06-03T01:20:57.757 INFO:tasks.workunit.client.1.smithi044.stdout:6/817: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba/dc0/dff/l120 0 2024-06-03T01:20:57.760 INFO:tasks.workunit.client.1.smithi044.stdout:0/761: mknod d3/da6/d56/c119 0 2024-06-03T01:20:57.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/682: rename d5/d16/f5e to d5/d16/d1d/fdf 0 2024-06-03T01:20:57.763 INFO:tasks.workunit.client.1.smithi044.stdout:1/739: dwrite d0/d3/d20/d40/d45/d4b/d5c/d94/fca [0,4194304] 0 2024-06-03T01:20:57.763 INFO:tasks.workunit.client.1.smithi044.stdout:9/493: creat d6/dd/d51/d46/f9f x:0 0 0 2024-06-03T01:20:57.764 INFO:tasks.workunit.client.1.smithi044.stdout:4/687: mknod d17/d1c/d7e/ceb 0 2024-06-03T01:20:57.765 INFO:tasks.workunit.client.1.smithi044.stdout:4/688: read d17/d1c/d75/f4b [104939,37504] 0 2024-06-03T01:20:57.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/818: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/f121 x:0 0 0 2024-06-03T01:20:57.766 INFO:tasks.workunit.client.1.smithi044.stdout:6/819: write d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/f121 [6008,65183] 0 2024-06-03T01:20:57.767 INFO:tasks.workunit.client.1.smithi044.stdout:5/578: symlink d2/d11/d1e/d21/lb9 0 2024-06-03T01:20:57.773 INFO:tasks.workunit.client.1.smithi044.stdout:9/494: write d6/dd/f36 [3012880,15285] 0 2024-06-03T01:20:57.773 INFO:tasks.workunit.client.1.smithi044.stdout:0/762: mknod d3/d9/d55/d5a/dde/c11a 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:9/495: dread - d6/dd/d51/d46/f5c zero size 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:0/763: dread - d3/d9/d36/fd7 zero size 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:0/764: dread - d3/da6/dc1/dc2/de4/ff4 zero size 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:0/765: write d3/da6/d56/f5d [2458126,103017] 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:8/683: truncate d5/d16/d1d/fdf 1070365 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:8/684: write d5/d55/d56/d5c/dbb/f57 [81367,76161] 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:8/685: fdatasync d5/f25 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:4/689: link d17/d1c/d75/d7a/db8/d35/f41 d17/d1c/d75/d7a/db8/d59/fec 0 2024-06-03T01:20:57.774 INFO:tasks.workunit.client.1.smithi044.stdout:4/690: fdatasync d17/d1c/d75/f48 0 2024-06-03T01:20:57.775 INFO:tasks.workunit.client.1.smithi044.stdout:7/653: sync 2024-06-03T01:20:57.775 INFO:tasks.workunit.client.1.smithi044.stdout:2/493: sync 2024-06-03T01:20:57.777 INFO:tasks.workunit.client.1.smithi044.stdout:5/579: rmdir d2/d11/d1e/d21/d24/d9e 39 2024-06-03T01:20:57.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/766: creat d3/d6b/de8/d115/f11b x:0 0 0 2024-06-03T01:20:57.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/767: chown d3/f85 888145960 1 2024-06-03T01:20:57.782 INFO:tasks.workunit.client.1.smithi044.stdout:9/496: dread d6/dd/d51/f86 [0,4194304] 0 2024-06-03T01:20:57.783 INFO:tasks.workunit.client.1.smithi044.stdout:6/820: dwrite d2/d23/d55/d5f/f63 [0,4194304] 0 2024-06-03T01:20:57.783 INFO:tasks.workunit.client.1.smithi044.stdout:8/686: rename d5/d20/d2b/lc8 to d5/d20/d2b/d98/le0 0 2024-06-03T01:20:57.785 INFO:tasks.workunit.client.1.smithi044.stdout:4/691: mkdir d17/d1c/d75/d7a/db8/d8d/ded 0 2024-06-03T01:20:57.785 INFO:tasks.workunit.client.1.smithi044.stdout:4/692: chown d17/d1c/d75/d7a/db8/d6a/f73 421014 1 2024-06-03T01:20:57.788 INFO:tasks.workunit.client.1.smithi044.stdout:5/580: link d2/d11/d12/d2b/d44/c41 d2/d11/d12/d2b/d4a/d57/cba 0 2024-06-03T01:20:57.789 INFO:tasks.workunit.client.1.smithi044.stdout:0/768: symlink d3/d6b/d95/dae/dd1/l11c 0 2024-06-03T01:20:57.793 INFO:tasks.workunit.client.1.smithi044.stdout:7/654: dwrite d0/d3/d8/d32/d33/f42 [0,4194304] 0 2024-06-03T01:20:57.795 INFO:tasks.workunit.client.1.smithi044.stdout:9/497: rename d6/d26 to d6/dd/d51/d16/d27/d5e/da0 0 2024-06-03T01:20:57.795 INFO:tasks.workunit.client.1.smithi044.stdout:9/498: truncate d6/dd/d51/d16/f1b 1209838 0 2024-06-03T01:20:57.796 INFO:tasks.workunit.client.1.smithi044.stdout:8/687: creat d5/d20/d2b/d98/ddd/fe1 x:0 0 0 2024-06-03T01:20:57.796 INFO:tasks.workunit.client.1.smithi044.stdout:8/688: write d5/d20/d2b/d98/fa7 [1178322,113626] 0 2024-06-03T01:20:57.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/693: creat d17/d1c/d75/d7a/db8/d8d/fee x:0 0 0 2024-06-03T01:20:57.798 INFO:tasks.workunit.client.1.smithi044.stdout:5/581: mkdir d2/d11/d1e/d21/d24/d9e/dbb 0 2024-06-03T01:20:57.801 INFO:tasks.workunit.client.1.smithi044.stdout:0/769: unlink d3/da6/d56/f5d 0 2024-06-03T01:20:57.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/821: rmdir d2/d23/d30 39 2024-06-03T01:20:57.809 INFO:tasks.workunit.client.1.smithi044.stdout:7/655: rename d0/d3/fca to d0/d3/fcf 0 2024-06-03T01:20:57.811 INFO:tasks.workunit.client.1.smithi044.stdout:9/499: mknod d6/dd/d51/d16/d61/ca1 0 2024-06-03T01:20:57.811 INFO:tasks.workunit.client.1.smithi044.stdout:9/500: write d6/dd/d51/d16/d27/f39 [768136,51190] 0 2024-06-03T01:20:57.812 INFO:tasks.workunit.client.1.smithi044.stdout:8/689: mkdir d5/d55/d56/d5c/dbb/de2 0 2024-06-03T01:20:57.817 INFO:tasks.workunit.client.1.smithi044.stdout:4/694: dwrite d17/d1c/d75/d7a/db8/d28/f9d [0,4194304] 0 2024-06-03T01:20:57.817 INFO:tasks.workunit.client.1.smithi044.stdout:0/770: creat d3/da6/d56/d62/de7/df3/f11d x:0 0 0 2024-06-03T01:20:57.817 INFO:tasks.workunit.client.1.smithi044.stdout:6/822: rmdir d2/d23/d30/d1e/d38/d6c/dc2 39 2024-06-03T01:20:57.818 INFO:tasks.workunit.client.1.smithi044.stdout:9/501: symlink d6/d38/la2 0 2024-06-03T01:20:57.818 INFO:tasks.workunit.client.1.smithi044.stdout:9/502: readlink d6/dd/d51/d16/d27/d5e/da0/l9a 0 2024-06-03T01:20:57.819 INFO:tasks.workunit.client.1.smithi044.stdout:6/823: dread d2/d23/d30/f7f [0,4194304] 0 2024-06-03T01:20:57.825 INFO:tasks.workunit.client.1.smithi044.stdout:4/695: creat d17/d1c/d75/d7a/db8/d6a/d84/fef x:0 0 0 2024-06-03T01:20:57.825 INFO:tasks.workunit.client.1.smithi044.stdout:4/696: read d17/f7c [208374,77332] 0 2024-06-03T01:20:57.825 INFO:tasks.workunit.client.1.smithi044.stdout:0/771: rmdir d3/da6/d20 39 2024-06-03T01:20:57.825 INFO:tasks.workunit.client.1.smithi044.stdout:0/772: dread - d3/da6/d21/fc4 zero size 2024-06-03T01:20:57.825 INFO:tasks.workunit.client.1.smithi044.stdout:9/503: getdents d6/dd/d51/d3b 0 2024-06-03T01:20:57.826 INFO:tasks.workunit.client.1.smithi044.stdout:9/504: fsync d6/dd/d51/f86 0 2024-06-03T01:20:57.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/824: creat d2/d23/d30/d1e/d38/f122 x:0 0 0 2024-06-03T01:20:57.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/825: readlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/le1 0 2024-06-03T01:20:57.827 INFO:tasks.workunit.client.1.smithi044.stdout:9/505: write d6/f65 [16437,29090] 0 2024-06-03T01:20:57.827 INFO:tasks.workunit.client.1.smithi044.stdout:9/506: creat d6/dd/fa3 x:0 0 0 2024-06-03T01:20:57.828 INFO:tasks.workunit.client.1.smithi044.stdout:9/507: fsync d6/dd/d51/f13 0 2024-06-03T01:20:57.828 INFO:tasks.workunit.client.1.smithi044.stdout:9/508: fdatasync d6/dd/f3d 0 2024-06-03T01:20:57.828 INFO:tasks.workunit.client.1.smithi044.stdout:9/509: fdatasync d6/f12 0 2024-06-03T01:20:57.828 INFO:tasks.workunit.client.1.smithi044.stdout:7/656: dwrite d0/d3/d8/d95/d63/d7f/d9c/f50 [0,4194304] 0 2024-06-03T01:20:57.834 INFO:tasks.workunit.client.1.smithi044.stdout:8/690: rename d5/d20/d2b to d5/d16/d81/de3 0 2024-06-03T01:20:57.834 INFO:tasks.workunit.client.1.smithi044.stdout:8/691: chown d5/d55/d56/d5c/d68/laf 256734899 1 2024-06-03T01:20:57.837 INFO:tasks.workunit.client.1.smithi044.stdout:4/697: link d17/d1c/d75/d7a/db8/d28/d2b/l37 d17/d1c/d75/d7a/lf0 0 2024-06-03T01:20:57.842 INFO:tasks.workunit.client.1.smithi044.stdout:6/826: creat d2/d23/d55/f123 x:0 0 0 2024-06-03T01:20:57.843 INFO:tasks.workunit.client.1.smithi044.stdout:9/510: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:57.847 INFO:tasks.workunit.client.1.smithi044.stdout:7/657: rmdir d0/d3/d8/d95/d63/d7f/d9c/da4 39 2024-06-03T01:20:57.849 INFO:tasks.workunit.client.1.smithi044.stdout:8/692: creat d5/fe4 x:0 0 0 2024-06-03T01:20:57.849 INFO:tasks.workunit.client.1.smithi044.stdout:6/827: creat d2/d23/d44/d4d/d11d/f124 x:0 0 0 2024-06-03T01:20:57.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/511: creat d6/dd/d51/d16/d4d/d55/d68/fa4 x:0 0 0 2024-06-03T01:20:57.860 INFO:tasks.workunit.client.1.smithi044.stdout:8/693: write d5/f6a [698576,3147] 0 2024-06-03T01:20:57.861 INFO:tasks.workunit.client.1.smithi044.stdout:7/658: link d0/d3/d8/da0/fbe d0/d3/da8/fd0 0 2024-06-03T01:20:57.863 INFO:tasks.workunit.client.1.smithi044.stdout:9/512: mknod d6/dd/d51/d16/ca5 0 2024-06-03T01:20:57.863 INFO:tasks.workunit.client.1.smithi044.stdout:9/513: fdatasync d6/dd/f36 0 2024-06-03T01:20:57.864 INFO:tasks.workunit.client.1.smithi044.stdout:9/514: write d6/dd/d51/d16/d4d/d55/d68/fa4 [652664,124216] 0 2024-06-03T01:20:57.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/828: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fe3 [0,4194304] 0 2024-06-03T01:20:57.875 INFO:tasks.workunit.client.1.smithi044.stdout:7/659: write d0/d16/fb8 [2327709,120881] 0 2024-06-03T01:20:57.876 INFO:tasks.workunit.client.1.smithi044.stdout:7/660: fdatasync d0/d3/d8/d32/f38 0 2024-06-03T01:20:57.876 INFO:tasks.workunit.client.1.smithi044.stdout:9/515: write d6/dd/d51/fe [4074576,59353] 0 2024-06-03T01:20:57.878 INFO:tasks.workunit.client.1.smithi044.stdout:6/829: creat d2/d23/d55/d5f/db2/d3f/d106/d116/da4/f125 x:0 0 0 2024-06-03T01:20:57.879 INFO:tasks.workunit.client.1.smithi044.stdout:7/661: mkdir d0/d3/d8/d32/d33/dd1 0 2024-06-03T01:20:57.879 INFO:tasks.workunit.client.1.smithi044.stdout:6/830: creat d2/d23/d55/d5f/d85/f126 x:0 0 0 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/144: sync 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:9/266: sync 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:9/267: chown d0/d23/d25 5684850 1 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:1/107: sync 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/145: creat dc/d12/d1b/f34 x:0 0 0 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/146: chown dc/d12/d1b/f34 492799963 1 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/147: write dc/d12/d22/f2e [1019476,2019] 0 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/148: fsync dc/f16 0 2024-06-03T01:20:57.881 INFO:tasks.workunit.client.0.smithi032.stdout:6/162: sync 2024-06-03T01:20:57.882 INFO:tasks.workunit.client.0.smithi032.stdout:0/213: sync 2024-06-03T01:20:57.882 INFO:tasks.workunit.client.0.smithi032.stdout:5/212: sync 2024-06-03T01:20:57.882 INFO:tasks.workunit.client.0.smithi032.stdout:2/192: sync 2024-06-03T01:20:57.882 INFO:tasks.workunit.client.1.smithi044.stdout:9/516: dread d6/dd/d51/d16/d4d/d55/d68/fa4 [0,4194304] 0 2024-06-03T01:20:57.882 INFO:tasks.workunit.client.0.smithi032.stdout:8/128: sync 2024-06-03T01:20:57.885 INFO:tasks.workunit.client.0.smithi032.stdout:9/268: symlink d0/d23/d24/d26/d41/l58 0 2024-06-03T01:20:57.887 INFO:tasks.workunit.client.0.smithi032.stdout:5/213: write da/f3e [535234,128049] 0 2024-06-03T01:20:57.887 INFO:tasks.workunit.client.0.smithi032.stdout:5/214: read da/dd/df/d10/d16/d18/d35/f3a [14747,90046] 0 2024-06-03T01:20:57.888 INFO:tasks.workunit.client.0.smithi032.stdout:7/149: rename dc/d12/d1b/l28 to dc/d12/d1b/l35 0 2024-06-03T01:20:57.888 INFO:tasks.workunit.client.0.smithi032.stdout:7/150: truncate fa 1701635 0 2024-06-03T01:20:57.890 INFO:tasks.workunit.client.0.smithi032.stdout:8/129: dread dc/de/f11 [0,4194304] 0 2024-06-03T01:20:57.890 INFO:tasks.workunit.client.0.smithi032.stdout:6/163: unlink c5 0 2024-06-03T01:20:57.890 INFO:tasks.workunit.client.0.smithi032.stdout:6/164: creat d8/d11/f35 x:0 0 0 2024-06-03T01:20:57.891 INFO:tasks.workunit.client.0.smithi032.stdout:6/165: readlink d8/d11/d20/d31/l33 0 2024-06-03T01:20:57.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/269: rename d0/d23/c4e to d0/d23/d24/c59 0 2024-06-03T01:20:57.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/270: fdatasync d0/fa 0 2024-06-03T01:20:57.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/271: fsync d0/ff 0 2024-06-03T01:20:57.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/272: read - d0/d23/d25/f37 zero size 2024-06-03T01:20:57.893 INFO:tasks.workunit.client.0.smithi032.stdout:7/151: truncate f6 1822765 0 2024-06-03T01:20:57.893 INFO:tasks.workunit.client.0.smithi032.stdout:7/152: chown dc/d12/f13 13417 1 2024-06-03T01:20:57.894 INFO:tasks.workunit.client.0.smithi032.stdout:8/130: mknod dc/de/d1e/c26 0 2024-06-03T01:20:57.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/273: dread d0/d11/d1d/f1e [0,4194304] 0 2024-06-03T01:20:57.895 INFO:tasks.workunit.client.0.smithi032.stdout:9/274: chown d0/d23/c40 478 1 2024-06-03T01:20:57.895 INFO:tasks.workunit.client.0.smithi032.stdout:7/153: write dc/f15 [372178,24211] 0 2024-06-03T01:20:57.895 INFO:tasks.workunit.client.0.smithi032.stdout:7/154: read dc/f30 [1116710,120581] 0 2024-06-03T01:20:57.895 INFO:tasks.workunit.client.1.smithi044.stdout:9/517: dread d6/dd/f17 [0,4194304] 0 2024-06-03T01:20:57.896 INFO:tasks.workunit.client.1.smithi044.stdout:9/518: chown d6/dd/d51/c3c 0 1 2024-06-03T01:20:57.896 INFO:tasks.workunit.client.0.smithi032.stdout:8/131: symlink dc/d14/l27 0 2024-06-03T01:20:57.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/519: truncate d6/dd/f36 1036652 0 2024-06-03T01:20:57.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/520: write d6/dd/f87 [82894,120842] 0 2024-06-03T01:20:57.899 INFO:tasks.workunit.client.0.smithi032.stdout:9/275: write d0/d23/d25/f4c [1524800,106969] 0 2024-06-03T01:20:57.899 INFO:tasks.workunit.client.0.smithi032.stdout:9/276: chown d0/d23/l2d 18278629 1 2024-06-03T01:20:57.899 INFO:tasks.workunit.client.0.smithi032.stdout:7/155: getdents dc/d12 0 2024-06-03T01:20:57.899 INFO:tasks.workunit.client.0.smithi032.stdout:7/156: chown f6 1531593070 1 2024-06-03T01:20:57.902 INFO:tasks.workunit.client.0.smithi032.stdout:9/277: symlink d0/d11/d1d/d38/l5a 0 2024-06-03T01:20:57.902 INFO:tasks.workunit.client.0.smithi032.stdout:9/278: fsync d0/fe 0 2024-06-03T01:20:57.903 INFO:tasks.workunit.client.0.smithi032.stdout:5/215: write da/f3e [1659634,114905] 0 2024-06-03T01:20:57.905 INFO:tasks.workunit.client.0.smithi032.stdout:5/216: dread da/dd/df/f32 [0,4194304] 0 2024-06-03T01:20:57.906 INFO:tasks.workunit.client.0.smithi032.stdout:5/217: dread da/dd/df/d10/d16/d18/d35/f3a [0,4194304] 0 2024-06-03T01:20:57.907 INFO:tasks.workunit.client.1.smithi044.stdout:3/544: sync 2024-06-03T01:20:57.910 INFO:tasks.workunit.client.1.smithi044.stdout:3/545: unlink df/d13/d1b/f64 0 2024-06-03T01:20:57.911 INFO:tasks.workunit.client.0.smithi032.stdout:6/166: read d8/ff [2725910,129281] 0 2024-06-03T01:20:57.915 INFO:tasks.workunit.client.0.smithi032.stdout:6/167: mkdir d8/d11/d20/d36 0 2024-06-03T01:20:57.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/546: rmdir df/d1e/d2a/d70 39 2024-06-03T01:20:57.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/547: write df/f10 [634613,47483] 0 2024-06-03T01:20:57.917 INFO:tasks.workunit.client.1.smithi044.stdout:9/521: dread d6/dd/d51/db/f22 [0,4194304] 0 2024-06-03T01:20:57.917 INFO:tasks.workunit.client.1.smithi044.stdout:9/522: creat d6/dd/d51/d46/fa6 x:0 0 0 2024-06-03T01:20:57.918 INFO:tasks.workunit.client.0.smithi032.stdout:6/168: creat d8/d11/d20/d31/f37 x:0 0 0 2024-06-03T01:20:57.918 INFO:tasks.workunit.client.0.smithi032.stdout:6/169: chown d8/fd 3 1 2024-06-03T01:20:57.920 INFO:tasks.workunit.client.0.smithi032.stdout:0/214: write d1/f9 [534491,98595] 0 2024-06-03T01:20:57.920 INFO:tasks.workunit.client.0.smithi032.stdout:0/215: dread - d1/da/d10/f39 zero size 2024-06-03T01:20:57.920 INFO:tasks.workunit.client.0.smithi032.stdout:0/216: read d1/f17 [1402922,5681] 0 2024-06-03T01:20:57.920 INFO:tasks.workunit.client.0.smithi032.stdout:6/170: creat d8/d11/d20/d36/f38 x:0 0 0 2024-06-03T01:20:57.920 INFO:tasks.workunit.client.0.smithi032.stdout:6/171: chown d8/d11/d20 5955 1 2024-06-03T01:20:57.921 INFO:tasks.workunit.client.1.smithi044.stdout:3/548: rename df/d13/c14 to df/d13/d39/d3d/cb4 0 2024-06-03T01:20:57.921 INFO:tasks.workunit.client.0.smithi032.stdout:0/217: symlink d1/da/d10/d29/d31/l41 0 2024-06-03T01:20:57.923 INFO:tasks.workunit.client.0.smithi032.stdout:1/108: dwrite d18/f1c [0,4194304] 0 2024-06-03T01:20:57.923 INFO:tasks.workunit.client.0.smithi032.stdout:9/279: write d0/ff [2674307,72950] 0 2024-06-03T01:20:57.923 INFO:tasks.workunit.client.0.smithi032.stdout:0/218: symlink d1/da/l42 0 2024-06-03T01:20:57.924 INFO:tasks.workunit.client.1.smithi044.stdout:9/523: rename d6/dd/d51/d16/ca5 to d6/dd/d51/d16/d5f/d8b/ca7 0 2024-06-03T01:20:57.926 INFO:tasks.workunit.client.1.smithi044.stdout:9/524: creat d6/dd/d51/d16/d5f/fa8 x:0 0 0 2024-06-03T01:20:57.927 INFO:tasks.workunit.client.1.smithi044.stdout:3/549: symlink df/d13/d1b/lb5 0 2024-06-03T01:20:57.927 INFO:tasks.workunit.client.1.smithi044.stdout:3/550: creat df/d1e/d3a/fb6 x:0 0 0 2024-06-03T01:20:57.930 INFO:tasks.workunit.client.0.smithi032.stdout:6/172: write f4 [1532622,128645] 0 2024-06-03T01:20:57.932 INFO:tasks.workunit.client.1.smithi044.stdout:9/525: mknod d6/dd/d51/d16/d27/ca9 0 2024-06-03T01:20:57.932 INFO:tasks.workunit.client.1.smithi044.stdout:9/526: chown d6/dd/d51/d16/d27/f64 68 1 2024-06-03T01:20:57.935 INFO:tasks.workunit.client.0.smithi032.stdout:7/157: fsync dc/d12/d1b/f34 0 2024-06-03T01:20:57.938 INFO:tasks.workunit.client.1.smithi044.stdout:6/831: dwrite d2/d23/d30/f45 [0,4194304] 0 2024-06-03T01:20:57.938 INFO:tasks.workunit.client.1.smithi044.stdout:6/832: readlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/lcf 0 2024-06-03T01:20:57.938 INFO:tasks.workunit.client.1.smithi044.stdout:6/833: truncate d2/d23/d55/f123 1012601 0 2024-06-03T01:20:57.938 INFO:tasks.workunit.client.1.smithi044.stdout:3/551: rename df/d13/d39/d3d/d43/d71/d7c to df/d13/d6d/d9a/db7 0 2024-06-03T01:20:57.939 INFO:tasks.workunit.client.0.smithi032.stdout:2/193: dwrite d5/f23 [0,4194304] 0 2024-06-03T01:20:57.939 INFO:tasks.workunit.client.0.smithi032.stdout:2/194: readlink d5/l7 0 2024-06-03T01:20:57.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/218: dwrite da/f17 [4194304,4194304] 0 2024-06-03T01:20:57.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/219: read - da/dd/df/d10/d16/d18/f27 zero size 2024-06-03T01:20:57.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/132: dwrite f1 [4194304,4194304] 0 2024-06-03T01:20:57.943 INFO:tasks.workunit.client.1.smithi044.stdout:3/552: symlink df/d13/d39/d3d/d43/d71/lb8 0 2024-06-03T01:20:57.943 INFO:tasks.workunit.client.1.smithi044.stdout:3/553: chown df/c42 3 1 2024-06-03T01:20:57.943 INFO:tasks.workunit.client.1.smithi044.stdout:3/554: chown df/d1e/f37 59334 1 2024-06-03T01:20:57.944 INFO:tasks.workunit.client.0.smithi032.stdout:2/195: getdents d5/da/d1a/d2a 0 2024-06-03T01:20:57.944 INFO:tasks.workunit.client.0.smithi032.stdout:2/196: chown d5/l14 3 1 2024-06-03T01:20:57.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/109: dread fa [0,4194304] 0 2024-06-03T01:20:57.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/110: creat d18/f24 x:0 0 0 2024-06-03T01:20:57.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/111: truncate d18/f1c 4778470 0 2024-06-03T01:20:57.953 INFO:tasks.workunit.client.1.smithi044.stdout:9/527: dread d6/dd/f3d [0,4194304] 0 2024-06-03T01:20:57.954 INFO:tasks.workunit.client.0.smithi032.stdout:8/133: write dc/de/f11 [3169791,69275] 0 2024-06-03T01:20:57.955 INFO:tasks.workunit.client.1.smithi044.stdout:6/834: read d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/f9e [3495678,81041] 0 2024-06-03T01:20:57.956 INFO:tasks.workunit.client.1.smithi044.stdout:6/835: write d2/d23/d55/d5f/db2/d3f/fc5 [590060,50824] 0 2024-06-03T01:20:57.956 INFO:tasks.workunit.client.0.smithi032.stdout:8/134: mkdir dc/d25/d28 0 2024-06-03T01:20:57.958 INFO:tasks.workunit.client.0.smithi032.stdout:8/135: dread dc/f13 [0,4194304] 0 2024-06-03T01:20:57.961 INFO:tasks.workunit.client.0.smithi032.stdout:7/158: dread dc/d12/d1b/f1d [0,4194304] 0 2024-06-03T01:20:57.961 INFO:tasks.workunit.client.0.smithi032.stdout:7/159: chown dc/d12/l31 423276 1 2024-06-03T01:20:57.961 INFO:tasks.workunit.client.0.smithi032.stdout:7/160: dread - dc/d12/d22/f29 zero size 2024-06-03T01:20:57.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/219: dwrite d1/da/d28/f33 [0,4194304] 0 2024-06-03T01:20:57.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/220: dread - d1/da/d10/f39 zero size 2024-06-03T01:20:57.963 INFO:tasks.workunit.client.0.smithi032.stdout:6/173: dwrite d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:20:57.964 INFO:tasks.workunit.client.0.smithi032.stdout:9/280: dwrite d0/d11/f22 [8388608,4194304] 0 2024-06-03T01:20:57.965 INFO:tasks.workunit.client.1.smithi044.stdout:9/528: read d6/dd/d51/d16/d27/f53 [833091,107930] 0 2024-06-03T01:20:57.965 INFO:tasks.workunit.client.1.smithi044.stdout:9/529: write d6/dd/d51/d16/d27/f53 [1681538,81793] 0 2024-06-03T01:20:57.968 INFO:tasks.workunit.client.1.smithi044.stdout:9/530: mknod d6/caa 0 2024-06-03T01:20:57.969 INFO:tasks.workunit.client.1.smithi044.stdout:9/531: readlink d6/dd/d51/d46/l83 0 2024-06-03T01:20:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: Deploying cephadm binary to smithi032 2024-06-03T01:20:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: Deploying cephadm binary to smithi044 2024-06-03T01:20:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: [03/Jun/2024:01:20:56] ENGINE Serving on https://172.21.15.44:7150 2024-06-03T01:20:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: [03/Jun/2024:01:20:56] ENGINE Bus STARTED 2024-06-03T01:20:57.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: mgrmap e22: smithi044.wszjlw(active, since 1.03919s) 2024-06-03T01:20:57.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:57 smithi044 ceph-mon[36956]: pgmap v3: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:20:57.974 INFO:tasks.workunit.client.0.smithi032.stdout:2/197: dwrite d5/da/f1e [0,4194304] 0 2024-06-03T01:20:57.975 INFO:tasks.workunit.client.1.smithi044.stdout:9/532: rename f2 to d6/dd/d51/d3b/fab 0 2024-06-03T01:20:57.975 INFO:tasks.workunit.client.1.smithi044.stdout:9/533: readlink d6/dd/d51/d16/d27/l59 0 2024-06-03T01:20:57.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/198: dread d5/f25 [0,4194304] 0 2024-06-03T01:20:57.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/199: chown d5/l24 3 1 2024-06-03T01:20:57.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/534: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac 0 2024-06-03T01:20:57.980 INFO:tasks.workunit.client.0.smithi032.stdout:7/161: dwrite dc/dd/f20 [0,4194304] 0 2024-06-03T01:20:57.980 INFO:tasks.workunit.client.0.smithi032.stdout:7/162: read dc/d12/f13 [52590,1645] 0 2024-06-03T01:20:57.986 INFO:tasks.workunit.client.1.smithi044.stdout:1/740: sync 2024-06-03T01:20:57.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/221: truncate d1/da/f16 2133756 0 2024-06-03T01:20:57.988 INFO:tasks.workunit.client.1.smithi044.stdout:1/741: mkdir d0/d3/d20/d7/d102 0 2024-06-03T01:20:57.989 INFO:tasks.workunit.client.1.smithi044.stdout:1/742: creat d0/dd2/dfe/f103 x:0 0 0 2024-06-03T01:20:57.990 INFO:tasks.workunit.client.1.smithi044.stdout:1/743: fdatasync d0/d3/d20/d7/d13/f24 0 2024-06-03T01:20:57.990 INFO:tasks.workunit.client.1.smithi044.stdout:1/744: read d0/d3/d14/fe5 [83235,30342] 0 2024-06-03T01:20:57.991 INFO:tasks.workunit.client.1.smithi044.stdout:1/745: rmdir d0/d3/d20/d40/d45 39 2024-06-03T01:20:57.993 INFO:tasks.workunit.client.0.smithi032.stdout:6/174: rmdir d8/d25 39 2024-06-03T01:20:57.993 INFO:tasks.workunit.client.1.smithi044.stdout:1/746: link d0/d3/d20/d40/d45/lac d0/d3/d20/d7/d13/d2d/l104 0 2024-06-03T01:20:57.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/281: mknod d0/d23/d24/c5b 0 2024-06-03T01:20:57.997 INFO:tasks.workunit.client.1.smithi044.stdout:9/535: getdents d6/dd/d51/d3b 0 2024-06-03T01:20:57.997 INFO:tasks.workunit.client.1.smithi044.stdout:9/536: fdatasync d6/dd/d51/d16/f3f 0 2024-06-03T01:20:58.001 INFO:tasks.workunit.client.0.smithi032.stdout:8/136: getdents dc/d14 0 2024-06-03T01:20:58.003 INFO:tasks.workunit.client.0.smithi032.stdout:2/200: getdents d5/d38 0 2024-06-03T01:20:58.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/175: write f3 [2534379,23178] 0 2024-06-03T01:20:58.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/176: read f4 [441496,65769] 0 2024-06-03T01:20:58.005 INFO:tasks.workunit.client.0.smithi032.stdout:6/177: write d8/fd [1133567,91803] 0 2024-06-03T01:20:58.005 INFO:tasks.workunit.client.0.smithi032.stdout:6/178: readlink d8/l10 0 2024-06-03T01:20:58.008 INFO:tasks.workunit.client.0.smithi032.stdout:7/163: link dc/f30 dc/d12/d1b/f36 0 2024-06-03T01:20:58.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/222: creat d1/f43 x:0 0 0 2024-06-03T01:20:58.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/223: fdatasync d1/da/f1a 0 2024-06-03T01:20:58.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/282: mknod d0/d23/d24/d26/d41/c5c 0 2024-06-03T01:20:58.014 INFO:tasks.workunit.client.1.smithi044.stdout:9/537: dread d6/dd/d51/d16/f37 [0,4194304] 0 2024-06-03T01:20:58.015 INFO:tasks.workunit.client.1.smithi044.stdout:9/538: getdents d6/dd/d51/d16/d61 0 2024-06-03T01:20:58.015 INFO:tasks.workunit.client.0.smithi032.stdout:7/164: dread dc/f14 [0,4194304] 0 2024-06-03T01:20:58.016 INFO:tasks.workunit.client.0.smithi032.stdout:8/137: symlink dc/d14/l29 0 2024-06-03T01:20:58.017 INFO:tasks.workunit.client.1.smithi044.stdout:9/539: mkdir d6/dd/d51/d16/d4d/dad 0 2024-06-03T01:20:58.017 INFO:tasks.workunit.client.1.smithi044.stdout:9/540: write d6/dd/d51/d16/d5f/d79/f97 [4376,100943] 0 2024-06-03T01:20:58.019 INFO:tasks.workunit.client.1.smithi044.stdout:1/747: dwrite d0/d3/d14/fe5 [0,4194304] 0 2024-06-03T01:20:58.019 INFO:tasks.workunit.client.1.smithi044.stdout:9/541: mkdir d6/dd/d51/d16/d4d/d55/dae 0 2024-06-03T01:20:58.020 INFO:tasks.workunit.client.0.smithi032.stdout:2/201: creat d5/f3c x:0 0 0 2024-06-03T01:20:58.022 INFO:tasks.workunit.client.0.smithi032.stdout:0/224: creat d1/da/d10/d29/f44 x:0 0 0 2024-06-03T01:20:58.022 INFO:tasks.workunit.client.0.smithi032.stdout:0/225: chown d1/l7 17358139 1 2024-06-03T01:20:58.022 INFO:tasks.workunit.client.0.smithi032.stdout:0/226: truncate d1/f21 1507372 0 2024-06-03T01:20:58.022 INFO:tasks.workunit.client.0.smithi032.stdout:0/227: chown d1/l38 3 1 2024-06-03T01:20:58.025 INFO:tasks.workunit.client.0.smithi032.stdout:7/165: chown c4 4 1 2024-06-03T01:20:58.026 INFO:tasks.workunit.client.0.smithi032.stdout:7/166: write dc/f15 [1201996,43805] 0 2024-06-03T01:20:58.026 INFO:tasks.workunit.client.0.smithi032.stdout:7/167: dread - dc/f2d zero size 2024-06-03T01:20:58.026 INFO:tasks.workunit.client.1.smithi044.stdout:1/748: rename d0/c7e to d0/dd2/dfe/dc2/c105 0 2024-06-03T01:20:58.026 INFO:tasks.workunit.client.1.smithi044.stdout:1/749: stat d0/d3/d20/d40/d45/d6f/d8c/cfb 0 2024-06-03T01:20:58.026 INFO:tasks.workunit.client.1.smithi044.stdout:1/750: chown d0/d3/d20/d40/d45/d4b/d5a/l97 124677952 1 2024-06-03T01:20:58.027 INFO:tasks.workunit.client.1.smithi044.stdout:9/542: link d6/dd/d51/d16/d4d/d55/d68/c9b d6/caf 0 2024-06-03T01:20:58.027 INFO:tasks.workunit.client.1.smithi044.stdout:9/543: truncate d6/dd/d51/d16/d5f/f80 417108 0 2024-06-03T01:20:58.028 INFO:tasks.workunit.client.0.smithi032.stdout:7/168: mknod dc/d12/d2a/c37 0 2024-06-03T01:20:58.028 INFO:tasks.workunit.client.1.smithi044.stdout:1/751: rename d0/d3/d20/d40/d45/d6b/c100 to d0/d3/d20/d40/d45/d6b/c106 0 2024-06-03T01:20:58.031 INFO:tasks.workunit.client.1.smithi044.stdout:1/752: symlink d0/d3/d20/d40/d45/d4b/l107 0 2024-06-03T01:20:58.031 INFO:tasks.workunit.client.1.smithi044.stdout:1/753: chown d0/d3/c43 8 1 2024-06-03T01:20:58.032 INFO:tasks.workunit.client.0.smithi032.stdout:0/228: dread d1/da/f16 [0,4194304] 0 2024-06-03T01:20:58.033 INFO:tasks.workunit.client.1.smithi044.stdout:9/544: getdents d6/dd/d51/d16/d4d/d55/d68 0 2024-06-03T01:20:58.033 INFO:tasks.workunit.client.1.smithi044.stdout:9/545: creat d6/dd/d51/d46/fb0 x:0 0 0 2024-06-03T01:20:58.033 INFO:tasks.workunit.client.1.smithi044.stdout:9/546: write d6/f84 [432548,24187] 0 2024-06-03T01:20:58.036 INFO:tasks.workunit.client.1.smithi044.stdout:9/547: creat d6/dd/d51/d16/d4d/d67/fb1 x:0 0 0 2024-06-03T01:20:58.042 INFO:tasks.workunit.client.0.smithi032.stdout:6/179: dwrite d8/d25/d19/f1d [0,4194304] 0 2024-06-03T01:20:58.042 INFO:tasks.workunit.client.0.smithi032.stdout:6/180: truncate d8/d25/d19/f1d 8792398 0 2024-06-03T01:20:58.043 INFO:tasks.workunit.client.0.smithi032.stdout:9/283: dwrite d0/d11/f48 [0,4194304] 0 2024-06-03T01:20:58.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/181: rename d8/d25 to d8/d11/d20/d2b/d39 0 2024-06-03T01:20:58.050 INFO:tasks.workunit.client.0.smithi032.stdout:9/284: creat d0/d11/d1d/d4a/f5d x:0 0 0 2024-06-03T01:20:58.051 INFO:tasks.workunit.client.0.smithi032.stdout:9/285: mknod d0/d23/d24/d26/d41/c5e 0 2024-06-03T01:20:58.052 INFO:tasks.workunit.client.0.smithi032.stdout:7/169: write f6 [333309,95048] 0 2024-06-03T01:20:58.053 INFO:tasks.workunit.client.1.smithi044.stdout:2/494: sync 2024-06-03T01:20:58.069 INFO:tasks.workunit.client.1.smithi044.stdout:2/495: dread db/d15/f70 [0,4194304] 0 2024-06-03T01:20:58.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/170: read dc/dd/f20 [2211186,96923] 0 2024-06-03T01:20:58.077 INFO:tasks.workunit.client.0.smithi032.stdout:7/171: creat dc/d12/d24/f38 x:0 0 0 2024-06-03T01:20:58.077 INFO:tasks.workunit.client.0.smithi032.stdout:7/172: write dc/d12/d22/f29 [108336,18922] 0 2024-06-03T01:20:58.078 INFO:tasks.workunit.client.0.smithi032.stdout:7/173: chown dc/dd/fe 1913914389 1 2024-06-03T01:20:58.079 INFO:tasks.workunit.client.0.smithi032.stdout:7/174: creat dc/d12/d2a/d2f/f39 x:0 0 0 2024-06-03T01:20:58.081 INFO:tasks.workunit.client.0.smithi032.stdout:7/175: stat dc/l27 0 2024-06-03T01:20:58.082 INFO:tasks.workunit.client.0.smithi032.stdout:7/176: mknod dc/d12/d24/c3a 0 2024-06-03T01:20:58.082 INFO:tasks.workunit.client.0.smithi032.stdout:7/177: chown dc 363470 1 2024-06-03T01:20:58.083 INFO:tasks.workunit.client.0.smithi032.stdout:7/178: write dc/dd/fe [121218,39398] 0 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: Deploying cephadm binary to smithi032 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: Deploying cephadm binary to smithi044 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: [03/Jun/2024:01:20:56] ENGINE Serving on https://172.21.15.44:7150 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: [03/Jun/2024:01:20:56] ENGINE Bus STARTED 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: mgrmap e22: smithi044.wszjlw(active, since 1.03919s) 2024-06-03T01:20:58.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:57 smithi032 ceph-mon[29360]: pgmap v3: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:20:58.088 INFO:tasks.workunit.client.0.smithi032.stdout:7/179: mknod dc/d12/c3b 0 2024-06-03T01:20:58.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/754: dwrite d0/d3/d14/d81/fd5 [0,4194304] 0 2024-06-03T01:20:58.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/755: stat d0/d3/d20/d7/d13/f84 0 2024-06-03T01:20:58.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/756: read - d0/dd2/dfe/fdc zero size 2024-06-03T01:20:58.091 INFO:tasks.workunit.client.1.smithi044.stdout:2/496: dwrite db/f26 [0,4194304] 0 2024-06-03T01:20:58.091 INFO:tasks.workunit.client.1.smithi044.stdout:2/497: chown db/d1b/d2a/d2f 32853063 1 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/229: dwrite d1/da/f1a [0,4194304] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/230: chown d1/f9 3 1 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/231: creat d1/da/d10/d29/f45 x:0 0 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/138: dwrite dc/d14/f24 [0,4194304] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:9/286: dwrite d0/f5 [0,4194304] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:9/287: write d0/d11/d1d/f1e [40645,110030] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:2/202: dwrite d5/f13 [0,4194304] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:6/182: dwrite d8/d11/f35 [0,4194304] 0 2024-06-03T01:20:58.092 INFO:tasks.workunit.client.0.smithi032.stdout:6/183: truncate d8/d11/d20/d21/f27 129744 0 2024-06-03T01:20:58.093 INFO:tasks.workunit.client.0.smithi032.stdout:6/184: write f4 [2964801,10960] 0 2024-06-03T01:20:58.093 INFO:tasks.workunit.client.0.smithi032.stdout:0/232: dread d1/f2b [0,4194304] 0 2024-06-03T01:20:58.093 INFO:tasks.workunit.client.0.smithi032.stdout:0/233: fsync d1/da/f1a 0 2024-06-03T01:20:58.093 INFO:tasks.workunit.client.0.smithi032.stdout:7/180: mknod dc/d12/d24/c3c 0 2024-06-03T01:20:58.094 INFO:tasks.workunit.client.0.smithi032.stdout:7/181: write dc/f15 [778687,116740] 0 2024-06-03T01:20:58.094 INFO:tasks.workunit.client.0.smithi032.stdout:7/182: creat dc/f3d x:0 0 0 2024-06-03T01:20:58.094 INFO:tasks.workunit.client.0.smithi032.stdout:7/183: stat dc/d12/d24/d33 0 2024-06-03T01:20:58.094 INFO:tasks.workunit.client.0.smithi032.stdout:7/184: fdatasync dc/f14 0 2024-06-03T01:20:58.094 INFO:tasks.workunit.client.0.smithi032.stdout:6/185: fdatasync f4 0 2024-06-03T01:20:58.097 INFO:tasks.workunit.client.1.smithi044.stdout:1/757: unlink d0/d3/d20/d40/d45/d6f/lb8 0 2024-06-03T01:20:58.097 INFO:tasks.workunit.client.1.smithi044.stdout:1/758: dread - d0/d3/d20/f86 zero size 2024-06-03T01:20:58.102 INFO:tasks.workunit.client.1.smithi044.stdout:9/548: getdents d6/dd/d51/d46 0 2024-06-03T01:20:58.105 INFO:tasks.workunit.client.1.smithi044.stdout:2/498: rmdir db/d15/d49/d6c/d72 39 2024-06-03T01:20:58.105 INFO:tasks.workunit.client.1.smithi044.stdout:2/499: stat db/d15/d49/d6c/d72/d7a/f81 0 2024-06-03T01:20:58.105 INFO:tasks.workunit.client.1.smithi044.stdout:2/500: readlink db/d15/l44 0 2024-06-03T01:20:58.109 INFO:tasks.workunit.client.1.smithi044.stdout:9/549: dread d6/dd/d51/fe [0,4194304] 0 2024-06-03T01:20:58.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/550: dread d6/f65 [0,4194304] 0 2024-06-03T01:20:58.112 INFO:tasks.workunit.client.1.smithi044.stdout:2/501: dread db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:20:58.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/551: dwrite d6/dd/d51/d16/d61/f6c [0,4194304] 0 2024-06-03T01:20:58.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/552: fdatasync d6/dd/f36 0 2024-06-03T01:20:58.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/553: truncate d6/dd/f87 690635 0 2024-06-03T01:20:58.136 INFO:tasks.workunit.client.1.smithi044.stdout:9/554: creat d6/dd/d51/d16/fb2 x:0 0 0 2024-06-03T01:20:58.137 INFO:tasks.workunit.client.1.smithi044.stdout:9/555: truncate d6/dd/f87 1510347 0 2024-06-03T01:20:58.138 INFO:tasks.workunit.client.1.smithi044.stdout:9/556: rmdir d6/dd/d51/d16/d61 39 2024-06-03T01:20:58.139 INFO:tasks.workunit.client.1.smithi044.stdout:2/502: dwrite db/f1f [0,4194304] 0 2024-06-03T01:20:58.149 INFO:tasks.workunit.client.0.smithi032.stdout:9/288: unlink d0/d11/d1d/f56 0 2024-06-03T01:20:58.149 INFO:tasks.workunit.client.0.smithi032.stdout:9/289: dread - d0/d23/d25/f37 zero size 2024-06-03T01:20:58.152 INFO:tasks.workunit.client.0.smithi032.stdout:2/203: creat d5/da/d2e/f3d x:0 0 0 2024-06-03T01:20:58.154 INFO:tasks.workunit.client.0.smithi032.stdout:3/195: sync 2024-06-03T01:20:58.154 INFO:tasks.workunit.client.0.smithi032.stdout:4/163: sync 2024-06-03T01:20:58.154 INFO:tasks.workunit.client.0.smithi032.stdout:3/196: stat de/d29/d2e/d31/f21 0 2024-06-03T01:20:58.154 INFO:tasks.workunit.client.0.smithi032.stdout:4/164: getdents d4/db/da/d37 0 2024-06-03T01:20:58.155 INFO:tasks.workunit.client.0.smithi032.stdout:4/165: chown d4/db/le 98 1 2024-06-03T01:20:58.156 INFO:tasks.workunit.client.0.smithi032.stdout:7/185: mknod dc/d12/d22/c3e 0 2024-06-03T01:20:58.156 INFO:tasks.workunit.client.0.smithi032.stdout:7/186: write dc/f16 [2569295,29483] 0 2024-06-03T01:20:58.156 INFO:tasks.workunit.client.0.smithi032.stdout:7/187: truncate dc/d12/d1b/f34 62547 0 2024-06-03T01:20:58.157 INFO:tasks.workunit.client.0.smithi032.stdout:7/188: chown dc/d12/d2a/d2f 5846 1 2024-06-03T01:20:58.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/186: truncate d8/d11/d20/d2b/d39/f1f 420493 0 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:0/773: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:3/555: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:7/662: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:5/582: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:4/698: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:0/774: fsync d3/da6/d56/f60 0 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:7/663: chown d0/c13 241151154 1 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:5/583: truncate d2/d11/d1e/d21/d24/f38 5320826 0 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:5/584: truncate d2/d11/d1e/d21/d24/f69 611866 0 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:8/694: sync 2024-06-03T01:20:58.164 INFO:tasks.workunit.client.1.smithi044.stdout:6/836: sync 2024-06-03T01:20:58.165 INFO:tasks.workunit.client.1.smithi044.stdout:6/837: dread - d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/f101 zero size 2024-06-03T01:20:58.165 INFO:tasks.workunit.client.0.smithi032.stdout:3/197: dread f8 [0,4194304] 0 2024-06-03T01:20:58.165 INFO:tasks.workunit.client.0.smithi032.stdout:3/198: stat de/d29/d2e/d31/l2a 0 2024-06-03T01:20:58.166 INFO:tasks.workunit.client.0.smithi032.stdout:8/139: rename l4 to dc/de/l2a 0 2024-06-03T01:20:58.166 INFO:tasks.workunit.client.1.smithi044.stdout:9/557: dwrite d6/dd/d51/d16/d5f/f80 [0,4194304] 0 2024-06-03T01:20:58.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/503: dwrite db/d15/d49/d6c/d72/d7a/f8d [0,4194304] 0 2024-06-03T01:20:58.167 INFO:tasks.workunit.client.0.smithi032.stdout:9/290: link d0/d23/d24/d26/l36 d0/d11/l5f 0 2024-06-03T01:20:58.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/556: chown df/d13/d1b/f5e 163877 1 2024-06-03T01:20:58.167 INFO:tasks.workunit.client.1.smithi044.stdout:0/775: getdents d3/d5c/de3 0 2024-06-03T01:20:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/776: fdatasync d3/d9/d55/fd4 0 2024-06-03T01:20:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/777: stat d3/da6/dc1/dc2/ff9 0 2024-06-03T01:20:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/778: read - d3/da6/d20/faf zero size 2024-06-03T01:20:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:7/664: creat d0/d16/d23/fd2 x:0 0 0 2024-06-03T01:20:58.169 INFO:tasks.workunit.client.1.smithi044.stdout:5/585: symlink d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/lbc 0 2024-06-03T01:20:58.170 INFO:tasks.workunit.client.0.smithi032.stdout:4/166: truncate d4/db/da/d2d/d27/f30 3218410 0 2024-06-03T01:20:58.171 INFO:tasks.workunit.client.0.smithi032.stdout:4/167: dread d4/f7 [4194304,4194304] 0 2024-06-03T01:20:58.172 INFO:tasks.workunit.client.1.smithi044.stdout:6/838: unlink d2/d23/d55/d5f/db2/f115 0 2024-06-03T01:20:58.172 INFO:tasks.workunit.client.1.smithi044.stdout:6/839: fsync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/fdc 0 2024-06-03T01:20:58.173 INFO:tasks.workunit.client.0.smithi032.stdout:6/187: link d8/d11/c16 d8/d11/d28/c3a 0 2024-06-03T01:20:58.173 INFO:tasks.workunit.client.0.smithi032.stdout:6/188: chown d8/fb 988289 1 2024-06-03T01:20:58.173 INFO:tasks.workunit.client.0.smithi032.stdout:6/189: chown d8/ff 1 1 2024-06-03T01:20:58.175 INFO:tasks.workunit.client.0.smithi032.stdout:9/291: creat d0/d11/d1d/d38/d4f/f60 x:0 0 0 2024-06-03T01:20:58.175 INFO:tasks.workunit.client.0.smithi032.stdout:9/292: chown d0/d11/c18 83869415 1 2024-06-03T01:20:58.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/558: unlink d6/dd/d51/d16/d27/d5e/da0/c7a 0 2024-06-03T01:20:58.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/559: chown d6/dd/d51/d16/c2d 3485 1 2024-06-03T01:20:58.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/560: read - d6/dd/d51/d46/f5c zero size 2024-06-03T01:20:58.177 INFO:tasks.workunit.client.0.smithi032.stdout:6/190: getdents d8/d11/d20/d2b/d39 0 2024-06-03T01:20:58.177 INFO:tasks.workunit.client.0.smithi032.stdout:6/191: fdatasync d8/fb 0 2024-06-03T01:20:58.178 INFO:tasks.workunit.client.1.smithi044.stdout:8/695: dread d5/d20/d33/f3f [0,4194304] 0 2024-06-03T01:20:58.179 INFO:tasks.workunit.client.0.smithi032.stdout:8/140: mknod dc/d14/c2b 0 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.0.smithi032.stdout:7/189: dwrite dc/dd/fe [0,4194304] 0 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.0.smithi032.stdout:7/190: fsync dc/d12/d22/f2e 0 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.0.smithi032.stdout:7/191: write f6 [340495,49687] 0 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.0.smithi032.stdout:7/192: dread - dc/d12/d1b/f21 zero size 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.1.smithi044.stdout:2/504: rename db/d15/l3a to db/d15/d49/d6c/d72/d8f/l9d 0 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.1.smithi044.stdout:2/505: chown db/d15/d49/d6c/d72/d7a/d4e/l7f 173570 1 2024-06-03T01:20:58.180 INFO:tasks.workunit.client.1.smithi044.stdout:3/557: symlink df/d13/d39/d3d/d43/d71/lb9 0 2024-06-03T01:20:58.181 INFO:tasks.workunit.client.0.smithi032.stdout:9/293: creat d0/d11/d1d/d38/f61 x:0 0 0 2024-06-03T01:20:58.181 INFO:tasks.workunit.client.0.smithi032.stdout:2/204: dwrite d5/f19 [4194304,4194304] 0 2024-06-03T01:20:58.182 INFO:tasks.workunit.client.1.smithi044.stdout:0/779: symlink d3/da6/d56/d62/de7/l11e 0 2024-06-03T01:20:58.182 INFO:tasks.workunit.client.1.smithi044.stdout:0/780: dread - d3/d9/fc8 zero size 2024-06-03T01:20:58.182 INFO:tasks.workunit.client.1.smithi044.stdout:4/699: dread d17/d1c/d75/d7a/db8/d6a/f73 [0,4194304] 0 2024-06-03T01:20:58.182 INFO:tasks.workunit.client.1.smithi044.stdout:5/586: mknod d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/cbd 0 2024-06-03T01:20:58.183 INFO:tasks.workunit.client.0.smithi032.stdout:7/193: write fa [1161830,123109] 0 2024-06-03T01:20:58.184 INFO:tasks.workunit.client.1.smithi044.stdout:9/561: mknod d6/dd/d51/d16/d5f/d79/cb3 0 2024-06-03T01:20:58.185 INFO:tasks.workunit.client.1.smithi044.stdout:7/665: dread d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [0,4194304] 0 2024-06-03T01:20:58.185 INFO:tasks.workunit.client.1.smithi044.stdout:2/506: creat db/d1b/d2a/d67/d53/d9b/f9e x:0 0 0 2024-06-03T01:20:58.185 INFO:tasks.workunit.client.1.smithi044.stdout:2/507: write db/f12 [1610729,18304] 0 2024-06-03T01:20:58.187 INFO:tasks.workunit.client.0.smithi032.stdout:6/192: dread d8/d11/d20/d2b/d39/d19/f1d [0,4194304] 0 2024-06-03T01:20:58.188 INFO:tasks.workunit.client.1.smithi044.stdout:5/587: mknod d2/d11/d12/cbe 0 2024-06-03T01:20:58.191 INFO:tasks.workunit.client.1.smithi044.stdout:0/781: dread d3/d9/d36/f43 [0,4194304] 0 2024-06-03T01:20:58.193 INFO:tasks.workunit.client.1.smithi044.stdout:9/562: creat d6/dd/d51/db/fb4 x:0 0 0 2024-06-03T01:20:58.193 INFO:tasks.workunit.client.1.smithi044.stdout:4/700: rmdir d17/d1c/d75/d7a/db8/db9 39 2024-06-03T01:20:58.193 INFO:tasks.workunit.client.1.smithi044.stdout:4/701: write d17/d1c/d75/d7a/db8/d59/f7b [1584807,4148] 0 2024-06-03T01:20:58.193 INFO:tasks.workunit.client.1.smithi044.stdout:4/702: fsync d17/d1c/d75/d7a/db8/db9/fc3 0 2024-06-03T01:20:58.194 INFO:tasks.workunit.client.1.smithi044.stdout:4/703: fdatasync d17/d1c/d75/d7a/db8/db9/fc8 0 2024-06-03T01:20:58.194 INFO:tasks.workunit.client.1.smithi044.stdout:4/704: readlink d17/d1c/d70/d7d/l83 0 2024-06-03T01:20:58.195 INFO:tasks.workunit.client.1.smithi044.stdout:5/588: rmdir d2/d11/d12 39 2024-06-03T01:20:58.196 INFO:tasks.workunit.client.1.smithi044.stdout:0/782: creat d3/d9/d102/f11f x:0 0 0 2024-06-03T01:20:58.196 INFO:tasks.workunit.client.1.smithi044.stdout:0/783: getdents d3/d5c/de3 0 2024-06-03T01:20:58.196 INFO:tasks.workunit.client.1.smithi044.stdout:0/784: chown d3/d9/d55/d73/f77 117648518 1 2024-06-03T01:20:58.197 INFO:tasks.workunit.client.1.smithi044.stdout:0/785: truncate d3/f69 2103750 0 2024-06-03T01:20:58.197 INFO:tasks.workunit.client.1.smithi044.stdout:8/696: dread d5/d55/fad [0,4194304] 0 2024-06-03T01:20:58.197 INFO:tasks.workunit.client.1.smithi044.stdout:8/697: write d5/d55/d74/db4/fd7 [1009426,8834] 0 2024-06-03T01:20:58.197 INFO:tasks.workunit.client.1.smithi044.stdout:9/563: mknod d6/d38/cb5 0 2024-06-03T01:20:58.198 INFO:tasks.workunit.client.0.smithi032.stdout:9/294: creat d0/d23/d25/f62 x:0 0 0 2024-06-03T01:20:58.198 INFO:tasks.workunit.client.0.smithi032.stdout:9/295: write d0/f1b [5352680,67587] 0 2024-06-03T01:20:58.198 INFO:tasks.workunit.client.0.smithi032.stdout:2/205: truncate d5/da/f29 7242633 0 2024-06-03T01:20:58.198 INFO:tasks.workunit.client.1.smithi044.stdout:2/508: getdents db/d1b/d5d 0 2024-06-03T01:20:58.198 INFO:tasks.workunit.client.0.smithi032.stdout:7/194: mknod dc/d12/d2a/c3f 0 2024-06-03T01:20:58.199 INFO:tasks.workunit.client.1.smithi044.stdout:3/558: dread df/f17 [8388608,4194304] 0 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:7/666: dread d0/d3/d8/d32/f90 [0,4194304] 0 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:7/667: write d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/f80 [843813,62994] 0 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/589: creat d2/d11/d12/dad/fbf x:0 0 0 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/590: dread - d2/d11/d12/dad/fbf zero size 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/591: truncate d2/d11/d12/d2b/d4a/d57/fb4 570280 0 2024-06-03T01:20:58.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/592: fsync d2/d11/d1e/d2f/f64 0 2024-06-03T01:20:58.201 INFO:tasks.workunit.client.0.smithi032.stdout:8/141: symlink dc/de/l2c 0 2024-06-03T01:20:58.202 INFO:tasks.workunit.client.0.smithi032.stdout:6/193: mkdir d8/d11/d20/d3b 0 2024-06-03T01:20:58.203 INFO:tasks.workunit.client.1.smithi044.stdout:2/509: creat db/d1b/d2a/d2f/f9f x:0 0 0 2024-06-03T01:20:58.203 INFO:tasks.workunit.client.1.smithi044.stdout:2/510: chown db/d15/d49/d6c/d72/d8f 1300989642 1 2024-06-03T01:20:58.204 INFO:tasks.workunit.client.1.smithi044.stdout:7/668: truncate d0/d16/d23/f24 4253647 0 2024-06-03T01:20:58.206 INFO:tasks.workunit.client.1.smithi044.stdout:4/705: dread d17/d1c/d75/d7a/db8/f71 [0,4194304] 0 2024-06-03T01:20:58.206 INFO:tasks.workunit.client.1.smithi044.stdout:4/706: chown d17/d1c/d75/f62 504 1 2024-06-03T01:20:58.207 INFO:tasks.workunit.client.0.smithi032.stdout:4/168: dwrite d4/db/da/d2d/d27/f2b [0,4194304] 0 2024-06-03T01:20:58.207 INFO:tasks.workunit.client.0.smithi032.stdout:4/169: readlink d4/db/l24 0 2024-06-03T01:20:58.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/786: write d3/d9/d55/fd4 [2118729,105978] 0 2024-06-03T01:20:58.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/787: creat d3/da6/d20/d25/dbf/f120 x:0 0 0 2024-06-03T01:20:58.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/593: mknod d2/d11/d1e/d21/d24/d93/d9d/cc0 0 2024-06-03T01:20:58.211 INFO:tasks.workunit.client.0.smithi032.stdout:9/296: rename d0/f1b to d0/d11/f63 0 2024-06-03T01:20:58.211 INFO:tasks.workunit.client.0.smithi032.stdout:3/199: dwrite de/d29/f3a [0,4194304] 0 2024-06-03T01:20:58.211 INFO:tasks.workunit.client.0.smithi032.stdout:2/206: creat d5/da/d1a/f3e x:0 0 0 2024-06-03T01:20:58.213 INFO:tasks.workunit.client.1.smithi044.stdout:2/511: rename db/d1b/d2a/d52/d54/l75 to db/d15/d49/d6c/d72/d7a/la0 0 2024-06-03T01:20:58.215 INFO:tasks.workunit.client.0.smithi032.stdout:6/194: creat d8/d11/d20/d2b/f3c x:0 0 0 2024-06-03T01:20:58.215 INFO:tasks.workunit.client.0.smithi032.stdout:6/195: write f4 [2595599,17528] 0 2024-06-03T01:20:58.215 INFO:tasks.workunit.client.0.smithi032.stdout:9/297: symlink d0/d11/d1d/d38/l64 0 2024-06-03T01:20:58.216 INFO:tasks.workunit.client.1.smithi044.stdout:7/669: creat d0/d3/d8/d95/d63/d7f/fd3 x:0 0 0 2024-06-03T01:20:58.217 INFO:tasks.workunit.client.1.smithi044.stdout:4/707: creat d17/d1c/d75/d7a/db8/db9/ff1 x:0 0 0 2024-06-03T01:20:58.217 INFO:tasks.workunit.client.1.smithi044.stdout:4/708: chown d17/d1c/d75/d7a/db3/fdc 2040 1 2024-06-03T01:20:58.218 INFO:tasks.workunit.client.1.smithi044.stdout:3/559: write df/d13/d1b/d56/f67 [2067403,128022] 0 2024-06-03T01:20:58.219 INFO:tasks.workunit.client.1.smithi044.stdout:9/564: dread d6/dd/d51/d16/f3f [0,4194304] 0 2024-06-03T01:20:58.219 INFO:tasks.workunit.client.1.smithi044.stdout:9/565: write d6/dd/f21 [4825250,74363] 0 2024-06-03T01:20:58.219 INFO:tasks.workunit.client.1.smithi044.stdout:9/566: read - d6/dd/d51/d16/d5f/d79/f90 zero size 2024-06-03T01:20:58.222 INFO:tasks.workunit.client.0.smithi032.stdout:4/170: dread d4/db/da/d2d/d27/f30 [0,4194304] 0 2024-06-03T01:20:58.222 INFO:tasks.workunit.client.0.smithi032.stdout:4/171: write d4/db/da/f31 [595971,68720] 0 2024-06-03T01:20:58.222 INFO:tasks.workunit.client.0.smithi032.stdout:4/172: fdatasync d4/db/da/d1d/f23 0 2024-06-03T01:20:58.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/594: rename d2/d11/d1e/d3b/f79 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/fc1 0 2024-06-03T01:20:58.226 INFO:tasks.workunit.client.1.smithi044.stdout:2/512: truncate db/dd/f60 1171228 0 2024-06-03T01:20:58.226 INFO:tasks.workunit.client.1.smithi044.stdout:2/513: stat db/d1b/d2a/d2f/d4c 0 2024-06-03T01:20:58.226 INFO:tasks.workunit.client.1.smithi044.stdout:7/670: mknod d0/d3/d8/d95/d63/d89/cd4 0 2024-06-03T01:20:58.227 INFO:tasks.workunit.client.0.smithi032.stdout:7/195: dwrite dc/d12/d2a/d2f/f39 [0,4194304] 0 2024-06-03T01:20:58.230 INFO:tasks.workunit.client.1.smithi044.stdout:3/560: stat df/d13/d1b/d4b/d69/f95 0 2024-06-03T01:20:58.230 INFO:tasks.workunit.client.1.smithi044.stdout:3/561: dread - df/d1e/d3a/f87 zero size 2024-06-03T01:20:58.231 INFO:tasks.workunit.client.1.smithi044.stdout:8/698: dwrite d5/d16/d81/f83 [0,4194304] 0 2024-06-03T01:20:58.233 INFO:tasks.workunit.client.1.smithi044.stdout:0/788: dwrite d3/f8 [0,4194304] 0 2024-06-03T01:20:58.235 INFO:tasks.workunit.client.0.smithi032.stdout:3/200: creat de/d29/f3b x:0 0 0 2024-06-03T01:20:58.239 INFO:tasks.workunit.client.1.smithi044.stdout:2/514: creat db/d1b/d27/d79/fa1 x:0 0 0 2024-06-03T01:20:58.239 INFO:tasks.workunit.client.0.smithi032.stdout:2/207: mkdir d5/da/d1a/d2a/d36/d3f 0 2024-06-03T01:20:58.239 INFO:tasks.workunit.client.0.smithi032.stdout:6/196: write d8/d11/d20/d2b/d39/d19/f1d [4205427,46315] 0 2024-06-03T01:20:58.239 INFO:tasks.workunit.client.0.smithi032.stdout:6/197: fsync d8/d11/f35 0 2024-06-03T01:20:58.240 INFO:tasks.workunit.client.1.smithi044.stdout:7/671: mkdir d0/d3/d8/d95/d84/dd5 0 2024-06-03T01:20:58.242 INFO:tasks.workunit.client.0.smithi032.stdout:4/173: mknod d4/db/da/d2d/c38 0 2024-06-03T01:20:58.242 INFO:tasks.workunit.client.0.smithi032.stdout:4/174: fdatasync d4/db/da/d1d/f23 0 2024-06-03T01:20:58.244 INFO:tasks.workunit.client.1.smithi044.stdout:0/789: symlink d3/da6/d20/l121 0 2024-06-03T01:20:58.245 INFO:tasks.workunit.client.1.smithi044.stdout:3/562: creat df/d1e/d2a/d70/fba x:0 0 0 2024-06-03T01:20:58.245 INFO:tasks.workunit.client.1.smithi044.stdout:0/790: chown d3/da6/d56/d8e/cff 12266719 1 2024-06-03T01:20:58.247 INFO:tasks.workunit.client.1.smithi044.stdout:7/672: creat d0/d3/d8/d32/d33/fd6 x:0 0 0 2024-06-03T01:20:58.247 INFO:tasks.workunit.client.1.smithi044.stdout:8/699: dread d5/d20/d47/d64/f8b [0,4194304] 0 2024-06-03T01:20:58.248 INFO:tasks.workunit.client.1.smithi044.stdout:0/791: unlink d3/da6/d21/fc4 0 2024-06-03T01:20:58.248 INFO:tasks.workunit.client.1.smithi044.stdout:3/563: symlink df/d1e/d78/da4/lbb 0 2024-06-03T01:20:58.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/112: sync 2024-06-03T01:20:58.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/113: write d18/f24 [696176,97728] 0 2024-06-03T01:20:58.249 INFO:tasks.workunit.client.0.smithi032.stdout:5/220: sync 2024-06-03T01:20:58.251 INFO:tasks.workunit.client.1.smithi044.stdout:7/673: rename d0/d3/d8/d95/d63/d89/da3/f66 to d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/fd7 0 2024-06-03T01:20:58.251 INFO:tasks.workunit.client.1.smithi044.stdout:7/674: readlink d0/d16/l77 0 2024-06-03T01:20:58.254 INFO:tasks.workunit.client.1.smithi044.stdout:8/700: mkdir d5/d55/d56/d5c/d68/de5 0 2024-06-03T01:20:58.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/792: rmdir d3/d6b/d95/db3 39 2024-06-03T01:20:58.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/793: creat d3/da6/dc1/dc2/de4/f122 x:0 0 0 2024-06-03T01:20:58.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/794: write d3/fc6 [484996,21799] 0 2024-06-03T01:20:58.255 INFO:tasks.workunit.client.1.smithi044.stdout:3/564: rmdir df/d13/d6d/d9a 39 2024-06-03T01:20:58.257 INFO:tasks.workunit.client.0.smithi032.stdout:2/208: dread d5/da/f1e [0,4194304] 0 2024-06-03T01:20:58.257 INFO:tasks.workunit.client.0.smithi032.stdout:6/198: mknod d8/d11/d20/d2b/d39/c3d 0 2024-06-03T01:20:58.257 INFO:tasks.workunit.client.0.smithi032.stdout:6/199: creat d8/d11/f3e x:0 0 0 2024-06-03T01:20:58.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/795: creat d3/d9/d55/d5a/f123 x:0 0 0 2024-06-03T01:20:58.257 INFO:tasks.workunit.client.1.smithi044.stdout:8/701: dread d5/d16/d1d/d59/f6f [0,4194304] 0 2024-06-03T01:20:58.258 INFO:tasks.workunit.client.1.smithi044.stdout:8/702: chown d5/d55 644 1 2024-06-03T01:20:58.258 INFO:tasks.workunit.client.0.smithi032.stdout:8/142: rename dc/d14/f20 to dc/d14/f2d 0 2024-06-03T01:20:58.259 INFO:tasks.workunit.client.1.smithi044.stdout:3/565: unlink df/d1e/d3a/l9f 0 2024-06-03T01:20:58.259 INFO:tasks.workunit.client.1.smithi044.stdout:3/566: dread - df/d1e/d3a/f88 zero size 2024-06-03T01:20:58.259 INFO:tasks.workunit.client.1.smithi044.stdout:0/796: symlink d3/d6b/l124 0 2024-06-03T01:20:58.260 INFO:tasks.workunit.client.0.smithi032.stdout:9/298: mknod d0/c65 0 2024-06-03T01:20:58.260 INFO:tasks.workunit.client.1.smithi044.stdout:8/703: truncate d5/d20/d33/f4b 1812856 0 2024-06-03T01:20:58.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/567: mkdir df/d1e/d2a/d70/dbc 0 2024-06-03T01:20:58.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/568: fsync df/d13/d6d/d9a/faa 0 2024-06-03T01:20:58.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/797: mkdir d3/da6/d20/d25/d125 0 2024-06-03T01:20:58.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/798: creat d3/d9/f126 x:0 0 0 2024-06-03T01:20:58.264 INFO:tasks.workunit.client.0.smithi032.stdout:5/221: symlink da/dd/df/d10/d16/d18/d1a/d30/l43 0 2024-06-03T01:20:58.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/569: creat df/d13/d39/d3d/d6a/d9c/fbd x:0 0 0 2024-06-03T01:20:58.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/570: creat df/d1e/d2a/d70/fbe x:0 0 0 2024-06-03T01:20:58.266 INFO:tasks.workunit.client.0.smithi032.stdout:2/209: getdents d5/da 0 2024-06-03T01:20:58.267 INFO:tasks.workunit.client.0.smithi032.stdout:6/200: mkdir d8/d11/d20/d31/d3f 0 2024-06-03T01:20:58.268 INFO:tasks.workunit.client.0.smithi032.stdout:8/143: write dc/d14/f22 [186097,78834] 0 2024-06-03T01:20:58.269 INFO:tasks.workunit.client.0.smithi032.stdout:9/299: mknod d0/d11/d1d/d38/c66 0 2024-06-03T01:20:58.271 INFO:tasks.workunit.client.1.smithi044.stdout:3/571: truncate df/d1e/d2a/f30 1427895 0 2024-06-03T01:20:58.271 INFO:tasks.workunit.client.1.smithi044.stdout:3/572: write df/f19 [2502491,109802] 0 2024-06-03T01:20:58.271 INFO:tasks.workunit.client.1.smithi044.stdout:3/573: dread - df/d1e/d3a/fb6 zero size 2024-06-03T01:20:58.272 INFO:tasks.workunit.client.0.smithi032.stdout:6/201: creat d8/d11/d20/d2b/d39/d19/f40 x:0 0 0 2024-06-03T01:20:58.273 INFO:tasks.workunit.client.1.smithi044.stdout:4/709: dwrite d17/d1c/d75/d7a/db8/d28/d2b/f92 [0,4194304] 0 2024-06-03T01:20:58.273 INFO:tasks.workunit.client.0.smithi032.stdout:8/144: creat dc/de/d1e/f2e x:0 0 0 2024-06-03T01:20:58.274 INFO:tasks.workunit.client.1.smithi044.stdout:0/799: getdents d3/da6/d56/d62/de7/df3 0 2024-06-03T01:20:58.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/704: dread d5/d55/d74/d87/f7b [0,4194304] 0 2024-06-03T01:20:58.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/705: chown d5/d16/l19 10 1 2024-06-03T01:20:58.275 INFO:tasks.workunit.client.0.smithi032.stdout:7/196: dwrite dc/f16 [0,4194304] 0 2024-06-03T01:20:58.275 INFO:tasks.workunit.client.0.smithi032.stdout:7/197: truncate dc/dd/f20 4763861 0 2024-06-03T01:20:58.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/595: dwrite d2/d11/d1e/d21/d24/d93/d9d/f15 [0,4194304] 0 2024-06-03T01:20:58.276 INFO:tasks.workunit.client.1.smithi044.stdout:5/596: write d2/d11/d1e/d21/d24/d93/d9d/d28/f2c [80779,23621] 0 2024-06-03T01:20:58.276 INFO:tasks.workunit.client.0.smithi032.stdout:9/300: unlink d0/d23/d25/f37 0 2024-06-03T01:20:58.278 INFO:tasks.workunit.client.0.smithi032.stdout:5/222: getdents da/dd 0 2024-06-03T01:20:58.278 INFO:tasks.workunit.client.0.smithi032.stdout:5/223: stat da/dd/df/d10/d16/f36 0 2024-06-03T01:20:58.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/145: link dc/de/l21 dc/d25/l2f 0 2024-06-03T01:20:58.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/146: readlink dc/de/l21 0 2024-06-03T01:20:58.281 INFO:tasks.workunit.client.1.smithi044.stdout:3/574: creat df/d1e/d78/da4/fbf x:0 0 0 2024-06-03T01:20:58.281 INFO:tasks.workunit.client.1.smithi044.stdout:3/575: write df/d1e/d2a/d70/f96 [116676,37229] 0 2024-06-03T01:20:58.283 INFO:tasks.workunit.client.1.smithi044.stdout:4/710: creat d17/d63/dae/ff2 x:0 0 0 2024-06-03T01:20:58.284 INFO:tasks.workunit.client.1.smithi044.stdout:1/759: sync 2024-06-03T01:20:58.284 INFO:tasks.workunit.client.1.smithi044.stdout:1/760: write d0/d3/d20/d7/d13/d2d/dcd/ff3 [671683,126235] 0 2024-06-03T01:20:58.284 INFO:tasks.workunit.client.1.smithi044.stdout:3/576: dread df/d13/d39/d3d/d43/d71/fa2 [0,4194304] 0 2024-06-03T01:20:58.284 INFO:tasks.workunit.client.1.smithi044.stdout:3/577: stat df/d13/f16 0 2024-06-03T01:20:58.285 INFO:tasks.workunit.client.1.smithi044.stdout:7/675: dwrite d0/d16/d23/f78 [0,4194304] 0 2024-06-03T01:20:58.285 INFO:tasks.workunit.client.0.smithi032.stdout:3/201: dwrite de/f11 [0,4194304] 0 2024-06-03T01:20:58.285 INFO:tasks.workunit.client.0.smithi032.stdout:1/114: dwrite d18/f1e [0,4194304] 0 2024-06-03T01:20:58.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/800: rename d3/d9/d55/d5a/dde/dea/cf5 to d3/d6b/de8/c127 0 2024-06-03T01:20:58.287 INFO:tasks.workunit.client.0.smithi032.stdout:0/234: sync 2024-06-03T01:20:58.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/198: creat dc/d12/d2a/f40 x:0 0 0 2024-06-03T01:20:58.290 INFO:tasks.workunit.client.0.smithi032.stdout:8/147: symlink dc/l30 0 2024-06-03T01:20:58.293 INFO:tasks.workunit.client.0.smithi032.stdout:5/224: dread da/dd/df/d10/d16/d25/f3d [0,4194304] 0 2024-06-03T01:20:58.295 INFO:tasks.workunit.client.0.smithi032.stdout:3/202: mkdir de/d29/d2e/d31/d3c 0 2024-06-03T01:20:58.295 INFO:tasks.workunit.client.0.smithi032.stdout:2/210: dwrite d5/da/d1a/f26 [0,4194304] 0 2024-06-03T01:20:58.296 INFO:tasks.workunit.client.0.smithi032.stdout:6/202: dwrite d8/d11/d20/d2b/d39/f1f [0,4194304] 0 2024-06-03T01:20:58.296 INFO:tasks.workunit.client.0.smithi032.stdout:6/203: chown d8/lc 534 1 2024-06-03T01:20:58.299 INFO:tasks.workunit.client.0.smithi032.stdout:7/199: symlink dc/d12/d24/l41 0 2024-06-03T01:20:58.299 INFO:tasks.workunit.client.0.smithi032.stdout:8/148: truncate dc/de/f10 3422822 0 2024-06-03T01:20:58.299 INFO:tasks.workunit.client.0.smithi032.stdout:5/225: mkdir da/dd/df/d10/d16/d18/d35/d44 0 2024-06-03T01:20:58.299 INFO:tasks.workunit.client.0.smithi032.stdout:5/226: stat da/cb 0 2024-06-03T01:20:58.299 INFO:tasks.workunit.client.1.smithi044.stdout:5/597: symlink d2/lc2 0 2024-06-03T01:20:58.300 INFO:tasks.workunit.client.0.smithi032.stdout:0/235: rename d1/da/d10/d29/d31 to d1/d46 0 2024-06-03T01:20:58.300 INFO:tasks.workunit.client.1.smithi044.stdout:5/598: dread d2/d11/d12/d2b/d4a/d57/fb4 [0,4194304] 0 2024-06-03T01:20:58.300 INFO:tasks.workunit.client.1.smithi044.stdout:5/599: chown d2/d11/d1e/d21/d24/d93/d9d/f49 1480 1 2024-06-03T01:20:58.301 INFO:tasks.workunit.client.1.smithi044.stdout:5/600: fdatasync d2/d11/d1e/f95 0 2024-06-03T01:20:58.303 INFO:tasks.workunit.client.0.smithi032.stdout:2/211: mknod d5/c40 0 2024-06-03T01:20:58.303 INFO:tasks.workunit.client.0.smithi032.stdout:2/212: chown d5/da/f1e 5510579 1 2024-06-03T01:20:58.303 INFO:tasks.workunit.client.0.smithi032.stdout:2/213: creat d5/da/d1a/f41 x:0 0 0 2024-06-03T01:20:58.305 INFO:tasks.workunit.client.0.smithi032.stdout:6/204: creat d8/d11/d20/d3b/f41 x:0 0 0 2024-06-03T01:20:58.306 INFO:tasks.workunit.client.0.smithi032.stdout:6/205: dread - d8/d11/d20/d3b/f41 zero size 2024-06-03T01:20:58.306 INFO:tasks.workunit.client.0.smithi032.stdout:6/206: readlink d8/d11/d20/d31/l33 0 2024-06-03T01:20:58.306 INFO:tasks.workunit.client.1.smithi044.stdout:4/711: unlink d17/d1c/d75/d7a/db8/d35/l6b 0 2024-06-03T01:20:58.307 INFO:tasks.workunit.client.1.smithi044.stdout:3/578: creat df/d1e/d3a/d84/fc0 x:0 0 0 2024-06-03T01:20:58.310 INFO:tasks.workunit.client.1.smithi044.stdout:7/676: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/fd8 x:0 0 0 2024-06-03T01:20:58.310 INFO:tasks.workunit.client.1.smithi044.stdout:7/677: fdatasync d0/d3/d8/d95/fa6 0 2024-06-03T01:20:58.310 INFO:tasks.workunit.client.1.smithi044.stdout:7/678: chown d0/d3/d8/d95/d63/d7f/d9c/da4/dc9 0 1 2024-06-03T01:20:58.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/115: dwrite d18/f22 [0,4194304] 0 2024-06-03T01:20:58.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/116: readlink d18/l1d 0 2024-06-03T01:20:58.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/117: chown d18/c20 6565633 1 2024-06-03T01:20:58.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/118: write d18/f1a [1583531,121018] 0 2024-06-03T01:20:58.312 INFO:tasks.workunit.client.1.smithi044.stdout:4/712: dread d17/d1c/d70/fe4 [0,4194304] 0 2024-06-03T01:20:58.312 INFO:tasks.workunit.client.1.smithi044.stdout:4/713: write d17/d1c/d75/d7a/db8/f67 [402013,45289] 0 2024-06-03T01:20:58.312 INFO:tasks.workunit.client.0.smithi032.stdout:9/301: dwrite d0/d23/d24/d26/f39 [0,4194304] 0 2024-06-03T01:20:58.312 INFO:tasks.workunit.client.0.smithi032.stdout:5/227: symlink da/dd/df/d10/d16/d25/l45 0 2024-06-03T01:20:58.313 INFO:tasks.workunit.client.1.smithi044.stdout:0/801: symlink d3/d9/d55/d5a/dde/l128 0 2024-06-03T01:20:58.314 INFO:tasks.workunit.client.0.smithi032.stdout:3/203: rename de/c12 to de/d29/d2e/d31/c3d 0 2024-06-03T01:20:58.314 INFO:tasks.workunit.client.0.smithi032.stdout:3/204: chown f8 5579 1 2024-06-03T01:20:58.318 INFO:tasks.workunit.client.1.smithi044.stdout:8/706: getdents d5/d55/d74/db4 0 2024-06-03T01:20:58.318 INFO:tasks.workunit.client.1.smithi044.stdout:8/707: stat d5/f15 0 2024-06-03T01:20:58.318 INFO:tasks.workunit.client.1.smithi044.stdout:8/708: dread - d5/fe4 zero size 2024-06-03T01:20:58.318 INFO:tasks.workunit.client.0.smithi032.stdout:2/214: symlink d5/d38/l42 0 2024-06-03T01:20:58.319 INFO:tasks.workunit.client.0.smithi032.stdout:7/200: dwrite dc/d12/d1b/f36 [4194304,4194304] 0 2024-06-03T01:20:58.320 INFO:tasks.workunit.client.0.smithi032.stdout:1/119: read d18/f22 [422871,51206] 0 2024-06-03T01:20:58.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/601: symlink d2/d11/d1e/d21/d24/d93/d9d/d28/d66/lc3 0 2024-06-03T01:20:58.321 INFO:tasks.workunit.client.0.smithi032.stdout:6/207: mknod d8/d11/d20/d2b/d39/c42 0 2024-06-03T01:20:58.325 INFO:tasks.workunit.client.0.smithi032.stdout:8/149: mkdir dc/d14/d31 0 2024-06-03T01:20:58.328 INFO:tasks.workunit.client.1.smithi044.stdout:7/679: mknod d0/d3/d8/d95/d63/d89/da3/cd9 0 2024-06-03T01:20:58.332 INFO:tasks.workunit.client.0.smithi032.stdout:9/302: truncate d0/fe 902520 0 2024-06-03T01:20:58.334 INFO:tasks.workunit.client.1.smithi044.stdout:4/714: unlink d17/d1c/d75/d7a/db8/d28/f3e 0 2024-06-03T01:20:58.334 INFO:tasks.workunit.client.1.smithi044.stdout:4/715: truncate d17/d1c/d75/d7a/db8/d59/fda 297276 0 2024-06-03T01:20:58.336 INFO:tasks.workunit.client.0.smithi032.stdout:5/228: rename da/dd/df/d10/d16/d25/f3d to da/dd/df/f46 0 2024-06-03T01:20:58.337 INFO:tasks.workunit.client.0.smithi032.stdout:7/201: write dc/d12/d1b/f1d [5821749,92800] 0 2024-06-03T01:20:58.337 INFO:tasks.workunit.client.1.smithi044.stdout:0/802: mkdir d3/da6/d20/d25/dbf/d129 0 2024-06-03T01:20:58.338 INFO:tasks.workunit.client.1.smithi044.stdout:8/709: symlink d5/d20/d33/le6 0 2024-06-03T01:20:58.339 INFO:tasks.workunit.client.0.smithi032.stdout:3/205: link de/d29/d2e/d31/d24/f39 de/d29/f3e 0 2024-06-03T01:20:58.339 INFO:tasks.workunit.client.0.smithi032.stdout:3/206: getdents de/d29/d2e/d31/d3c 0 2024-06-03T01:20:58.339 INFO:tasks.workunit.client.0.smithi032.stdout:3/207: chown de/d29/d2e/d31/c3d 380964 1 2024-06-03T01:20:58.340 INFO:tasks.workunit.client.0.smithi032.stdout:0/236: mkdir d1/d47 0 2024-06-03T01:20:58.340 INFO:tasks.workunit.client.0.smithi032.stdout:1/120: readlink l7 0 2024-06-03T01:20:58.341 INFO:tasks.workunit.client.1.smithi044.stdout:1/761: rename d0/d16 to d0/d3/d20/d7/d13/d2d/dcd/d108 0 2024-06-03T01:20:58.343 INFO:tasks.workunit.client.1.smithi044.stdout:4/716: link d17/d1c/d75/d7a/db8/d8d/fee d17/d1c/d9c/ff3 0 2024-06-03T01:20:58.343 INFO:tasks.workunit.client.1.smithi044.stdout:4/717: write d17/d1c/d9c/ff3 [417571,91628] 0 2024-06-03T01:20:58.344 INFO:tasks.workunit.client.0.smithi032.stdout:8/150: mkdir dc/d25/d28/d32 0 2024-06-03T01:20:58.344 INFO:tasks.workunit.client.1.smithi044.stdout:1/762: mkdir d0/dd2/dfe/d109 0 2024-06-03T01:20:58.345 INFO:tasks.workunit.client.0.smithi032.stdout:9/303: rename d0/d11/d1d/l33 to d0/d11/d1d/l67 0 2024-06-03T01:20:58.346 INFO:tasks.workunit.client.1.smithi044.stdout:0/803: rename d3/d9/d55/d5a/d29/c70 to d3/d6b/de8/d115/c12a 0 2024-06-03T01:20:58.347 INFO:tasks.workunit.client.0.smithi032.stdout:7/202: truncate dc/d12/d22/f2e 409525 0 2024-06-03T01:20:58.348 INFO:tasks.workunit.client.1.smithi044.stdout:8/710: rename d5/l8 to d5/d55/d56/d5c/dbb/d79/le7 0 2024-06-03T01:20:58.350 INFO:tasks.workunit.client.0.smithi032.stdout:0/237: link d1/f18 d1/d46/f48 0 2024-06-03T01:20:58.350 INFO:tasks.workunit.client.1.smithi044.stdout:0/804: creat d3/d6b/d95/db3/f12b x:0 0 0 2024-06-03T01:20:58.352 INFO:tasks.workunit.client.1.smithi044.stdout:8/711: symlink d5/d20/d47/d64/le8 0 2024-06-03T01:20:58.352 INFO:tasks.workunit.client.1.smithi044.stdout:8/712: fsync d5/d55/d74/db4/fcb 0 2024-06-03T01:20:58.355 INFO:tasks.workunit.client.0.smithi032.stdout:1/121: write d18/f19 [3095263,26215] 0 2024-06-03T01:20:58.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/203: mkdir dc/d12/d22/d42 0 2024-06-03T01:20:58.356 INFO:tasks.workunit.client.1.smithi044.stdout:8/713: creat d5/d55/d56/d5c/d68/de5/fe9 x:0 0 0 2024-06-03T01:20:58.357 INFO:tasks.workunit.client.1.smithi044.stdout:8/714: stat d5/d16/l6c 0 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/238: mknod d1/da/d10/d29/c49 0 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/239: write d1/da/d10/f2d [1314823,109878] 0 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/240: rename d1 to d1/d46/d3a/d4a 22 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/241: chown d1/da/d10/c25 6145902 1 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/242: truncate d1/da/d10/d29/f44 763686 0 2024-06-03T01:20:58.358 INFO:tasks.workunit.client.0.smithi032.stdout:9/304: getdents d0/d23/d25 0 2024-06-03T01:20:58.362 INFO:tasks.workunit.client.1.smithi044.stdout:0/805: write d3/da6/d20/fdc [373350,81634] 0 2024-06-03T01:20:58.363 INFO:tasks.workunit.client.0.smithi032.stdout:0/243: symlink d1/da/d10/d29/l4b 0 2024-06-03T01:20:58.364 INFO:tasks.workunit.client.0.smithi032.stdout:2/215: dwrite d5/da/f29 [0,4194304] 0 2024-06-03T01:20:58.364 INFO:tasks.workunit.client.0.smithi032.stdout:2/216: dread - d5/da/d1a/f41 zero size 2024-06-03T01:20:58.366 INFO:tasks.workunit.client.1.smithi044.stdout:7/680: dwrite d0/d3/d8/d95/d63/d7f/f91 [4194304,4194304] 0 2024-06-03T01:20:58.366 INFO:tasks.workunit.client.1.smithi044.stdout:7/681: creat d0/d3/d8/d95/d63/d89/fda x:0 0 0 2024-06-03T01:20:58.367 INFO:tasks.workunit.client.0.smithi032.stdout:9/305: mknod d0/d11/d1d/d38/c68 0 2024-06-03T01:20:58.368 INFO:tasks.workunit.client.1.smithi044.stdout:1/763: read d0/f29 [2553296,14703] 0 2024-06-03T01:20:58.368 INFO:tasks.workunit.client.1.smithi044.stdout:5/602: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [0,4194304] 0 2024-06-03T01:20:58.370 INFO:tasks.workunit.client.0.smithi032.stdout:2/217: truncate d5/da/f2f 529758 0 2024-06-03T01:20:58.372 INFO:tasks.workunit.client.1.smithi044.stdout:5/603: symlink d2/d11/d12/d2b/d44/lc4 0 2024-06-03T01:20:58.372 INFO:tasks.workunit.client.1.smithi044.stdout:5/604: dread - d2/d11/d1e/d21/d24/d93/d9d/fb6 zero size 2024-06-03T01:20:58.372 INFO:tasks.workunit.client.1.smithi044.stdout:5/605: truncate d2/d11/d1e/f95 43765 0 2024-06-03T01:20:58.373 INFO:tasks.workunit.client.0.smithi032.stdout:2/218: link d5/c30 d5/da/d1a/d2a/d36/c43 0 2024-06-03T01:20:58.379 INFO:tasks.workunit.client.0.smithi032.stdout:5/229: rmdir da/dd/df 39 2024-06-03T01:20:58.380 INFO:tasks.workunit.client.0.smithi032.stdout:5/230: chown da/f12 43532 1 2024-06-03T01:20:58.380 INFO:tasks.workunit.client.0.smithi032.stdout:5/231: chown da/dd/df/f46 181 1 2024-06-03T01:20:58.381 INFO:tasks.workunit.client.0.smithi032.stdout:8/151: dread dc/de/f10 [0,4194304] 0 2024-06-03T01:20:58.381 INFO:tasks.workunit.client.1.smithi044.stdout:5/606: dread d2/d11/d1e/d2f/f6e [0,4194304] 0 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.0.smithi032.stdout:5/232: symlink da/dd/df/d10/d16/d18/d35/l47 0 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.0.smithi032.stdout:5/233: chown da/dd/df/d10/d16/d25/l45 930 1 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.0.smithi032.stdout:5/234: chown da/l1e 5817 1 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.0.smithi032.stdout:3/208: dwrite de/d29/f3b [0,4194304] 0 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/682: dread d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [4194304,4194304] 0 2024-06-03T01:20:58.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/683: write d0/d3/d8/d95/d63/d7f/d9c/f50 [4481324,66747] 0 2024-06-03T01:20:58.383 INFO:tasks.workunit.client.1.smithi044.stdout:7/684: write d0/f7c [658961,114022] 0 2024-06-03T01:20:58.384 INFO:tasks.workunit.client.0.smithi032.stdout:8/152: mkdir dc/d25/d28/d33 0 2024-06-03T01:20:58.384 INFO:tasks.workunit.client.1.smithi044.stdout:5/607: link d2/d11/d12/d2b/l78 d2/d11/d1e/lc5 0 2024-06-03T01:20:58.386 INFO:tasks.workunit.client.0.smithi032.stdout:5/235: getdents da/dd/df/d10/d16/d25 0 2024-06-03T01:20:58.389 INFO:tasks.workunit.client.1.smithi044.stdout:4/718: dwrite d17/d1c/d9c/fbd [0,4194304] 0 2024-06-03T01:20:58.389 INFO:tasks.workunit.client.1.smithi044.stdout:4/719: read d17/d1c/d75/fe1 [1638651,12207] 0 2024-06-03T01:20:58.391 INFO:tasks.workunit.client.0.smithi032.stdout:5/236: dread da/dd/df/f33 [0,4194304] 0 2024-06-03T01:20:58.394 INFO:tasks.workunit.client.1.smithi044.stdout:7/685: rmdir d0/d3/d8/d32 39 2024-06-03T01:20:58.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/209: rename de/d29/d2e/d31/d23/f2d to de/d29/f3f 0 2024-06-03T01:20:58.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/210: write de/d29/d2e/d31/f38 [681778,32605] 0 2024-06-03T01:20:58.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/211: dread - de/d29/d2e/d31/f1e zero size 2024-06-03T01:20:58.398 INFO:tasks.workunit.client.1.smithi044.stdout:7/686: rename d0/d3/d8/d32/d33/f5b to d0/d3/d8/d32/d33/fdb 0 2024-06-03T01:20:58.398 INFO:tasks.workunit.client.1.smithi044.stdout:7/687: dread - d0/d3/d8/d95/d63/d7f/d9c/f92 zero size 2024-06-03T01:20:58.399 INFO:tasks.workunit.client.0.smithi032.stdout:8/153: symlink dc/de/l34 0 2024-06-03T01:20:58.400 INFO:tasks.workunit.client.1.smithi044.stdout:5/608: dread d2/d11/d12/d2b/d4a/d57/f50 [0,4194304] 0 2024-06-03T01:20:58.401 INFO:tasks.workunit.client.1.smithi044.stdout:7/688: mknod d0/d3/d8/d95/d63/d7f/d9c/cdc 0 2024-06-03T01:20:58.401 INFO:tasks.workunit.client.0.smithi032.stdout:8/154: rename dc/f1f to dc/d25/f35 0 2024-06-03T01:20:58.404 INFO:tasks.workunit.client.1.smithi044.stdout:5/609: rmdir d2/d11/d1e/d21/d24/d9e/db1/db7 0 2024-06-03T01:20:58.406 INFO:tasks.workunit.client.0.smithi032.stdout:3/212: dread de/f15 [0,4194304] 0 2024-06-03T01:20:58.407 INFO:tasks.workunit.client.0.smithi032.stdout:3/213: chown de/d29/d2e/d31/f1b 6115127 1 2024-06-03T01:20:58.407 INFO:tasks.workunit.client.0.smithi032.stdout:1/122: dwrite d18/f24 [0,4194304] 0 2024-06-03T01:20:58.408 INFO:tasks.workunit.client.0.smithi032.stdout:3/214: mknod de/c40 0 2024-06-03T01:20:58.410 INFO:tasks.workunit.client.1.smithi044.stdout:5/610: dread d2/fa [0,4194304] 0 2024-06-03T01:20:58.413 INFO:tasks.workunit.client.1.smithi044.stdout:5/611: unlink d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f4e 0 2024-06-03T01:20:58.415 INFO:tasks.workunit.client.1.smithi044.stdout:5/612: symlink d2/d11/d12/d2b/lc6 0 2024-06-03T01:20:58.415 INFO:tasks.workunit.client.1.smithi044.stdout:5/613: chown d2/d11/d12/d2b/d44/f5f 1 1 2024-06-03T01:20:58.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/715: rename d5/d55/d56/d5c/dbb to d5/d55/d56/d5c/d68/de5/dea 0 2024-06-03T01:20:58.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/716: creat d5/d55/d56/feb x:0 0 0 2024-06-03T01:20:58.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/717: creat d5/d55/d56/fec x:0 0 0 2024-06-03T01:20:58.417 INFO:tasks.workunit.client.1.smithi044.stdout:7/689: rename d0/d16/d23/c3a to d0/d3/d8/d95/d63/d7f/cdd 0 2024-06-03T01:20:58.419 INFO:tasks.workunit.client.1.smithi044.stdout:9/567: sync 2024-06-03T01:20:58.419 INFO:tasks.workunit.client.1.smithi044.stdout:6/840: sync 2024-06-03T01:20:58.419 INFO:tasks.workunit.client.1.smithi044.stdout:2/515: sync 2024-06-03T01:20:58.419 INFO:tasks.workunit.client.1.smithi044.stdout:2/516: write db/d1b/d2a/d52/f8b [1141233,75138] 0 2024-06-03T01:20:58.421 INFO:tasks.workunit.client.1.smithi044.stdout:9/568: write d6/dd/f87 [675588,121827] 0 2024-06-03T01:20:58.422 INFO:tasks.workunit.client.1.smithi044.stdout:9/569: truncate d6/dd/d51/db/fb4 736971 0 2024-06-03T01:20:58.422 INFO:tasks.workunit.client.1.smithi044.stdout:0/806: dwrite d3/f6e [0,4194304] 0 2024-06-03T01:20:58.422 INFO:tasks.workunit.client.1.smithi044.stdout:0/807: chown d3/d9/d36/f108 98149252 1 2024-06-03T01:20:58.424 INFO:tasks.workunit.client.0.smithi032.stdout:0/244: dwrite d1/f35 [4194304,4194304] 0 2024-06-03T01:20:58.425 INFO:tasks.workunit.client.1.smithi044.stdout:1/764: dwrite d0/d3/d20/fff [0,4194304] 0 2024-06-03T01:20:58.426 INFO:tasks.workunit.client.0.smithi032.stdout:1/123: dread d18/f1e [0,4194304] 0 2024-06-03T01:20:58.426 INFO:tasks.workunit.client.0.smithi032.stdout:1/124: readlink d18/l1d 0 2024-06-03T01:20:58.426 INFO:tasks.workunit.client.0.smithi032.stdout:7/204: read dc/d12/d22/f2e [24633,73948] 0 2024-06-03T01:20:58.426 INFO:tasks.workunit.client.0.smithi032.stdout:7/205: write dc/f30 [8507960,4231] 0 2024-06-03T01:20:58.426 INFO:tasks.workunit.client.0.smithi032.stdout:7/206: creat dc/d12/d2a/f43 x:0 0 0 2024-06-03T01:20:58.427 INFO:tasks.workunit.client.0.smithi032.stdout:7/207: write dc/d12/d1b/f34 [627011,82760] 0 2024-06-03T01:20:58.427 INFO:tasks.workunit.client.1.smithi044.stdout:5/614: rename d2/d11/d1e/d21/d24/d93/d9d/f15 to d2/d11/d12/d2b/d4a/d57/fc7 0 2024-06-03T01:20:58.429 INFO:tasks.workunit.client.0.smithi032.stdout:5/237: dwrite f4 [0,4194304] 0 2024-06-03T01:20:58.429 INFO:tasks.workunit.client.0.smithi032.stdout:5/238: fsync da/dd/df/d10/d16/d18/d35/f3a 0 2024-06-03T01:20:58.429 INFO:tasks.workunit.client.1.smithi044.stdout:1/765: dread d0/d3/d20/d40/d45/d4b/f90 [0,4194304] 0 2024-06-03T01:20:58.430 INFO:tasks.workunit.client.0.smithi032.stdout:9/306: dwrite d0/d23/d25/f52 [0,4194304] 0 2024-06-03T01:20:58.430 INFO:tasks.workunit.client.1.smithi044.stdout:4/720: dwrite d17/d1c/d75/d7a/db8/f57 [0,4194304] 0 2024-06-03T01:20:58.439 INFO:tasks.workunit.client.0.smithi032.stdout:5/239: dread da/f3e [0,4194304] 0 2024-06-03T01:20:58.440 INFO:tasks.workunit.client.1.smithi044.stdout:9/570: rename d6/dd/d51/d16/d5f/d8b/d8d/f8f to d6/dd/d51/d16/d4d/dad/fb6 0 2024-06-03T01:20:58.440 INFO:tasks.workunit.client.1.smithi044.stdout:9/571: truncate d6/dd/d51/d16/d4d/d55/f5b 381852 0 2024-06-03T01:20:58.441 INFO:tasks.workunit.client.1.smithi044.stdout:5/615: mknod d2/d11/d12/dad/db3/cc8 0 2024-06-03T01:20:58.442 INFO:tasks.workunit.client.0.smithi032.stdout:7/208: truncate dc/dd/fe 1435196 0 2024-06-03T01:20:58.442 INFO:tasks.workunit.client.0.smithi032.stdout:7/209: creat dc/d12/d2a/d2f/f44 x:0 0 0 2024-06-03T01:20:58.444 INFO:tasks.workunit.client.0.smithi032.stdout:8/155: getdents dc 0 2024-06-03T01:20:58.444 INFO:tasks.workunit.client.0.smithi032.stdout:8/156: write dc/d25/f35 [238224,73312] 0 2024-06-03T01:20:58.447 INFO:tasks.workunit.client.0.smithi032.stdout:0/245: rename d1/da/d10/c1d to d1/da/d28/c4c 0 2024-06-03T01:20:58.447 INFO:tasks.workunit.client.1.smithi044.stdout:5/616: dread d2/d11/d12/d2b/d4a/d57/f50 [0,4194304] 0 2024-06-03T01:20:58.449 INFO:tasks.workunit.client.0.smithi032.stdout:9/307: rmdir d0/d11 39 2024-06-03T01:20:58.449 INFO:tasks.workunit.client.0.smithi032.stdout:9/308: chown d0/c4d 239428163 1 2024-06-03T01:20:58.452 INFO:tasks.workunit.client.0.smithi032.stdout:1/125: getdents d18 0 2024-06-03T01:20:58.453 INFO:tasks.workunit.client.1.smithi044.stdout:0/808: dwrite d3/f6d [4194304,4194304] 0 2024-06-03T01:20:58.453 INFO:tasks.workunit.client.0.smithi032.stdout:5/240: rmdir da/dd/df/d10/d16/d18/d35 39 2024-06-03T01:20:58.454 INFO:tasks.workunit.client.1.smithi044.stdout:4/721: dwrite d17/d1c/d75/d7a/db8/d28/d2b/f2e [0,4194304] 0 2024-06-03T01:20:58.454 INFO:tasks.workunit.client.1.smithi044.stdout:4/722: dread - d17/d1c/d75/d7a/db8/d28/d2b/fe8 zero size 2024-06-03T01:20:58.454 INFO:tasks.workunit.client.1.smithi044.stdout:8/718: rmdir d5/d55/d56/d5c/d68 39 2024-06-03T01:20:58.455 INFO:tasks.workunit.client.0.smithi032.stdout:7/210: mknod dc/d12/d2a/c45 0 2024-06-03T01:20:58.456 INFO:tasks.workunit.client.1.smithi044.stdout:9/572: mkdir d6/dd/d51/d16/d27/d5e/db7 0 2024-06-03T01:20:58.456 INFO:tasks.workunit.client.0.smithi032.stdout:8/157: truncate dc/d14/f24 1244012 0 2024-06-03T01:20:58.457 INFO:tasks.workunit.client.0.smithi032.stdout:0/246: creat d1/da/f4d x:0 0 0 2024-06-03T01:20:58.458 INFO:tasks.workunit.client.0.smithi032.stdout:9/309: mknod d0/d23/d24/c69 0 2024-06-03T01:20:58.460 INFO:tasks.workunit.client.1.smithi044.stdout:0/809: truncate d3/f14 157110 0 2024-06-03T01:20:58.461 INFO:tasks.workunit.client.0.smithi032.stdout:1/126: creat d18/f25 x:0 0 0 2024-06-03T01:20:58.461 INFO:tasks.workunit.client.0.smithi032.stdout:5/241: truncate da/dd/df/d10/d16/d18/d35/f3a 168069 0 2024-06-03T01:20:58.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/211: rename dc/d12/d2a/c37 to dc/d12/d1b/c46 0 2024-06-03T01:20:58.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/212: creat dc/d12/d22/f47 x:0 0 0 2024-06-03T01:20:58.464 INFO:tasks.workunit.client.1.smithi044.stdout:8/719: symlink d5/d20/led 0 2024-06-03T01:20:58.464 INFO:tasks.workunit.client.0.smithi032.stdout:8/158: creat dc/d25/d28/d32/f36 x:0 0 0 2024-06-03T01:20:58.465 INFO:tasks.workunit.client.0.smithi032.stdout:0/247: mknod d1/d47/c4e 0 2024-06-03T01:20:58.465 INFO:tasks.workunit.client.1.smithi044.stdout:9/573: mknod d6/dd/d51/d16/d27/d5e/d71/d89/cb8 0 2024-06-03T01:20:58.466 INFO:tasks.workunit.client.0.smithi032.stdout:9/310: symlink d0/d11/d1d/d38/l6a 0 2024-06-03T01:20:58.467 INFO:tasks.workunit.client.1.smithi044.stdout:0/810: link d3/da6/d20/d25/f65 d3/d6b/d95/dae/dd1/f12c 0 2024-06-03T01:20:58.467 INFO:tasks.workunit.client.0.smithi032.stdout:5/242: creat da/dd/f48 x:0 0 0 2024-06-03T01:20:58.467 INFO:tasks.workunit.client.0.smithi032.stdout:5/243: chown da/dd/f2d 4 1 2024-06-03T01:20:58.468 INFO:tasks.workunit.client.0.smithi032.stdout:5/244: dread - da/dd/df/d10/f42 zero size 2024-06-03T01:20:58.469 INFO:tasks.workunit.client.1.smithi044.stdout:8/720: mkdir d5/d55/d56/d5c/dee 0 2024-06-03T01:20:58.469 INFO:tasks.workunit.client.1.smithi044.stdout:8/721: dread - d5/d55/d56/d5c/d68/de5/dea/d79/fd3 zero size 2024-06-03T01:20:58.470 INFO:tasks.workunit.client.0.smithi032.stdout:7/213: mknod dc/d12/c48 0 2024-06-03T01:20:58.470 INFO:tasks.workunit.client.0.smithi032.stdout:7/214: chown dc/d12/d24/l41 517645619 1 2024-06-03T01:20:58.470 INFO:tasks.workunit.client.0.smithi032.stdout:7/215: dread - dc/d12/f25 zero size 2024-06-03T01:20:58.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/127: dread f15 [0,4194304] 0 2024-06-03T01:20:58.470 INFO:tasks.workunit.client.0.smithi032.stdout:8/159: creat dc/de/f37 x:0 0 0 2024-06-03T01:20:58.471 INFO:tasks.workunit.client.0.smithi032.stdout:0/248: truncate d1/da/d28/f33 2986379 0 2024-06-03T01:20:58.471 INFO:tasks.workunit.client.0.smithi032.stdout:0/249: write d1/da/d10/f39 [12189,115296] 0 2024-06-03T01:20:58.471 INFO:tasks.workunit.client.1.smithi044.stdout:5/617: dwrite d2/d11/d1e/d21/d24/d93/d9d/f49 [0,4194304] 0 2024-06-03T01:20:58.471 INFO:tasks.workunit.client.1.smithi044.stdout:5/618: truncate d2/d11/d12/d2b/d4a/d57/f6b 761622 0 2024-06-03T01:20:58.471 INFO:tasks.workunit.client.1.smithi044.stdout:4/723: dwrite d17/d1c/d75/f62 [0,4194304] 0 2024-06-03T01:20:58.472 INFO:tasks.workunit.client.0.smithi032.stdout:9/311: mknod d0/d11/d1d/d38/c6b 0 2024-06-03T01:20:58.473 INFO:tasks.workunit.client.1.smithi044.stdout:9/574: rename d6/dd/d51/d16/d4d/d55/d68/fa4 to d6/dd/d51/d16/d5f/d8b/d8d/fb9 0 2024-06-03T01:20:58.473 INFO:tasks.workunit.client.0.smithi032.stdout:5/245: link da/f2a da/dd/df/d10/d16/d25/f49 0 2024-06-03T01:20:58.474 INFO:tasks.workunit.client.0.smithi032.stdout:1/128: stat d18/f22 0 2024-06-03T01:20:58.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/575: dread d6/dd/d51/d16/f1b [0,4194304] 0 2024-06-03T01:20:58.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/576: fsync d6/f65 0 2024-06-03T01:20:58.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/577: fdatasync d6/dd/d51/d46/f5c 0 2024-06-03T01:20:58.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/578: dread - d6/dd/d51/d46/f5c zero size 2024-06-03T01:20:58.476 INFO:tasks.workunit.client.0.smithi032.stdout:8/160: creat dc/d14/d31/f38 x:0 0 0 2024-06-03T01:20:58.476 INFO:tasks.workunit.client.1.smithi044.stdout:0/811: creat d3/d6b/d95/dae/dd1/f12d x:0 0 0 2024-06-03T01:20:58.476 INFO:tasks.workunit.client.0.smithi032.stdout:0/250: creat d1/da/d10/f4f x:0 0 0 2024-06-03T01:20:58.478 INFO:tasks.workunit.client.0.smithi032.stdout:9/312: rename d0/d11/c18 to d0/d11/d1d/d38/c6c 0 2024-06-03T01:20:58.479 INFO:tasks.workunit.client.0.smithi032.stdout:1/129: chown d18/l1d 10 1 2024-06-03T01:20:58.479 INFO:tasks.workunit.client.0.smithi032.stdout:1/130: write d18/f25 [954436,20076] 0 2024-06-03T01:20:58.481 INFO:tasks.workunit.client.0.smithi032.stdout:8/161: symlink dc/l39 0 2024-06-03T01:20:58.482 INFO:tasks.workunit.client.1.smithi044.stdout:8/722: creat d5/d55/d74/d82/fef x:0 0 0 2024-06-03T01:20:58.484 INFO:tasks.workunit.client.0.smithi032.stdout:0/251: unlink d1/da/d10/f39 0 2024-06-03T01:20:58.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/579: read d6/dd/d51/d3b/fab [2267303,31966] 0 2024-06-03T01:20:58.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/580: chown d6/dd/d51/d16/d4d/dad/fb6 4193 1 2024-06-03T01:20:58.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/581: dread - d6/dd/d51/d16/d27/d5e/d71/f78 zero size 2024-06-03T01:20:58.487 INFO:tasks.workunit.client.0.smithi032.stdout:8/162: readlink dc/de/l19 0 2024-06-03T01:20:58.487 INFO:tasks.workunit.client.0.smithi032.stdout:0/252: chown d1/da/d10/l22 90213389 1 2024-06-03T01:20:58.487 INFO:tasks.workunit.client.1.smithi044.stdout:4/724: mkdir d17/d1c/d75/d7a/db8/d6a/d84/df4 0 2024-06-03T01:20:58.487 INFO:tasks.workunit.client.1.smithi044.stdout:4/725: readlink d17/d1c/l25 0 2024-06-03T01:20:58.490 INFO:tasks.workunit.client.0.smithi032.stdout:1/131: link d18/l1d d18/l26 0 2024-06-03T01:20:58.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/579: sync 2024-06-03T01:20:58.493 INFO:tasks.workunit.client.1.smithi044.stdout:8/723: creat d5/d16/d81/de3/d98/ddd/ff0 x:0 0 0 2024-06-03T01:20:58.495 INFO:tasks.workunit.client.1.smithi044.stdout:4/726: link d17/d1c/d7e/lc5 d17/d1c/d75/d7a/db8/d6a/d84/lf5 0 2024-06-03T01:20:58.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/313: dread d0/d23/d24/f3b [0,4194304] 0 2024-06-03T01:20:58.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/314: creat d0/d11/d1d/d38/f6d x:0 0 0 2024-06-03T01:20:58.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/315: read - d0/f57 zero size 2024-06-03T01:20:58.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/132: mkdir d18/d27 0 2024-06-03T01:20:58.496 INFO:tasks.workunit.client.1.smithi044.stdout:4/727: read d17/f69 [105974,10918] 0 2024-06-03T01:20:58.496 INFO:tasks.workunit.client.1.smithi044.stdout:4/728: truncate d17/d1c/d75/d7a/db8/d35/fb4 1788828 0 2024-06-03T01:20:58.497 INFO:tasks.workunit.client.1.smithi044.stdout:3/580: unlink df/d13/f1a 0 2024-06-03T01:20:58.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/316: symlink d0/d23/d25/l6e 0 2024-06-03T01:20:58.499 INFO:tasks.workunit.client.0.smithi032.stdout:8/163: dread f1 [4194304,4194304] 0 2024-06-03T01:20:58.499 INFO:tasks.workunit.client.0.smithi032.stdout:8/164: write dc/d25/f35 [1084034,30520] 0 2024-06-03T01:20:58.500 INFO:tasks.workunit.client.1.smithi044.stdout:8/724: symlink d5/d16/d81/lf1 0 2024-06-03T01:20:58.500 INFO:tasks.workunit.client.1.smithi044.stdout:2/517: dwrite db/f1f [4194304,4194304] 0 2024-06-03T01:20:58.500 INFO:tasks.workunit.client.1.smithi044.stdout:2/518: fsync db/d1b/f22 0 2024-06-03T01:20:58.502 INFO:tasks.workunit.client.1.smithi044.stdout:4/729: getdents d17/d1c/d75/d7a/db8/d28/d2b/d8e 0 2024-06-03T01:20:58.502 INFO:tasks.workunit.client.1.smithi044.stdout:4/730: getdents d17/d1c/d75/d7a/db8/d8d/ded 0 2024-06-03T01:20:58.503 INFO:tasks.workunit.client.1.smithi044.stdout:4/731: stat d17/d1c/d75/d7a/db8/d59/l5b 0 2024-06-03T01:20:58.503 INFO:tasks.workunit.client.1.smithi044.stdout:3/581: truncate df/d13/d39/d3d/d50/f6f 2608602 0 2024-06-03T01:20:58.503 INFO:tasks.workunit.client.0.smithi032.stdout:8/165: unlink dc/de/f10 0 2024-06-03T01:20:58.504 INFO:tasks.workunit.client.1.smithi044.stdout:5/619: write d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [2088760,92363] 0 2024-06-03T01:20:58.508 INFO:tasks.workunit.client.1.smithi044.stdout:4/732: creat d17/d1c/d75/d7a/db8/d59/ff6 x:0 0 0 2024-06-03T01:20:58.509 INFO:tasks.workunit.client.1.smithi044.stdout:0/812: dwrite d3/d9/d55/fd4 [0,4194304] 0 2024-06-03T01:20:58.510 INFO:tasks.workunit.client.1.smithi044.stdout:1/766: dwrite d0/d3/d20/d40/d45/fea [0,4194304] 0 2024-06-03T01:20:58.510 INFO:tasks.workunit.client.1.smithi044.stdout:1/767: chown d0/d3/d20/d7/d13/d2d/l104 22374 1 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/216: dwrite f6 [0,4194304] 0 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/217: chown dc/d12/d22 55914828 1 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/218: write dc/d12/d1b/f2c [826013,59838] 0 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/219: dread - dc/f3d zero size 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/220: write dc/d12/d1b/f21 [773664,67614] 0 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/246: dwrite da/dd/df/f32 [0,4194304] 0 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/247: stat l5 0 2024-06-03T01:20:58.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/248: dread - da/dd/df/d10/d16/d18/f27 zero size 2024-06-03T01:20:58.512 INFO:tasks.workunit.client.1.smithi044.stdout:3/582: rename df/d13/f3b to df/d13/d39/d3d/d43/fc1 0 2024-06-03T01:20:58.513 INFO:tasks.workunit.client.1.smithi044.stdout:3/583: write df/f17 [10223609,29295] 0 2024-06-03T01:20:58.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/620: symlink d2/d11/d1e/lc9 0 2024-06-03T01:20:58.533 INFO:tasks.workunit.client.1.smithi044.stdout:5/621: truncate d2/d11/d12/d2b/f84 451317 0 2024-06-03T01:20:58.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/733: unlink d17/d1c/d70/d7d/l91 0 2024-06-03T01:20:58.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/249: mknod da/dd/df/d10/d16/d18/d35/d44/c4a 0 2024-06-03T01:20:58.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/250: dread - da/dd/df/d10/f42 zero size 2024-06-03T01:20:58.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/251: truncate da/dd/f29 778496 0 2024-06-03T01:20:58.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/252: chown da/dd/df/d10/d16/d25/l45 216639 1 2024-06-03T01:20:58.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/253: rename da/dd/df/d10/d16 to da/dd/df/d10/d16/d18/d4b 22 2024-06-03T01:20:58.537 INFO:tasks.workunit.client.1.smithi044.stdout:0/813: link d3/da6/d20/faf d3/d9/d55/d7b/f12e 0 2024-06-03T01:20:58.539 INFO:tasks.workunit.client.0.smithi032.stdout:7/221: rmdir dc/d12/d22 39 2024-06-03T01:20:58.540 INFO:tasks.workunit.client.1.smithi044.stdout:8/725: dwrite d5/d55/fad [0,4194304] 0 2024-06-03T01:20:58.541 INFO:tasks.workunit.client.1.smithi044.stdout:8/726: chown d5/d16/f3c 33827567 1 2024-06-03T01:20:58.541 INFO:tasks.workunit.client.1.smithi044.stdout:3/584: mkdir df/d1e/d2a/d9b/dc2 0 2024-06-03T01:20:58.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/222: getdents dc/d12 0 2024-06-03T01:20:58.543 INFO:tasks.workunit.client.0.smithi032.stdout:1/133: dwrite d18/f25 [0,4194304] 0 2024-06-03T01:20:58.543 INFO:tasks.workunit.client.0.smithi032.stdout:1/134: write fa [283481,126419] 0 2024-06-03T01:20:58.543 INFO:tasks.workunit.client.0.smithi032.stdout:9/317: dwrite d0/fe [0,4194304] 0 2024-06-03T01:20:58.544 INFO:tasks.workunit.client.1.smithi044.stdout:1/768: dwrite d0/d3/d20/d7/d13/fd8 [0,4194304] 0 2024-06-03T01:20:58.544 INFO:tasks.workunit.client.1.smithi044.stdout:5/622: getdents d2/d11/d1e/d21/d24/d9e/db1 0 2024-06-03T01:20:58.544 INFO:tasks.workunit.client.0.smithi032.stdout:7/223: unlink dc/dd/fe 0 2024-06-03T01:20:58.545 INFO:tasks.workunit.client.0.smithi032.stdout:1/135: chown d18/l21 14 1 2024-06-03T01:20:58.547 INFO:tasks.workunit.client.1.smithi044.stdout:2/519: dwrite db/d15/d49/d6c/f8c [0,4194304] 0 2024-06-03T01:20:58.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/734: rename d17/d1c/d75/d7a/db8/d8d/f9e to d17/d1c/d75/d7a/db8/d59/de5/ff7 0 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/224: symlink dc/d12/d24/d33/l49 0 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/225: chown dc/d12/d2a/c3f 1552154951 1 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/226: readlink l0 0 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/227: write dc/d12/d1b/f2c [892276,103111] 0 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/228: stat dc/d12/d1b/f36 0 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/229: chown dc/d12/d22 0 1 2024-06-03T01:20:58.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/230: truncate fa 2399955 0 2024-06-03T01:20:58.556 INFO:tasks.workunit.client.0.smithi032.stdout:9/318: dread d0/d11/f22 [0,4194304] 0 2024-06-03T01:20:58.557 INFO:tasks.workunit.client.1.smithi044.stdout:0/814: mknod d3/d6b/d95/dae/c12f 0 2024-06-03T01:20:58.558 INFO:tasks.workunit.client.0.smithi032.stdout:7/231: rename dc/d12/l31 to dc/d12/d24/d33/l4a 0 2024-06-03T01:20:58.564 INFO:tasks.workunit.client.0.smithi032.stdout:7/232: creat dc/d12/d24/d33/f4b x:0 0 0 2024-06-03T01:20:58.566 INFO:tasks.workunit.client.1.smithi044.stdout:1/769: symlink d0/d3/d20/d7/d13/d2d/d101/l10a 0 2024-06-03T01:20:58.568 INFO:tasks.workunit.client.0.smithi032.stdout:5/254: dwrite da/dd/fe [0,4194304] 0 2024-06-03T01:20:58.569 INFO:tasks.workunit.client.1.smithi044.stdout:5/623: creat d2/d11/d12/d2b/d4a/fca x:0 0 0 2024-06-03T01:20:58.569 INFO:tasks.workunit.client.0.smithi032.stdout:9/319: dread d0/ff [0,4194304] 0 2024-06-03T01:20:58.571 INFO:tasks.workunit.client.1.smithi044.stdout:4/735: dread d17/d1c/d75/d7a/db8/d59/fec [0,4194304] 0 2024-06-03T01:20:58.571 INFO:tasks.workunit.client.1.smithi044.stdout:4/736: truncate d17/d1c/d75/fe1 4536127 0 2024-06-03T01:20:58.572 INFO:tasks.workunit.client.1.smithi044.stdout:2/520: creat db/fa2 x:0 0 0 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:2/521: write db/dd/fe [3572792,101073] 0 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:2/522: fdatasync db/d15/d49/d6c/f8c 0 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:2/523: fsync db/d1b/d3f/f66 0 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:2/524: truncate db/d1b/d2a/d67/f96 1633171 0 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:2/525: chown db/d1b/d2a/f2b 3555 1 2024-06-03T01:20:58.573 INFO:tasks.workunit.client.1.smithi044.stdout:8/727: dwrite d5/d16/d1d/fdf [0,4194304] 0 2024-06-03T01:20:58.574 INFO:tasks.workunit.client.1.smithi044.stdout:1/770: mkdir d0/d3/d14/d81/d47/d10b 0 2024-06-03T01:20:58.576 INFO:tasks.workunit.client.0.smithi032.stdout:9/320: symlink d0/d23/d25/l6f 0 2024-06-03T01:20:58.577 INFO:tasks.workunit.client.0.smithi032.stdout:9/321: dread d0/d23/d24/f2a [0,4194304] 0 2024-06-03T01:20:58.579 INFO:tasks.workunit.client.1.smithi044.stdout:2/526: rename db/d1b/d2a/d67/f96 to db/d15/d49/d6c/fa3 0 2024-06-03T01:20:58.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/255: dread da/dd/df/f46 [0,4194304] 0 2024-06-03T01:20:58.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/256: write da/dd/df/d10/d16/d18/f1d [1868160,105505] 0 2024-06-03T01:20:58.583 INFO:tasks.workunit.client.1.smithi044.stdout:5/624: dread d2/d11/d12/d2b/f34 [0,4194304] 0 2024-06-03T01:20:58.585 INFO:tasks.workunit.client.1.smithi044.stdout:8/728: mknod d5/d55/d56/cf2 0 2024-06-03T01:20:58.585 INFO:tasks.workunit.client.1.smithi044.stdout:8/729: chown d5/d55/d74/f78 8 1 2024-06-03T01:20:58.586 INFO:tasks.workunit.client.0.smithi032.stdout:5/257: unlink da/dd/df/d10/c24 0 2024-06-03T01:20:58.588 INFO:tasks.workunit.client.1.smithi044.stdout:1/771: symlink d0/dd2/dfe/dbe/d9e/l10c 0 2024-06-03T01:20:58.588 INFO:tasks.workunit.client.0.smithi032.stdout:5/258: creat da/dd/df/d10/f4c x:0 0 0 2024-06-03T01:20:58.589 INFO:tasks.workunit.client.0.smithi032.stdout:5/259: write da/dd/df/f33 [1521200,7655] 0 2024-06-03T01:20:58.589 INFO:tasks.workunit.client.0.smithi032.stdout:5/260: creat da/dd/df/d10/f4d x:0 0 0 2024-06-03T01:20:58.589 INFO:tasks.workunit.client.0.smithi032.stdout:5/261: dread - da/dd/df/d10/f42 zero size 2024-06-03T01:20:58.589 INFO:tasks.workunit.client.1.smithi044.stdout:4/737: getdents d17/d1c/d75 0 2024-06-03T01:20:58.589 INFO:tasks.workunit.client.1.smithi044.stdout:4/738: fdatasync d17/d1c/d75/d7a/db8/d28/d2b/d8e/fc9 0 2024-06-03T01:20:58.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/527: creat db/d15/fa4 x:0 0 0 2024-06-03T01:20:58.591 INFO:tasks.workunit.client.1.smithi044.stdout:5/625: mknod d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/ccb 0 2024-06-03T01:20:58.592 INFO:tasks.workunit.client.1.smithi044.stdout:4/739: dread d17/d1c/d75/d7a/db8/d59/de5/ff7 [0,4194304] 0 2024-06-03T01:20:58.592 INFO:tasks.workunit.client.0.smithi032.stdout:9/322: dread d0/f5 [0,4194304] 0 2024-06-03T01:20:58.592 INFO:tasks.workunit.client.0.smithi032.stdout:0/253: truncate d1/da/d28/f33 2131092 0 2024-06-03T01:20:58.593 INFO:tasks.workunit.client.0.smithi032.stdout:5/262: getdents da/dd 0 2024-06-03T01:20:58.594 INFO:tasks.workunit.client.0.smithi032.stdout:5/263: chown da/dd/df/d10/d16/d18/d1a/d30/c31 1 1 2024-06-03T01:20:58.594 INFO:tasks.workunit.client.0.smithi032.stdout:5/264: fdatasync da/dd/df/d10/f4c 0 2024-06-03T01:20:58.594 INFO:tasks.workunit.client.0.smithi032.stdout:5/265: chown da/dd/df/d10/d16/d18/d35/d44 997 1 2024-06-03T01:20:58.595 INFO:tasks.workunit.client.0.smithi032.stdout:9/323: truncate d0/f19 3753428 0 2024-06-03T01:20:58.596 INFO:tasks.workunit.client.1.smithi044.stdout:2/528: mkdir db/d1b/d27/da5 0 2024-06-03T01:20:58.596 INFO:tasks.workunit.client.0.smithi032.stdout:0/254: mkdir d1/da/d28/d3b/d50 0 2024-06-03T01:20:58.596 INFO:tasks.workunit.client.0.smithi032.stdout:0/255: write d1/f17 [284814,94048] 0 2024-06-03T01:20:58.596 INFO:tasks.workunit.client.1.smithi044.stdout:5/626: dread d2/fa [0,4194304] 0 2024-06-03T01:20:58.597 INFO:tasks.workunit.client.0.smithi032.stdout:5/266: rmdir da/dd/df/d10/d16/d18 39 2024-06-03T01:20:58.597 INFO:tasks.workunit.client.0.smithi032.stdout:5/267: write da/f3e [1498508,43919] 0 2024-06-03T01:20:58.597 INFO:tasks.workunit.client.0.smithi032.stdout:7/233: dwrite dc/d12/d1b/f36 [0,4194304] 0 2024-06-03T01:20:58.597 INFO:tasks.workunit.client.0.smithi032.stdout:7/234: creat dc/d12/d24/d33/f4c x:0 0 0 2024-06-03T01:20:58.598 INFO:tasks.workunit.client.1.smithi044.stdout:8/730: dread d5/f25 [0,4194304] 0 2024-06-03T01:20:58.598 INFO:tasks.workunit.client.0.smithi032.stdout:9/324: mknod d0/d11/d1d/d4a/c70 0 2024-06-03T01:20:58.598 INFO:tasks.workunit.client.0.smithi032.stdout:9/325: write d0/d11/d1d/d38/f6d [346005,125593] 0 2024-06-03T01:20:58.599 INFO:tasks.workunit.client.1.smithi044.stdout:1/772: rename d0/d3/d20/d40 to d0/dd2/dfe/dbe/d9e/d10d 0 2024-06-03T01:20:58.605 INFO:tasks.workunit.client.0.smithi032.stdout:7/235: dread dc/d12/d1b/f34 [0,4194304] 0 2024-06-03T01:20:58.605 INFO:tasks.workunit.client.0.smithi032.stdout:1/136: dwrite d18/f24 [0,4194304] 0 2024-06-03T01:20:58.605 INFO:tasks.workunit.client.0.smithi032.stdout:7/236: fsync dc/f16 0 2024-06-03T01:20:58.605 INFO:tasks.workunit.client.0.smithi032.stdout:1/137: write f17 [4975524,118453] 0 2024-06-03T01:20:58.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/815: dwrite d3/d9/d36/fd7 [0,4194304] 0 2024-06-03T01:20:58.606 INFO:tasks.workunit.client.1.smithi044.stdout:3/585: dwrite df/d1e/d78/da4/fbf [0,4194304] 0 2024-06-03T01:20:58.606 INFO:tasks.workunit.client.0.smithi032.stdout:4/175: sync 2024-06-03T01:20:58.607 INFO:tasks.workunit.client.1.smithi044.stdout:2/529: link db/d15/d49/d6c/d72/d7a/d4e/c5a db/d15/d49/d6c/d72/d8f/ca6 0 2024-06-03T01:20:58.607 INFO:tasks.workunit.client.1.smithi044.stdout:2/530: truncate db/d15/d49/f8e 87329 0 2024-06-03T01:20:58.611 INFO:tasks.workunit.client.0.smithi032.stdout:5/268: dread da/dd/df/d10/d16/d25/f49 [0,4194304] 0 2024-06-03T01:20:58.611 INFO:tasks.workunit.client.0.smithi032.stdout:5/269: chown da/dd/df/d10/d16/d18/f1d 577 1 2024-06-03T01:20:58.611 INFO:tasks.workunit.client.0.smithi032.stdout:9/326: mkdir d0/d11/d1d/d4a/d71 0 2024-06-03T01:20:58.611 INFO:tasks.workunit.client.0.smithi032.stdout:9/327: creat d0/d23/d24/f72 x:0 0 0 2024-06-03T01:20:58.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/586: dread df/f19 [0,4194304] 0 2024-06-03T01:20:58.616 INFO:tasks.workunit.client.1.smithi044.stdout:5/627: dread d2/d11/d1e/d21/d24/f38 [0,4194304] 0 2024-06-03T01:20:58.619 INFO:tasks.workunit.client.0.smithi032.stdout:7/237: symlink dc/l4d 0 2024-06-03T01:20:58.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/176: rename d4/db/da/d2d/d27/f30 to d4/db/da/d2d/d27/f39 0 2024-06-03T01:20:58.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/177: read d4/f7 [148063,8701] 0 2024-06-03T01:20:58.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/178: chown d4/db/da/d1d/f33 898112 1 2024-06-03T01:20:58.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/270: rename da/dd/fe to da/dd/f4e 0 2024-06-03T01:20:58.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/816: rename d3/d6b/cda to d3/da6/dc1/dc2/df8/c130 0 2024-06-03T01:20:58.625 INFO:tasks.workunit.client.0.smithi032.stdout:7/238: mkdir dc/d12/d22/d42/d4e 0 2024-06-03T01:20:58.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/531: mkdir db/d1b/d5d/da7 0 2024-06-03T01:20:58.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/532: truncate db/d1b/d27/d79/fa1 144636 0 2024-06-03T01:20:58.626 INFO:tasks.workunit.client.1.smithi044.stdout:2/533: chown db/d15/d49/d6c/d72/d7a/l3d 52699 1 2024-06-03T01:20:58.626 INFO:tasks.workunit.client.1.smithi044.stdout:2/534: chown db/d15/d49/d6c/d72/d7a/d4e/d7e 13587749 1 2024-06-03T01:20:58.626 INFO:tasks.workunit.client.0.smithi032.stdout:1/138: symlink d18/l28 0 2024-06-03T01:20:58.626 INFO:tasks.workunit.client.1.smithi044.stdout:3/587: creat df/d1e/d2a/d9b/fc3 x:0 0 0 2024-06-03T01:20:58.627 INFO:tasks.workunit.client.0.smithi032.stdout:5/271: creat da/dd/df/d10/d16/d18/d35/d44/f4f x:0 0 0 2024-06-03T01:20:58.628 INFO:tasks.workunit.client.1.smithi044.stdout:5/628: mknod d2/d11/d3e/ccc 0 2024-06-03T01:20:58.628 INFO:tasks.workunit.client.1.smithi044.stdout:0/817: mknod d3/d9/d55/d7b/dfc/d10c/c131 0 2024-06-03T01:20:58.629 INFO:tasks.workunit.client.0.smithi032.stdout:7/239: link dc/l27 dc/d12/d1b/l4f 0 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/272: creat da/dd/df/d10/d16/d18/f50 x:0 0 0 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/273: write da/dd/df/d10/f42 [396563,128644] 0 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.0.smithi032.stdout:7/240: chown c3 1 1 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.1.smithi044.stdout:0/818: mknod d3/da6/d21/d94/da3/dad/c132 0 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.1.smithi044.stdout:3/588: symlink df/d13/d39/lc4 0 2024-06-03T01:20:58.634 INFO:tasks.workunit.client.1.smithi044.stdout:5/629: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/fcd x:0 0 0 2024-06-03T01:20:58.635 INFO:tasks.workunit.client.0.smithi032.stdout:5/274: mkdir da/dd/df/d10/d16/d18/d1a/d30/d51 0 2024-06-03T01:20:58.635 INFO:tasks.workunit.client.0.smithi032.stdout:5/275: truncate da/dd/df/d10/f4d 812434 0 2024-06-03T01:20:58.635 INFO:tasks.workunit.client.0.smithi032.stdout:5/276: dread - da/dd/df/d10/f4c zero size 2024-06-03T01:20:58.635 INFO:tasks.workunit.client.0.smithi032.stdout:1/139: unlink cd 0 2024-06-03T01:20:58.635 INFO:tasks.workunit.client.0.smithi032.stdout:4/179: write d4/db/da/f2f [1779231,78462] 0 2024-06-03T01:20:58.638 INFO:tasks.workunit.client.0.smithi032.stdout:4/180: truncate d4/db/da/d2d/d27/f2b 3802258 0 2024-06-03T01:20:58.638 INFO:tasks.workunit.client.0.smithi032.stdout:4/181: creat d4/f3a x:0 0 0 2024-06-03T01:20:58.640 INFO:tasks.workunit.client.0.smithi032.stdout:1/140: mkdir d18/d29 0 2024-06-03T01:20:58.642 INFO:tasks.workunit.client.1.smithi044.stdout:1/773: dwrite d0/d3/d14/dae/ff9 [0,4194304] 0 2024-06-03T01:20:58.644 INFO:tasks.workunit.client.0.smithi032.stdout:4/182: creat d4/db/da/d37/f3b x:0 0 0 2024-06-03T01:20:58.646 INFO:tasks.workunit.client.1.smithi044.stdout:2/535: dwrite db/d15/d49/d6c/d72/d7a/f81 [0,4194304] 0 2024-06-03T01:20:58.646 INFO:tasks.workunit.client.1.smithi044.stdout:4/740: dwrite d17/d1c/d75/d7a/db8/d59/f82 [0,4194304] 0 2024-06-03T01:20:58.647 INFO:tasks.workunit.client.1.smithi044.stdout:8/731: dwrite d5/d16/d81/de3/d98/ddd/fe1 [0,4194304] 0 2024-06-03T01:20:58.648 INFO:tasks.workunit.client.1.smithi044.stdout:5/630: write d2/d11/d1e/d21/f75 [3610692,19663] 0 2024-06-03T01:20:58.648 INFO:tasks.workunit.client.1.smithi044.stdout:1/774: rmdir d0/d3/d20/d7/ded 0 2024-06-03T01:20:58.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/277: write da/dd/f4e [4125987,109046] 0 2024-06-03T01:20:58.649 INFO:tasks.workunit.client.0.smithi032.stdout:9/328: dwrite d0/d11/d1d/d38/f3e [0,4194304] 0 2024-06-03T01:20:58.649 INFO:tasks.workunit.client.0.smithi032.stdout:5/278: stat da/dd/df/d10 0 2024-06-03T01:20:58.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/183: link d4/c34 d4/db/da/d1d/c3c 0 2024-06-03T01:20:58.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/184: write d4/db/da/fd [3415070,110673] 0 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/741: dread d17/d1c/d75/d7a/db8/d59/fb5 [0,4194304] 0 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/742: write d17/d1c/d75/d7a/db8/f4d [4039453,22711] 0 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/743: chown d17/d1c/d75/d7a/db8/d59/de5/ff7 159608605 1 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/744: fsync d17/d1c/d75/d7a/db3/fd9 0 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/745: chown d17/d1c/d75/d7a/db8/d59/f7b 552 1 2024-06-03T01:20:58.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/746: chown d17/d1c/d75/d7a/db8/c45 199 1 2024-06-03T01:20:58.654 INFO:tasks.workunit.client.0.smithi032.stdout:9/329: rename d0/l3f to d0/d11/l73 0 2024-06-03T01:20:58.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/279: creat da/dd/df/d10/d16/d18/d1a/d30/d51/f52 x:0 0 0 2024-06-03T01:20:58.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/280: dread - da/dd/df/d10/d16/d18/f50 zero size 2024-06-03T01:20:58.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/281: dread - da/dd/f48 zero size 2024-06-03T01:20:58.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/282: chown da/dd/df/d10/d16/d25/l45 94262472 1 2024-06-03T01:20:58.659 INFO:tasks.workunit.client.1.smithi044.stdout:5/631: mkdir d2/d11/d3e/dce 0 2024-06-03T01:20:58.660 INFO:tasks.workunit.client.1.smithi044.stdout:5/632: chown d2/d11/d12/d2b/d44/c41 1074384 1 2024-06-03T01:20:58.660 INFO:tasks.workunit.client.0.smithi032.stdout:9/330: symlink d0/d23/d24/d26/d41/l74 0 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:7/690: sync 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:6/841: sync 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/582: sync 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/583: fsync d6/dd/d51/d16/d27/f39 0 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:1/775: truncate d0/d3/d14/d81/f56 2322465 0 2024-06-03T01:20:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:1/776: chown d0/d3/d20/d7/d13/fde 1820551 1 2024-06-03T01:20:58.662 INFO:tasks.workunit.client.0.smithi032.stdout:7/241: dwrite dc/f2b [0,4194304] 0 2024-06-03T01:20:58.662 INFO:tasks.workunit.client.0.smithi032.stdout:5/283: mknod da/dd/df/d10/d16/d18/d35/d44/c53 0 2024-06-03T01:20:58.662 INFO:tasks.workunit.client.1.smithi044.stdout:9/584: write d6/dd/d45/f56 [28073,102635] 0 2024-06-03T01:20:58.663 INFO:tasks.workunit.client.1.smithi044.stdout:9/585: fdatasync d6/dd/d51/d46/f48 0 2024-06-03T01:20:58.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/589: dwrite df/f21 [0,4194304] 0 2024-06-03T01:20:58.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/590: creat df/d13/d39/d3d/fc5 x:0 0 0 2024-06-03T01:20:58.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/591: write df/f65 [199944,25702] 0 2024-06-03T01:20:58.663 INFO:tasks.workunit.client.1.smithi044.stdout:1/777: dread d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f90 [0,4194304] 0 2024-06-03T01:20:58.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/747: dread d17/d1c/d75/f62 [0,4194304] 0 2024-06-03T01:20:58.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/748: truncate d17/d1c/f3a 9264819 0 2024-06-03T01:20:58.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/749: write d17/d1c/d75/d7a/db8/db9/fc8 [812922,103082] 0 2024-06-03T01:20:58.667 INFO:tasks.workunit.client.0.smithi032.stdout:4/185: getdents d4/db 0 2024-06-03T01:20:58.667 INFO:tasks.workunit.client.1.smithi044.stdout:6/842: creat d2/d23/d55/d5f/d99/f127 x:0 0 0 2024-06-03T01:20:58.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/586: link d6/dd/f87 d6/dd/d51/d16/d27/d5e/d71/fba 0 2024-06-03T01:20:58.670 INFO:tasks.workunit.client.0.smithi032.stdout:9/331: symlink d0/d23/l75 0 2024-06-03T01:20:58.671 INFO:tasks.workunit.client.1.smithi044.stdout:0/819: rename d3/da6/dc1/dc2/df8/c130 to d3/da6/d21/c133 0 2024-06-03T01:20:58.671 INFO:tasks.workunit.client.1.smithi044.stdout:0/820: fsync d3/d6b/d95/db3/ffe 0 2024-06-03T01:20:58.671 INFO:tasks.workunit.client.1.smithi044.stdout:1/778: truncate d0/d3/d20/fff 1800833 0 2024-06-03T01:20:58.672 INFO:tasks.workunit.client.1.smithi044.stdout:1/779: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/c58 8853163 1 2024-06-03T01:20:58.672 INFO:tasks.workunit.client.1.smithi044.stdout:1/780: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/fc0 [468736,83043] 0 2024-06-03T01:20:58.676 INFO:tasks.workunit.client.0.smithi032.stdout:5/284: mknod da/dd/df/d10/d16/d18/d1a/c54 0 2024-06-03T01:20:58.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/691: write d0/d3/d8/d32/f90 [2128672,79358] 0 2024-06-03T01:20:58.679 INFO:tasks.workunit.client.0.smithi032.stdout:7/242: write f6 [4149421,118388] 0 2024-06-03T01:20:58.683 INFO:tasks.workunit.client.1.smithi044.stdout:9/587: read d6/dd/d51/d16/d5f/f80 [660280,77813] 0 2024-06-03T01:20:58.683 INFO:tasks.workunit.client.1.smithi044.stdout:9/588: fsync d6/dd/f14 0 2024-06-03T01:20:58.685 INFO:tasks.workunit.client.1.smithi044.stdout:6/843: creat d2/d23/d55/d5f/d99/dcc/f128 x:0 0 0 2024-06-03T01:20:58.687 INFO:tasks.workunit.client.1.smithi044.stdout:3/592: creat df/d13/d1b/fc6 x:0 0 0 2024-06-03T01:20:58.688 INFO:tasks.workunit.client.0.smithi032.stdout:1/141: getdents d18 0 2024-06-03T01:20:58.689 INFO:tasks.workunit.client.1.smithi044.stdout:4/750: write d17/d1c/d75/f4b [7940704,23728] 0 2024-06-03T01:20:58.695 INFO:tasks.workunit.client.1.smithi044.stdout:3/593: dread df/d13/d39/d3d/d6a/fa8 [0,4194304] 0 2024-06-03T01:20:58.695 INFO:tasks.workunit.client.1.smithi044.stdout:2/536: rename db/d15/d49/d6c/d72/d8f/ca6 to db/d15/d49/d6c/d72/d7a/ca8 0 2024-06-03T01:20:58.695 INFO:tasks.workunit.client.1.smithi044.stdout:2/537: write db/d1b/f36 [566756,106825] 0 2024-06-03T01:20:58.697 INFO:tasks.workunit.client.1.smithi044.stdout:0/821: rmdir d3/da6/d56/d62/de7 39 2024-06-03T01:20:58.699 INFO:tasks.workunit.client.1.smithi044.stdout:1/781: symlink d0/d3/d14/dae/l10e 0 2024-06-03T01:20:58.700 INFO:tasks.workunit.client.0.smithi032.stdout:1/142: rename d18/l26 to d18/l2a 0 2024-06-03T01:20:58.701 INFO:tasks.workunit.client.1.smithi044.stdout:7/692: mkdir d0/d3/d8/d95/d63/d89/da3/dde 0 2024-06-03T01:20:58.702 INFO:tasks.workunit.client.0.smithi032.stdout:4/186: dwrite d4/db/f36 [0,4194304] 0 2024-06-03T01:20:58.702 INFO:tasks.workunit.client.0.smithi032.stdout:4/187: readlink d4/l5 0 2024-06-03T01:20:58.703 INFO:tasks.workunit.client.1.smithi044.stdout:6/844: write d2/d23/d55/d5f/d99/f11f [4128308,62597] 0 2024-06-03T01:20:58.703 INFO:tasks.workunit.client.1.smithi044.stdout:6/845: truncate d2/d23/d55/d5f/d99/dcc/f128 759516 0 2024-06-03T01:20:58.704 INFO:tasks.workunit.client.1.smithi044.stdout:4/751: symlink d17/d1c/d75/d7a/db8/d6a/d84/df4/lf8 0 2024-06-03T01:20:58.704 INFO:tasks.workunit.client.1.smithi044.stdout:4/752: write d17/d1c/d75/d7a/db8/db9/fc8 [1735170,52167] 0 2024-06-03T01:20:58.704 INFO:tasks.workunit.client.1.smithi044.stdout:4/753: creat d17/d63/ff9 x:0 0 0 2024-06-03T01:20:58.708 INFO:tasks.workunit.client.1.smithi044.stdout:5/633: rename d2/d11/d12/d2b/d4a/d57/f50 to d2/d11/d1e/d21/d24/d9e/dbb/fcf 0 2024-06-03T01:20:58.708 INFO:tasks.workunit.client.1.smithi044.stdout:5/634: stat d2/d11/d3e/f59 0 2024-06-03T01:20:58.709 INFO:tasks.workunit.client.1.smithi044.stdout:2/538: creat db/d15/d49/d6c/fa9 x:0 0 0 2024-06-03T01:20:58.710 INFO:tasks.workunit.client.1.smithi044.stdout:0/822: write d3/da6/d20/d25/f7d [3857182,66563] 0 2024-06-03T01:20:58.710 INFO:tasks.workunit.client.0.smithi032.stdout:1/143: dread f17 [0,4194304] 0 2024-06-03T01:20:58.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/693: mkdir d0/d3/ddf 0 2024-06-03T01:20:58.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/694: dread - d0/d16/fc4 zero size 2024-06-03T01:20:58.717 INFO:tasks.workunit.client.1.smithi044.stdout:9/589: dwrite d6/f44 [0,4194304] 0 2024-06-03T01:20:58.717 INFO:tasks.workunit.client.0.smithi032.stdout:9/332: dwrite d0/d11/d1d/f1e [0,4194304] 0 2024-06-03T01:20:58.720 INFO:tasks.workunit.client.1.smithi044.stdout:4/754: write d17/d1c/d75/d7a/db8/d59/f82 [3665869,14919] 0 2024-06-03T01:20:58.720 INFO:tasks.workunit.client.0.smithi032.stdout:7/243: dwrite dc/d12/d1b/f21 [0,4194304] 0 2024-06-03T01:20:58.720 INFO:tasks.workunit.client.0.smithi032.stdout:7/244: fsync dc/d12/d1b/f34 0 2024-06-03T01:20:58.721 INFO:tasks.workunit.client.0.smithi032.stdout:7/245: readlink dc/l4d 0 2024-06-03T01:20:58.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/846: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/lb6 to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/l129 0 2024-06-03T01:20:58.727 INFO:tasks.workunit.client.0.smithi032.stdout:9/333: mkdir d0/d11/d76 0 2024-06-03T01:20:58.728 INFO:tasks.workunit.client.1.smithi044.stdout:2/539: creat db/d1b/d2a/d52/d54/faa x:0 0 0 2024-06-03T01:20:58.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/782: getdents d0/d3/d14/d81 0 2024-06-03T01:20:58.733 INFO:tasks.workunit.client.0.smithi032.stdout:7/246: rmdir dc/dd 39 2024-06-03T01:20:58.738 INFO:tasks.workunit.client.0.smithi032.stdout:9/334: dread d0/f5 [0,4194304] 0 2024-06-03T01:20:58.740 INFO:tasks.workunit.client.1.smithi044.stdout:1/783: link d0/d3/d20/d7/d13/d2d/fce d0/dd2/dfe/dc2/f10f 0 2024-06-03T01:20:58.740 INFO:tasks.workunit.client.1.smithi044.stdout:1/784: fdatasync d0/dd2/dfe/dbe/fc5 0 2024-06-03T01:20:58.741 INFO:tasks.workunit.client.0.smithi032.stdout:9/335: rename d0/d23/d24/d26/d41/l58 to d0/d23/l77 0 2024-06-03T01:20:58.743 INFO:tasks.workunit.client.0.smithi032.stdout:9/336: symlink d0/d11/l78 0 2024-06-03T01:20:58.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/590: mknod d6/dd/d51/db/cbb 0 2024-06-03T01:20:58.747 INFO:tasks.workunit.client.1.smithi044.stdout:1/785: rename d0/d3/d14/d81/f56 to d0/d3/d14/dae/f110 0 2024-06-03T01:20:58.747 INFO:tasks.workunit.client.1.smithi044.stdout:2/540: read db/d15/d49/d6c/d72/d7a/f8d [4081003,94236] 0 2024-06-03T01:20:58.748 INFO:tasks.workunit.client.1.smithi044.stdout:1/786: truncate d0/d3/d20/d7/d13/d2d/dcd/d108/f3a 1227022 0 2024-06-03T01:20:58.749 INFO:tasks.workunit.client.0.smithi032.stdout:4/188: dwrite d4/db/da/d2d/d27/f32 [0,4194304] 0 2024-06-03T01:20:58.749 INFO:tasks.workunit.client.0.smithi032.stdout:4/189: creat d4/db/da/f3d x:0 0 0 2024-06-03T01:20:58.749 INFO:tasks.workunit.client.0.smithi032.stdout:4/190: creat d4/db/da/d37/f3e x:0 0 0 2024-06-03T01:20:58.749 INFO:tasks.workunit.client.1.smithi044.stdout:9/591: mknod d6/dd/d51/d16/d27/d5e/d71/cbc 0 2024-06-03T01:20:58.751 INFO:tasks.workunit.client.1.smithi044.stdout:1/787: link d0/d3/d20/f54 d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/f111 0 2024-06-03T01:20:58.756 INFO:tasks.workunit.client.0.smithi032.stdout:9/337: write d0/f1 [1631872,40649] 0 2024-06-03T01:20:58.757 INFO:tasks.workunit.client.0.smithi032.stdout:9/338: mkdir d0/d11/d79 0 2024-06-03T01:20:58.757 INFO:tasks.workunit.client.0.smithi032.stdout:9/339: write d0/d11/d1d/d4a/f5d [183897,89641] 0 2024-06-03T01:20:58.763 INFO:tasks.workunit.client.0.smithi032.stdout:9/340: write d0/fe [3995590,102800] 0 2024-06-03T01:20:58.763 INFO:tasks.workunit.client.0.smithi032.stdout:9/341: chown d0/d23/d24/f2a 204613 1 2024-06-03T01:20:58.763 INFO:tasks.workunit.client.0.smithi032.stdout:9/342: rmdir d0/d11 39 2024-06-03T01:20:58.764 INFO:tasks.workunit.client.1.smithi044.stdout:9/592: write d6/dd/d51/d16/d27/f64 [369546,110331] 0 2024-06-03T01:20:58.764 INFO:tasks.workunit.client.1.smithi044.stdout:5/635: dwrite d2/d11/d1e/d2f/f64 [0,4194304] 0 2024-06-03T01:20:58.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/823: dwrite d3/f19 [0,4194304] 0 2024-06-03T01:20:58.765 INFO:tasks.workunit.client.1.smithi044.stdout:9/593: creat d6/dd/d51/d16/d4d/d55/d68/fbd x:0 0 0 2024-06-03T01:20:58.766 INFO:tasks.workunit.client.1.smithi044.stdout:5/636: mknod d2/d11/cd0 0 2024-06-03T01:20:58.766 INFO:tasks.workunit.client.1.smithi044.stdout:5/637: dread - d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/fcd zero size 2024-06-03T01:20:58.768 INFO:tasks.workunit.client.1.smithi044.stdout:0/824: link d3/d9/d55/d5a/dde/l128 d3/da6/d20/d25/dbf/d129/l134 0 2024-06-03T01:20:58.769 INFO:tasks.workunit.client.1.smithi044.stdout:4/755: dwrite d17/d1c/d75/d7a/db3/f96 [4194304,4194304] 0 2024-06-03T01:20:58.769 INFO:tasks.workunit.client.1.smithi044.stdout:4/756: fsync d17/d1c/d75/d7a/db3/f96 0 2024-06-03T01:20:58.770 INFO:tasks.workunit.client.0.smithi032.stdout:1/144: dwrite d18/f1a [0,4194304] 0 2024-06-03T01:20:58.770 INFO:tasks.workunit.client.1.smithi044.stdout:7/695: dwrite d0/d3/d8/d95/d63/d7f/fd3 [0,4194304] 0 2024-06-03T01:20:58.770 INFO:tasks.workunit.client.1.smithi044.stdout:7/696: dread - d0/d3/d8/d95/d63/d7f/f82 zero size 2024-06-03T01:20:58.771 INFO:tasks.workunit.client.1.smithi044.stdout:9/594: creat d6/dd/d51/d16/d27/d5e/d71/fbe x:0 0 0 2024-06-03T01:20:58.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/788: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/fd4 [0,4194304] 0 2024-06-03T01:20:58.771 INFO:tasks.workunit.client.0.smithi032.stdout:7/247: dwrite dc/d12/f25 [0,4194304] 0 2024-06-03T01:20:58.774 INFO:tasks.workunit.client.1.smithi044.stdout:5/638: mkdir d2/d11/d3e/dce/dd1 0 2024-06-03T01:20:58.775 INFO:tasks.workunit.client.0.smithi032.stdout:9/343: dread d0/ff [0,4194304] 0 2024-06-03T01:20:58.775 INFO:tasks.workunit.client.0.smithi032.stdout:9/344: creat d0/d11/d1d/d38/d4f/f7a x:0 0 0 2024-06-03T01:20:58.776 INFO:tasks.workunit.client.0.smithi032.stdout:4/191: getdents d4/db/da 0 2024-06-03T01:20:58.781 INFO:tasks.workunit.client.1.smithi044.stdout:4/757: creat d17/d1c/d7e/ffa x:0 0 0 2024-06-03T01:20:58.781 INFO:tasks.workunit.client.1.smithi044.stdout:4/758: fsync d17/d1c/d75/d7a/db8/d28/d2b/f4a 0 2024-06-03T01:20:58.781 INFO:tasks.workunit.client.0.smithi032.stdout:9/345: link d0/d11/d1d/d4a/c70 d0/d11/d79/c7b 0 2024-06-03T01:20:58.785 INFO:tasks.workunit.client.0.smithi032.stdout:1/145: truncate d18/f1a 2635832 0 2024-06-03T01:20:58.787 INFO:tasks.workunit.client.1.smithi044.stdout:7/697: creat d0/d3/da8/fe0 x:0 0 0 2024-06-03T01:20:58.789 INFO:tasks.workunit.client.1.smithi044.stdout:9/595: creat d6/dd/d51/d16/d4d/dad/fbf x:0 0 0 2024-06-03T01:20:58.789 INFO:tasks.workunit.client.1.smithi044.stdout:9/596: chown d6/la 28562 1 2024-06-03T01:20:58.791 INFO:tasks.workunit.client.0.smithi032.stdout:9/346: dread d0/d23/d25/f52 [0,4194304] 0 2024-06-03T01:20:58.791 INFO:tasks.workunit.client.0.smithi032.stdout:9/347: write d0/d23/d24/d26/f39 [4776774,67050] 0 2024-06-03T01:20:58.794 INFO:tasks.workunit.client.0.smithi032.stdout:4/192: dwrite d4/db/da/d1d/f33 [0,4194304] 0 2024-06-03T01:20:58.794 INFO:tasks.workunit.client.1.smithi044.stdout:1/789: mkdir d0/d3/d20/d7/d13/d112 0 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/759: write d17/d1c/d75/d7a/db8/d28/d2b/f1b [752394,130015] 0 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/760: truncate d17/d1c/d75/d7a/db8/d59/fb5 517072 0 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/761: readlink d17/d1c/d75/d7a/db8/d6a/l88 0 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/762: chown d17/d1c/d75/d7a/db8/d6a/d84/fef 3656 1 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/763: truncate d17/d1c/d75/d7a/db8/d28/f87 764984 0 2024-06-03T01:20:58.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/764: write d17/f69 [192811,126319] 0 2024-06-03T01:20:58.797 INFO:tasks.workunit.client.1.smithi044.stdout:5/639: rename d2/d11/d1e/d21/d24/d93/d9d/d28/d66/c98 to d2/d11/d1e/d21/d24/d9e/db1/cd2 0 2024-06-03T01:20:58.797 INFO:tasks.workunit.client.1.smithi044.stdout:5/640: chown d2/d11/d12/d2b/d4a/d57/fb4 249 1 2024-06-03T01:20:58.798 INFO:tasks.workunit.client.1.smithi044.stdout:9/597: write d6/dd/d51/d16/d61/f6c [3348193,124929] 0 2024-06-03T01:20:58.798 INFO:tasks.workunit.client.1.smithi044.stdout:9/598: write d6/dd/f36 [2720571,100391] 0 2024-06-03T01:20:58.802 INFO:tasks.workunit.client.0.smithi032.stdout:4/193: dread d4/db/da/f2f [0,4194304] 0 2024-06-03T01:20:58.802 INFO:tasks.workunit.client.0.smithi032.stdout:4/194: read - d4/f3a zero size 2024-06-03T01:20:58.802 INFO:tasks.workunit.client.0.smithi032.stdout:4/195: readlink d4/l5 0 2024-06-03T01:20:58.803 INFO:tasks.workunit.client.1.smithi044.stdout:7/698: creat d0/d3/d8/d95/d63/d89/da3/d9a/fe1 x:0 0 0 2024-06-03T01:20:58.805 INFO:tasks.workunit.client.1.smithi044.stdout:1/790: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f113 x:0 0 0 2024-06-03T01:20:58.805 INFO:tasks.workunit.client.1.smithi044.stdout:1/791: fsync d0/d3/d20/d7/fb1 0 2024-06-03T01:20:58.808 INFO:tasks.workunit.client.1.smithi044.stdout:4/765: mkdir d17/d1c/d70/dfb 0 2024-06-03T01:20:58.811 INFO:tasks.workunit.client.1.smithi044.stdout:5/641: unlink d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/fcd 0 2024-06-03T01:20:58.815 INFO:tasks.workunit.client.1.smithi044.stdout:7/699: symlink d0/le2 0 2024-06-03T01:20:58.818 INFO:tasks.workunit.client.1.smithi044.stdout:9/599: dread d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:58.818 INFO:tasks.workunit.client.1.smithi044.stdout:5/642: creat d2/d11/d1e/d21/d24/d93/d9d/d28/fd3 x:0 0 0 2024-06-03T01:20:58.820 INFO:tasks.workunit.client.1.smithi044.stdout:7/700: creat d0/d3/d8/d95/d63/d7f/d9c/fe3 x:0 0 0 2024-06-03T01:20:58.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/701: readlink d0/d16/d23/l4b 0 2024-06-03T01:20:58.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/702: read - d0/d3/d8/d95/d63/fc1 zero size 2024-06-03T01:20:58.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/703: write d0/f74 [5000415,108325] 0 2024-06-03T01:20:58.824 INFO:tasks.workunit.client.1.smithi044.stdout:9/600: unlink d6/dd/d51/d16/d27/f53 0 2024-06-03T01:20:58.824 INFO:tasks.workunit.client.1.smithi044.stdout:9/601: dread - d6/dd/d51/d16/d4d/dad/fbf zero size 2024-06-03T01:20:58.827 INFO:tasks.workunit.client.1.smithi044.stdout:5/643: creat d2/d11/d1e/d3b/da6/fd4 x:0 0 0 2024-06-03T01:20:58.827 INFO:tasks.workunit.client.1.smithi044.stdout:4/766: dread d17/d1c/d75/d7a/db8/d28/d2b/f21 [0,4194304] 0 2024-06-03T01:20:58.828 INFO:tasks.workunit.client.1.smithi044.stdout:7/704: mknod d0/d3/ce4 0 2024-06-03T01:20:58.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/767: link d17/d1c/d75/d7a/db8/l53 d17/d1c/d75/d7a/db8/d6a/lfc 0 2024-06-03T01:20:58.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/768: creat d17/d1c/d75/d7a/db3/ffd x:0 0 0 2024-06-03T01:20:58.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/769: creat d17/d1c/d75/d7a/db8/db9/ffe x:0 0 0 2024-06-03T01:20:58.832 INFO:tasks.workunit.client.1.smithi044.stdout:7/705: rmdir d0/d3/d8/d32/d33 39 2024-06-03T01:20:58.835 INFO:tasks.workunit.client.1.smithi044.stdout:4/770: mkdir d17/d1c/d70/dfb/dff 0 2024-06-03T01:20:58.835 INFO:tasks.workunit.client.1.smithi044.stdout:4/771: fsync d17/d1c/d75/d7a/db8/fa5 0 2024-06-03T01:20:58.835 INFO:tasks.workunit.client.1.smithi044.stdout:4/772: readlink d17/d1c/d9c/lad 0 2024-06-03T01:20:58.840 INFO:tasks.workunit.client.1.smithi044.stdout:9/602: dread d6/dd/d51/d16/f34 [0,4194304] 0 2024-06-03T01:20:58.840 INFO:tasks.workunit.client.1.smithi044.stdout:7/706: write d0/d3/d8/d32/d33/f67 [3417545,122236] 0 2024-06-03T01:20:58.840 INFO:tasks.workunit.client.1.smithi044.stdout:7/707: getdents d0/d3/d8/d95/d63/d7f/d9c/da4/dc9 0 2024-06-03T01:20:58.841 INFO:tasks.workunit.client.1.smithi044.stdout:1/792: dwrite d0/d3/d20/d7/d13/fb6 [0,4194304] 0 2024-06-03T01:20:58.841 INFO:tasks.workunit.client.1.smithi044.stdout:1/793: chown d0/dd2/dfe/dbe/le2 1004820 1 2024-06-03T01:20:58.842 INFO:tasks.workunit.client.0.smithi032.stdout:9/348: dwrite d0/f15 [0,4194304] 0 2024-06-03T01:20:58.842 INFO:tasks.workunit.client.1.smithi044.stdout:9/603: link d6/dd/d51/d16/f5a d6/dd/d51/d16/d27/d5e/db7/fc0 0 2024-06-03T01:20:58.843 INFO:tasks.workunit.client.0.smithi032.stdout:4/196: dwrite d4/f3a [0,4194304] 0 2024-06-03T01:20:58.845 INFO:tasks.workunit.client.0.smithi032.stdout:4/197: creat d4/db/da/f3f x:0 0 0 2024-06-03T01:20:58.845 INFO:tasks.workunit.client.0.smithi032.stdout:4/198: chown d4/db/da/d2d/d27/f39 2593092 1 2024-06-03T01:20:58.846 INFO:tasks.workunit.client.0.smithi032.stdout:9/349: getdents d0/d11/d1d/d38 0 2024-06-03T01:20:58.848 INFO:tasks.workunit.client.0.smithi032.stdout:4/199: rename d4/db/da/d2d/d27/f32 to d4/db/da/d37/f40 0 2024-06-03T01:20:58.848 INFO:tasks.workunit.client.0.smithi032.stdout:4/200: creat d4/f41 x:0 0 0 2024-06-03T01:20:58.849 INFO:tasks.workunit.client.1.smithi044.stdout:7/708: mkdir d0/d3/da8/de5 0 2024-06-03T01:20:58.851 INFO:tasks.workunit.client.0.smithi032.stdout:9/350: getdents d0/d11 0 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.0.smithi032.stdout:4/201: truncate d4/db/da/f1b 16610 0 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.0.smithi032.stdout:4/202: chown d4/db/da/d37 2099 1 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.0.smithi032.stdout:4/203: getdents d4/db 0 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.1.smithi044.stdout:8/732: sync 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.1.smithi044.stdout:7/709: dread d0/d16/f54 [0,4194304] 0 2024-06-03T01:20:58.852 INFO:tasks.workunit.client.1.smithi044.stdout:7/710: chown d0/d3/d8/d95/d63/d89/da3/d9a/lc6 41 1 2024-06-03T01:20:58.854 INFO:tasks.workunit.client.0.smithi032.stdout:4/204: link d4/db/da/d2d/c35 d4/db/da/d2d/c42 0 2024-06-03T01:20:58.856 INFO:tasks.workunit.client.0.smithi032.stdout:4/205: truncate d4/f7 4622464 0 2024-06-03T01:20:58.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/794: rename d0/dd2/dfe/ff0 to d0/d3/d20/d7/d2f/db7/f114 0 2024-06-03T01:20:58.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/795: readlink d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80/lcc 0 2024-06-03T01:20:58.857 INFO:tasks.workunit.client.1.smithi044.stdout:1/796: fsync d0/d3/d20/d7/d2f/db7/fc7 0 2024-06-03T01:20:58.858 INFO:tasks.workunit.client.1.smithi044.stdout:9/604: mkdir d6/dd/d51/d16/d27/d5e/d71/dc1 0 2024-06-03T01:20:58.860 INFO:tasks.workunit.client.1.smithi044.stdout:7/711: mkdir d0/d3/d8/d95/d63/de6 0 2024-06-03T01:20:58.864 INFO:tasks.workunit.client.1.smithi044.stdout:1/797: mkdir d0/dd2/dfe/d109/d115 0 2024-06-03T01:20:58.867 INFO:tasks.workunit.client.0.smithi032.stdout:9/351: dread d0/d11/d1d/f47 [0,4194304] 0 2024-06-03T01:20:58.869 INFO:tasks.workunit.client.1.smithi044.stdout:9/605: creat d6/dd/d51/d16/d4d/d55/fc2 x:0 0 0 2024-06-03T01:20:58.869 INFO:tasks.workunit.client.1.smithi044.stdout:9/606: truncate d6/dd/d51/d16/d27/d5e/d71/fbe 625537 0 2024-06-03T01:20:58.870 INFO:tasks.workunit.client.1.smithi044.stdout:9/607: read d6/dd/d51/d16/f1b [129011,53085] 0 2024-06-03T01:20:58.870 INFO:tasks.workunit.client.1.smithi044.stdout:1/798: truncate d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80/f92 2321391 0 2024-06-03T01:20:58.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/352: getdents d0/d23/d24 0 2024-06-03T01:20:58.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/353: chown d0/c4d 67728933 1 2024-06-03T01:20:58.876 INFO:tasks.workunit.client.1.smithi044.stdout:1/799: write d0/d3/d20/d7/d13/d2d/dcd/d108/f67 [4626,124792] 0 2024-06-03T01:20:58.877 INFO:tasks.workunit.client.0.smithi032.stdout:4/206: dwrite d4/db/da/f31 [0,4194304] 0 2024-06-03T01:20:58.878 INFO:tasks.workunit.client.0.smithi032.stdout:4/207: chown c3 2965 1 2024-06-03T01:20:58.878 INFO:tasks.workunit.client.0.smithi032.stdout:4/208: fsync d4/db/da/d2d/d27/f39 0 2024-06-03T01:20:58.878 INFO:tasks.workunit.client.0.smithi032.stdout:4/209: creat d4/f43 x:0 0 0 2024-06-03T01:20:58.878 INFO:tasks.workunit.client.0.smithi032.stdout:9/354: rmdir d0/d11/d1d 39 2024-06-03T01:20:58.879 INFO:tasks.workunit.client.1.smithi044.stdout:1/800: mkdir d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/d116 0 2024-06-03T01:20:58.879 INFO:tasks.workunit.client.1.smithi044.stdout:9/608: dread d6/dd/f36 [0,4194304] 0 2024-06-03T01:20:58.880 INFO:tasks.workunit.client.1.smithi044.stdout:8/733: dwrite d5/d16/f6e [0,4194304] 0 2024-06-03T01:20:58.881 INFO:tasks.workunit.client.1.smithi044.stdout:1/801: creat d0/d3/d14/d30/f117 x:0 0 0 2024-06-03T01:20:58.881 INFO:tasks.workunit.client.1.smithi044.stdout:1/802: chown d0/d3/d20/d7/d2f/c31 12 1 2024-06-03T01:20:58.883 INFO:tasks.workunit.client.0.smithi032.stdout:4/210: mkdir d4/db/da/d1d/d44 0 2024-06-03T01:20:58.883 INFO:tasks.workunit.client.0.smithi032.stdout:4/211: fsync d4/db/da/f31 0 2024-06-03T01:20:58.883 INFO:tasks.workunit.client.0.smithi032.stdout:4/212: truncate d4/db/da/f1b 496567 0 2024-06-03T01:20:58.886 INFO:tasks.workunit.client.0.smithi032.stdout:4/213: unlink d4/c20 0 2024-06-03T01:20:58.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/734: rename d5/d16/l1c to d5/d16/d1d/lf3 0 2024-06-03T01:20:58.887 INFO:tasks.workunit.client.0.smithi032.stdout:4/214: symlink d4/db/da/d37/l45 0 2024-06-03T01:20:58.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/803: link d0/dd2/dfe/dc2/cf8 d0/d3/d20/d7/d13/d112/c118 0 2024-06-03T01:20:58.889 INFO:tasks.workunit.client.1.smithi044.stdout:1/804: dread - d0/d3/d20/d7/d13/fab zero size 2024-06-03T01:20:58.891 INFO:tasks.workunit.client.0.smithi032.stdout:4/215: unlink d4/db/da/l17 0 2024-06-03T01:20:58.891 INFO:tasks.workunit.client.0.smithi032.stdout:4/216: write d4/db/da/f15 [530783,105734] 0 2024-06-03T01:20:58.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/735: stat d5/d16/d81/d9b/ld6 0 2024-06-03T01:20:58.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/736: write d5/d16/d81/de3/d98/fa7 [2063400,65827] 0 2024-06-03T01:20:58.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/737: read - d5/d20/d47/f91 zero size 2024-06-03T01:20:58.895 INFO:tasks.workunit.client.0.smithi032.stdout:4/217: unlink d4/db/da/d2d/d27/f39 0 2024-06-03T01:20:58.895 INFO:tasks.workunit.client.0.smithi032.stdout:4/218: write d4/f3a [4629227,130372] 0 2024-06-03T01:20:58.895 INFO:tasks.workunit.client.0.smithi032.stdout:4/219: rename d4/db/da to d4/db/da/d37/d46 22 2024-06-03T01:20:58.898 INFO:tasks.workunit.client.1.smithi044.stdout:1/805: rename d0/d3/d20/d7/ca1 to d0/c119 0 2024-06-03T01:20:58.901 INFO:tasks.workunit.client.1.smithi044.stdout:3/594: sync 2024-06-03T01:20:58.902 INFO:tasks.workunit.client.1.smithi044.stdout:9/609: dwrite d6/dd/d51/d16/f7b [0,4194304] 0 2024-06-03T01:20:58.903 INFO:tasks.workunit.client.0.smithi032.stdout:9/355: dwrite d0/d11/d1d/f47 [0,4194304] 0 2024-06-03T01:20:58.904 INFO:tasks.workunit.client.0.smithi032.stdout:4/220: write d4/db/da/d1d/f21 [4060052,79057] 0 2024-06-03T01:20:58.904 INFO:tasks.workunit.client.0.smithi032.stdout:4/221: stat d4/db/da/c12 0 2024-06-03T01:20:58.904 INFO:tasks.workunit.client.1.smithi044.stdout:3/595: link df/d1e/d3a/d84/lb3 df/d1e/d2a/d9b/dc2/lc7 0 2024-06-03T01:20:58.904 INFO:tasks.workunit.client.1.smithi044.stdout:3/596: creat df/d1e/d2a/fc8 x:0 0 0 2024-06-03T01:20:58.905 INFO:tasks.workunit.client.0.smithi032.stdout:9/356: creat d0/d11/d76/f7c x:0 0 0 2024-06-03T01:20:58.905 INFO:tasks.workunit.client.0.smithi032.stdout:4/222: dread d4/db/ff [0,4194304] 0 2024-06-03T01:20:58.907 INFO:tasks.workunit.client.0.smithi032.stdout:4/223: rmdir d4/db/da/d2d 39 2024-06-03T01:20:58.910 INFO:tasks.workunit.client.1.smithi044.stdout:9/610: mknod d6/dd/d45/cc3 0 2024-06-03T01:20:58.910 INFO:tasks.workunit.client.1.smithi044.stdout:9/611: chown d6/dd/d45/f81 315 1 2024-06-03T01:20:58.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/612: creat d6/dd/d51/d16/d27/d5e/d71/dc1/fc4 x:0 0 0 2024-06-03T01:20:58.914 INFO:tasks.workunit.client.1.smithi044.stdout:3/597: dread df/d13/d39/d3d/f44 [0,4194304] 0 2024-06-03T01:20:58.914 INFO:tasks.workunit.client.1.smithi044.stdout:3/598: chown df/f47 303671214 1 2024-06-03T01:20:58.914 INFO:tasks.workunit.client.1.smithi044.stdout:3/599: write df/d1e/d2a/fc8 [448774,29261] 0 2024-06-03T01:20:58.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/613: link d6/dd/f87 d6/dd/d51/d16/d4d/fc5 0 2024-06-03T01:20:58.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/600: write df/d13/d6d/d9a/faa [4883042,91526] 0 2024-06-03T01:20:58.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/601: write df/d13/d39/d3d/d43/d71/f94 [220931,93032] 0 2024-06-03T01:20:58.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/602: truncate df/d1e/f79 470547 0 2024-06-03T01:20:58.916 INFO:tasks.workunit.client.1.smithi044.stdout:9/614: mknod d6/dd/d51/d16/cc6 0 2024-06-03T01:20:58.919 INFO:tasks.workunit.client.1.smithi044.stdout:8/738: dwrite d5/d20/d33/f3f [4194304,4194304] 0 2024-06-03T01:20:58.922 INFO:tasks.workunit.client.1.smithi044.stdout:1/806: dwrite d0/dd2/f60 [0,4194304] 0 2024-06-03T01:20:58.923 INFO:tasks.workunit.client.0.smithi032.stdout:9/357: dwrite d0/d11/d1d/d38/d4f/f7a [0,4194304] 0 2024-06-03T01:20:58.923 INFO:tasks.workunit.client.0.smithi032.stdout:4/224: truncate d4/db/da/f31 2857392 0 2024-06-03T01:20:58.923 INFO:tasks.workunit.client.0.smithi032.stdout:4/225: dread - d4/db/da/f3d zero size 2024-06-03T01:20:58.923 INFO:tasks.workunit.client.1.smithi044.stdout:3/603: creat df/fc9 x:0 0 0 2024-06-03T01:20:58.924 INFO:tasks.workunit.client.1.smithi044.stdout:3/604: dread - df/d1e/d2a/d70/fa3 zero size 2024-06-03T01:20:58.924 INFO:tasks.workunit.client.1.smithi044.stdout:3/605: write df/d13/d1b/fc6 [368829,46341] 0 2024-06-03T01:20:58.926 INFO:tasks.workunit.client.1.smithi044.stdout:9/615: rmdir d6/dd/d51/d16/d27/d5e/d71/d89 39 2024-06-03T01:20:58.926 INFO:tasks.workunit.client.1.smithi044.stdout:9/616: stat d6/dd/d51/d16/c2d 0 2024-06-03T01:20:58.926 INFO:tasks.workunit.client.1.smithi044.stdout:9/617: fsync d6/dd/d51/db/f32 0 2024-06-03T01:20:58.932 INFO:tasks.workunit.client.1.smithi044.stdout:8/739: unlink d5/d16/d81/de3/d58/d89/fa6 0 2024-06-03T01:20:58.932 INFO:tasks.workunit.client.1.smithi044.stdout:1/807: getdents d0/dd2/dfe/d109 0 2024-06-03T01:20:58.932 INFO:tasks.workunit.client.1.smithi044.stdout:1/808: chown d0/d3/d14/dae/ff9 4 1 2024-06-03T01:20:58.932 INFO:tasks.workunit.client.1.smithi044.stdout:1/809: fdatasync d0/dd2/dfe/dc2/f10f 0 2024-06-03T01:20:58.938 INFO:tasks.workunit.client.1.smithi044.stdout:1/810: truncate d0/dd2/f57 2891437 0 2024-06-03T01:20:58.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/358: symlink d0/d11/l7d 0 2024-06-03T01:20:58.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/359: chown d0/d11/d1d/l42 1646 1 2024-06-03T01:20:58.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/360: fdatasync d0/d23/d24/f55 0 2024-06-03T01:20:58.940 INFO:tasks.workunit.client.0.smithi032.stdout:9/361: write d0/d11/d1d/f54 [666985,73875] 0 2024-06-03T01:20:58.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/362: creat d0/d11/d1d/f7e x:0 0 0 2024-06-03T01:20:58.942 INFO:tasks.workunit.client.1.smithi044.stdout:1/811: unlink d0/dd2/f78 0 2024-06-03T01:20:58.943 INFO:tasks.workunit.client.0.smithi032.stdout:9/363: creat d0/d11/d1d/d4a/f7f x:0 0 0 2024-06-03T01:20:58.943 INFO:tasks.workunit.client.0.smithi032.stdout:9/364: truncate d0/d11/d76/f7c 213283 0 2024-06-03T01:20:58.943 INFO:tasks.workunit.client.1.smithi044.stdout:1/812: read d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/fc0 [93755,25989] 0 2024-06-03T01:20:58.945 INFO:tasks.workunit.client.0.smithi032.stdout:6/208: sync 2024-06-03T01:20:58.948 INFO:tasks.workunit.client.0.smithi032.stdout:6/209: mknod d8/d11/d20/d36/c43 0 2024-06-03T01:20:58.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/210: creat d8/d11/d20/f44 x:0 0 0 2024-06-03T01:20:58.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/211: creat d8/d11/d20/d31/f45 x:0 0 0 2024-06-03T01:20:58.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/212: fdatasync d8/d11/f17 0 2024-06-03T01:20:58.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/213: chown d8/fb 3008 1 2024-06-03T01:20:58.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/214: write d8/d11/d20/d2b/f3c [513352,111554] 0 2024-06-03T01:20:58.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/215: write d8/d11/d20/d31/f45 [21121,110528] 0 2024-06-03T01:20:58.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/216: creat d8/d11/d20/d2b/d39/d19/f46 x:0 0 0 2024-06-03T01:20:58.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/217: readlink d8/l10 0 2024-06-03T01:20:58.952 INFO:tasks.workunit.client.0.smithi032.stdout:6/218: rmdir d8/d11/d20/d3b 39 2024-06-03T01:20:58.954 INFO:tasks.workunit.client.1.smithi044.stdout:9/618: dwrite d6/dd/d51/d16/d4d/d55/fc2 [0,4194304] 0 2024-06-03T01:20:58.955 INFO:tasks.workunit.client.0.smithi032.stdout:4/226: dwrite d4/db/da/d1d/f33 [0,4194304] 0 2024-06-03T01:20:58.955 INFO:tasks.workunit.client.1.smithi044.stdout:8/740: dwrite d5/d55/d56/d5c/fae [0,4194304] 0 2024-06-03T01:20:58.956 INFO:tasks.workunit.client.0.smithi032.stdout:4/227: symlink d4/db/da/d37/l47 0 2024-06-03T01:20:58.960 INFO:tasks.workunit.client.0.smithi032.stdout:4/228: link d4/f41 d4/db/da/d37/f48 0 2024-06-03T01:20:58.960 INFO:tasks.workunit.client.0.smithi032.stdout:4/229: fdatasync d4/f41 0 2024-06-03T01:20:58.960 INFO:tasks.workunit.client.0.smithi032.stdout:4/230: fsync d4/f43 0 2024-06-03T01:20:58.960 INFO:tasks.workunit.client.0.smithi032.stdout:4/231: creat d4/db/da/f49 x:0 0 0 2024-06-03T01:20:58.965 INFO:tasks.workunit.client.1.smithi044.stdout:8/741: creat d5/d16/d81/de3/d58/dd5/ff4 x:0 0 0 2024-06-03T01:20:58.965 INFO:tasks.workunit.client.1.smithi044.stdout:8/742: stat d5/d16/d81/de3/d58/d89/dbf/f80 0 2024-06-03T01:20:58.965 INFO:tasks.workunit.client.1.smithi044.stdout:1/813: dwrite d0/fba [0,4194304] 0 2024-06-03T01:20:58.965 INFO:tasks.workunit.client.1.smithi044.stdout:1/814: stat d0/d3/d20/d7/d13/f3d 0 2024-06-03T01:20:58.967 INFO:tasks.workunit.client.1.smithi044.stdout:8/743: mkdir d5/d20/df5 0 2024-06-03T01:20:58.967 INFO:tasks.workunit.client.1.smithi044.stdout:8/744: readlink d5/d55/d74/d82/l61 0 2024-06-03T01:20:58.969 INFO:tasks.workunit.client.0.smithi032.stdout:4/232: dread d4/db/da/d37/f40 [0,4194304] 0 2024-06-03T01:20:58.970 INFO:tasks.workunit.client.1.smithi044.stdout:8/745: dread d5/d55/d56/d5c/f72 [0,4194304] 0 2024-06-03T01:20:58.970 INFO:tasks.workunit.client.0.smithi032.stdout:4/233: mknod d4/db/da/d37/c4a 0 2024-06-03T01:20:58.971 INFO:tasks.workunit.client.0.smithi032.stdout:4/234: readlink d4/db/da/d37/l47 0 2024-06-03T01:20:58.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/746: symlink d5/d16/d1d/d32/lf6 0 2024-06-03T01:20:58.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/747: creat d5/d55/ff7 x:0 0 0 2024-06-03T01:20:58.972 INFO:tasks.workunit.client.0.smithi032.stdout:4/235: unlink d4/db/da/fd 0 2024-06-03T01:20:58.973 INFO:tasks.workunit.client.1.smithi044.stdout:1/815: link d0/d3/d20/d7/d13/f96 d0/dd2/f11a 0 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/816: write d0/d3/d20/d7/d2f/db7/f114 [616289,130161] 0 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:9/619: dwrite d6/dd/d51/db/f15 [0,4194304] 0 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/817: read d0/d3/d14/f8d [36503,3998] 0 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/818: dread - d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f113 zero size 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/819: chown d0/dd2/dfe/dbe/d9e/d10d/d45/l9b 0 1 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/820: chown d0/d3/d20/def 4738187 1 2024-06-03T01:20:58.974 INFO:tasks.workunit.client.1.smithi044.stdout:1/821: chown d0/d3/d14/d30/c39 0 1 2024-06-03T01:20:58.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/236: rmdir d4/db/da/d1d/d44 0 2024-06-03T01:20:58.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/237: read - d4/db/da/f3f zero size 2024-06-03T01:20:58.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/238: read d4/db/da/d1d/f21 [968362,103171] 0 2024-06-03T01:20:58.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/239: symlink d4/db/l4b 0 2024-06-03T01:20:58.976 INFO:tasks.workunit.client.1.smithi044.stdout:8/748: rename d5/d20/d47/d64/dd0/dca to d5/d55/d56/d5c/d68/de5/dea/d79/df8 0 2024-06-03T01:20:58.977 INFO:tasks.workunit.client.1.smithi044.stdout:6/847: sync 2024-06-03T01:20:58.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/620: truncate d6/dd/d51/d16/d4d/d55/f5b 71811 0 2024-06-03T01:20:58.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/621: fdatasync d6/dd/d51/d16/d4d/d55/fc2 0 2024-06-03T01:20:58.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/622: creat d6/dd/d51/d16/d27/d5e/d71/dc1/fc7 x:0 0 0 2024-06-03T01:20:58.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/623: read - d6/dd/d51/d16/d27/d5e/d71/dc1/fc7 zero size 2024-06-03T01:20:58.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/624: truncate d6/dd/d51/d3b/f5d 507049 0 2024-06-03T01:20:58.982 INFO:tasks.workunit.client.0.smithi032.stdout:4/240: write d4/db/da/d37/f40 [2519374,27701] 0 2024-06-03T01:20:58.983 INFO:tasks.workunit.client.1.smithi044.stdout:1/822: dread d0/d3/d20/d7/f17 [0,4194304] 0 2024-06-03T01:20:58.985 INFO:tasks.workunit.client.1.smithi044.stdout:6/848: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/cb7 to d2/d23/d30/d1e/d38/d6c/c12a 0 2024-06-03T01:20:58.988 INFO:tasks.workunit.client.0.smithi032.stdout:4/241: write d4/db/da/f31 [2228272,115067] 0 2024-06-03T01:20:58.989 INFO:tasks.workunit.client.1.smithi044.stdout:9/625: dread d6/dd/d51/d16/d4d/d55/fc2 [0,4194304] 0 2024-06-03T01:20:58.989 INFO:tasks.workunit.client.1.smithi044.stdout:9/626: creat d6/dd/d51/d16/d4d/d55/d68/fc8 x:0 0 0 2024-06-03T01:20:58.990 INFO:tasks.workunit.client.0.smithi032.stdout:4/242: creat d4/db/da/d1d/f4c x:0 0 0 2024-06-03T01:20:58.991 INFO:tasks.workunit.client.1.smithi044.stdout:6/849: unlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/db1/l129 0 2024-06-03T01:20:58.991 INFO:tasks.workunit.client.1.smithi044.stdout:6/850: readlink d2/d23/d30/d1e/d38/lb3 0 2024-06-03T01:20:58.991 INFO:tasks.workunit.client.1.smithi044.stdout:6/851: write d2/d23/d30/d1e/d38/d77/f8e [4369098,44387] 0 2024-06-03T01:20:58.992 INFO:tasks.workunit.client.0.smithi032.stdout:4/243: mknod d4/db/da/d2d/c4d 0 2024-06-03T01:20:58.993 INFO:tasks.workunit.client.0.smithi032.stdout:4/244: rmdir d4/db 39 2024-06-03T01:20:59.005 INFO:tasks.workunit.client.1.smithi044.stdout:1/823: dwrite d0/d3/d14/f38 [8388608,4194304] 0 2024-06-03T01:20:59.007 INFO:tasks.workunit.client.1.smithi044.stdout:1/824: mkdir d0/dd2/dfe/dbe/d9e/d11b 0 2024-06-03T01:20:59.007 INFO:tasks.workunit.client.1.smithi044.stdout:1/825: stat d0/dd2/dfe/dbe/d9e/d10d/fdd 0 2024-06-03T01:20:59.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/627: dwrite d6/dd/d51/d16/d4d/d55/d68/fbd [0,4194304] 0 2024-06-03T01:20:59.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/628: write d6/dd/d51/d46/f88 [15958,72786] 0 2024-06-03T01:20:59.012 INFO:tasks.workunit.client.1.smithi044.stdout:9/629: creat d6/dd/d51/d3b/fc9 x:0 0 0 2024-06-03T01:20:59.013 INFO:tasks.workunit.client.1.smithi044.stdout:9/630: chown d6/dd/d51/d16/d27/ca9 7780187 1 2024-06-03T01:20:59.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/826: read d0/d3/d20/d7/d13/d2d/dcd/d108/f26 [2000179,129237] 0 2024-06-03T01:20:59.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/827: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/fb0 [412024,84881] 0 2024-06-03T01:20:59.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/828: creat d0/d3/d20/d7/d13/d2d/d101/f11c x:0 0 0 2024-06-03T01:20:59.019 INFO:tasks.workunit.client.1.smithi044.stdout:6/852: dwrite d2/d23/d55/d5f/db2/d3f/f90 [0,4194304] 0 2024-06-03T01:20:59.020 INFO:tasks.workunit.client.1.smithi044.stdout:1/829: mknod d0/d3/d14/dae/c11d 0 2024-06-03T01:20:59.023 INFO:tasks.workunit.client.1.smithi044.stdout:6/853: mknod d2/d23/d44/c12b 0 2024-06-03T01:20:59.025 INFO:tasks.workunit.client.1.smithi044.stdout:1/830: truncate d0/dd2/dfe/dbe/d9e/d10d/fdd 1611602 0 2024-06-03T01:20:59.025 INFO:tasks.workunit.client.1.smithi044.stdout:1/831: dread - d0/d3/d20/d7/d13/d2d/dcd/d108/d64/ff6 zero size 2024-06-03T01:20:59.029 INFO:tasks.workunit.client.1.smithi044.stdout:9/631: dwrite d6/dd/d51/d16/d27/d5e/d71/dc1/fc7 [0,4194304] 0 2024-06-03T01:20:59.037 INFO:tasks.workunit.client.1.smithi044.stdout:6/854: write d2/d23/f35 [2272464,92931] 0 2024-06-03T01:20:59.041 INFO:tasks.workunit.client.1.smithi044.stdout:9/632: getdents d6/dd/d51/d16/d5f/d79 0 2024-06-03T01:20:59.043 INFO:tasks.workunit.client.1.smithi044.stdout:9/633: unlink d6/dd/d51/d46/l83 0 2024-06-03T01:20:59.044 INFO:tasks.workunit.client.1.smithi044.stdout:9/634: rename d6/dd/d51/d16/d27/c2e to d6/dd/d51/d16/d4d/d55/cca 0 2024-06-03T01:20:59.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/832: dwrite d0/d3/d14/dae/f110 [0,4194304] 0 2024-06-03T01:20:59.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/833: readlink d0/dd2/dfe/dbe/le4 0 2024-06-03T01:20:59.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/834: creat d0/dd2/dfe/f11e x:0 0 0 2024-06-03T01:20:59.048 INFO:tasks.workunit.client.1.smithi044.stdout:9/635: rename d6/l1e to d6/dd/d51/d16/d4d/d55/lcb 0 2024-06-03T01:20:59.048 INFO:tasks.workunit.client.1.smithi044.stdout:9/636: stat d6/dd/d51/d16/d27/d5e 0 2024-06-03T01:20:59.049 INFO:tasks.workunit.client.1.smithi044.stdout:5/644: sync 2024-06-03T01:20:59.050 INFO:tasks.workunit.client.1.smithi044.stdout:4/773: sync 2024-06-03T01:20:59.050 INFO:tasks.workunit.client.1.smithi044.stdout:0/825: sync 2024-06-03T01:20:59.050 INFO:tasks.workunit.client.1.smithi044.stdout:0/826: dread - d3/d9/f126 zero size 2024-06-03T01:20:59.050 INFO:tasks.workunit.client.1.smithi044.stdout:2/541: sync 2024-06-03T01:20:59.051 INFO:tasks.workunit.client.1.smithi044.stdout:9/637: mknod d6/dd/d51/d16/d4d/d55/d68/ccc 0 2024-06-03T01:20:59.051 INFO:tasks.workunit.client.1.smithi044.stdout:9/638: write d6/dd/d51/d16/d4d/d67/fb1 [717715,41302] 0 2024-06-03T01:20:59.052 INFO:tasks.workunit.client.1.smithi044.stdout:5/645: symlink d2/d11/d12/d2b/d4a/ld5 0 2024-06-03T01:20:59.052 INFO:tasks.workunit.client.1.smithi044.stdout:9/639: dread d6/dd/d51/d46/f48 [0,4194304] 0 2024-06-03T01:20:59.053 INFO:tasks.workunit.client.1.smithi044.stdout:9/640: fsync d6/dd/d51/d16/f1b 0 2024-06-03T01:20:59.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/774: creat d17/d1c/de0/f100 x:0 0 0 2024-06-03T01:20:59.054 INFO:tasks.workunit.client.1.smithi044.stdout:0/827: creat d3/d6b/d95/f135 x:0 0 0 2024-06-03T01:20:59.054 INFO:tasks.workunit.client.1.smithi044.stdout:0/828: chown d3/da6/dc1 65324 1 2024-06-03T01:20:59.055 INFO:tasks.workunit.client.1.smithi044.stdout:5/646: link d2/d11/d1e/d21/d24/d93/d9d/d28/f2c d2/d11/d1e/d3b/da6/fd6 0 2024-06-03T01:20:59.055 INFO:tasks.workunit.client.1.smithi044.stdout:1/835: dread d0/d3/d20/d7/d13/fb6 [0,4194304] 0 2024-06-03T01:20:59.056 INFO:tasks.workunit.client.1.smithi044.stdout:1/836: fsync d0/d3/d14/dae/ff9 0 2024-06-03T01:20:59.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/641: mkdir d6/dd/d51/d16/d4d/dad/dcd 0 2024-06-03T01:20:59.059 INFO:tasks.workunit.client.1.smithi044.stdout:2/542: dread db/d1b/d27/f63 [0,4194304] 0 2024-06-03T01:20:59.061 INFO:tasks.workunit.client.1.smithi044.stdout:5/647: creat d2/d11/d3e/fd7 x:0 0 0 2024-06-03T01:20:59.061 INFO:tasks.workunit.client.1.smithi044.stdout:5/648: chown d2/d11/d12/d2b/d44/l83 3 1 2024-06-03T01:20:59.061 INFO:tasks.workunit.client.1.smithi044.stdout:3/606: sync 2024-06-03T01:20:59.061 INFO:tasks.workunit.client.1.smithi044.stdout:7/712: sync 2024-06-03T01:20:59.062 INFO:tasks.workunit.client.1.smithi044.stdout:8/749: sync 2024-06-03T01:20:59.062 INFO:tasks.workunit.client.1.smithi044.stdout:9/642: mkdir d6/dd/d51/dce 0 2024-06-03T01:20:59.064 INFO:tasks.workunit.client.1.smithi044.stdout:8/750: mkdir d5/dd4/df9 0 2024-06-03T01:20:59.065 INFO:tasks.workunit.client.1.smithi044.stdout:9/643: truncate d6/dd/d45/f56 673533 0 2024-06-03T01:20:59.065 INFO:tasks.workunit.client.1.smithi044.stdout:1/837: dread d0/d3/d20/d7/d13/d2d/dcd/d108/f26 [0,4194304] 0 2024-06-03T01:20:59.065 INFO:tasks.workunit.client.1.smithi044.stdout:6/855: dwrite d2/d23/d30/d1e/d38/f72 [0,4194304] 0 2024-06-03T01:20:59.065 INFO:tasks.workunit.client.1.smithi044.stdout:1/838: truncate d0/dd2/dfe/dc2/fd1 26221 0 2024-06-03T01:20:59.067 INFO:tasks.workunit.client.1.smithi044.stdout:8/751: truncate d5/d16/d81/de3/f48 30336 0 2024-06-03T01:20:59.069 INFO:tasks.workunit.client.1.smithi044.stdout:1/839: mknod d0/d3/d20/d7/d13/d2d/dcd/c11f 0 2024-06-03T01:20:59.070 INFO:tasks.workunit.client.1.smithi044.stdout:8/752: mknod d5/d16/d81/de3/d58/d89/dbf/dd2/cfa 0 2024-06-03T01:20:59.073 INFO:tasks.workunit.client.1.smithi044.stdout:1/840: symlink d0/dd2/dfe/dbe/d9e/d10d/d45/l120 0 2024-06-03T01:20:59.073 INFO:tasks.workunit.client.1.smithi044.stdout:1/841: fsync d0/d3/d20/d7/d13/fab 0 2024-06-03T01:20:59.073 INFO:tasks.workunit.client.1.smithi044.stdout:8/753: creat d5/d55/d56/d5c/d68/de5/ffb x:0 0 0 2024-06-03T01:20:59.073 INFO:tasks.workunit.client.1.smithi044.stdout:8/754: stat d5/d16/d81/de3/f35 0 2024-06-03T01:20:59.076 INFO:tasks.workunit.client.1.smithi044.stdout:1/842: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d8c/f121 x:0 0 0 2024-06-03T01:20:59.076 INFO:tasks.workunit.client.1.smithi044.stdout:1/843: chown d0/dd2/dfe/dbe/d9e/d10d/f88 497963303 1 2024-06-03T01:20:59.082 INFO:tasks.workunit.client.1.smithi044.stdout:2/543: dread db/d1b/d5d/f86 [0,4194304] 0 2024-06-03T01:20:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:58 smithi032 ceph-mon[29360]: pgmap v4: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:20:59.087 INFO:tasks.workunit.client.1.smithi044.stdout:2/544: rmdir db/d1b/d2a/d52/d54 39 2024-06-03T01:20:59.087 INFO:tasks.workunit.client.1.smithi044.stdout:2/545: rename db/d15/d49/d6c to db/d15/d49/d6c/d72/d7a/dab 22 2024-06-03T01:20:59.088 INFO:tasks.workunit.client.1.smithi044.stdout:8/755: dread d5/d16/d81/de3/d58/f84 [0,4194304] 0 2024-06-03T01:20:59.088 INFO:tasks.workunit.client.1.smithi044.stdout:4/775: dwrite d17/d1c/d75/d7a/db3/fdc [0,4194304] 0 2024-06-03T01:20:59.088 INFO:tasks.workunit.client.1.smithi044.stdout:4/776: write d17/d1c/d75/d7a/db8/fca [140260,56750] 0 2024-06-03T01:20:59.090 INFO:tasks.workunit.client.1.smithi044.stdout:8/756: truncate d5/d55/d56/f8a 637930 0 2024-06-03T01:20:59.105 INFO:tasks.workunit.client.1.smithi044.stdout:2/546: read db/f26 [609119,53920] 0 2024-06-03T01:20:59.106 INFO:tasks.workunit.client.1.smithi044.stdout:2/547: chown db/d1b/d2a/d52/d54/f56 9 1 2024-06-03T01:20:59.117 INFO:tasks.workunit.client.0.smithi032.stdout:2/219: sync 2024-06-03T01:20:59.119 INFO:tasks.workunit.client.0.smithi032.stdout:2/220: rename d5/l15 to d5/da/d1a/d2a/d36/l44 0 2024-06-03T01:20:59.119 INFO:tasks.workunit.client.0.smithi032.stdout:2/221: write d5/da/f2c [1163011,27844] 0 2024-06-03T01:20:59.119 INFO:tasks.workunit.client.0.smithi032.stdout:2/222: chown d5/da/d1a/d2a/d36/d3f 0 1 2024-06-03T01:20:59.120 INFO:tasks.workunit.client.0.smithi032.stdout:2/223: rename d5/da/d1a/c3a to d5/da/d2e/c45 0 2024-06-03T01:20:59.121 INFO:tasks.workunit.client.1.smithi044.stdout:8/757: dread d5/d20/d33/f63 [0,4194304] 0 2024-06-03T01:20:59.122 INFO:tasks.workunit.client.1.smithi044.stdout:8/758: mkdir d5/d16/d1d/d32/dfc 0 2024-06-03T01:20:59.122 INFO:tasks.workunit.client.1.smithi044.stdout:8/759: dread - d5/d55/d74/d82/fef zero size 2024-06-03T01:20:59.126 INFO:tasks.workunit.client.1.smithi044.stdout:0/829: dwrite d3/da6/d21/f26 [4194304,4194304] 0 2024-06-03T01:20:59.127 INFO:tasks.workunit.client.0.smithi032.stdout:2/224: write d5/f1c [3261633,64226] 0 2024-06-03T01:20:59.129 INFO:tasks.workunit.client.1.smithi044.stdout:7/713: dwrite d0/d3/d8/f14 [4194304,4194304] 0 2024-06-03T01:20:59.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/644: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:59.134 INFO:tasks.workunit.client.1.smithi044.stdout:9/645: chown d6/dd/d51/d16/d61/ca1 45574967 1 2024-06-03T01:20:59.135 INFO:tasks.workunit.client.1.smithi044.stdout:7/714: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/le7 0 2024-06-03T01:20:59.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/607: dwrite df/d1e/f45 [0,4194304] 0 2024-06-03T01:20:59.135 INFO:tasks.workunit.client.1.smithi044.stdout:0/830: rmdir d3/d6b 39 2024-06-03T01:20:59.135 INFO:tasks.workunit.client.1.smithi044.stdout:3/608: chown df/d1e/d2a/d9b 23637518 1 2024-06-03T01:20:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:4/777: dwrite d17/d1c/f49 [0,4194304] 0 2024-06-03T01:20:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:2/548: dwrite db/d1b/f99 [0,4194304] 0 2024-06-03T01:20:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:5/649: dwrite d2/d11/d12/d2b/f34 [0,4194304] 0 2024-06-03T01:20:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:1/844: dwrite d0/dd2/f57 [0,4194304] 0 2024-06-03T01:20:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:6/856: dwrite d2/d23/d55/d5f/f59 [0,4194304] 0 2024-06-03T01:20:59.138 INFO:tasks.workunit.client.1.smithi044.stdout:0/831: creat d3/d9/d55/d5a/d29/d58/f136 x:0 0 0 2024-06-03T01:20:59.142 INFO:tasks.workunit.client.0.smithi032.stdout:2/225: dread d5/da/f2c [0,4194304] 0 2024-06-03T01:20:59.142 INFO:tasks.workunit.client.1.smithi044.stdout:9/646: write d6/f11 [2533948,116506] 0 2024-06-03T01:20:59.143 INFO:tasks.workunit.client.0.smithi032.stdout:2/226: mkdir d5/d38/d46 0 2024-06-03T01:20:59.145 INFO:tasks.workunit.client.0.smithi032.stdout:2/227: link d5/c40 d5/da/d1a/c47 0 2024-06-03T01:20:59.145 INFO:tasks.workunit.client.0.smithi032.stdout:2/228: write d5/f3c [30284,87075] 0 2024-06-03T01:20:59.145 INFO:tasks.workunit.client.0.smithi032.stdout:2/229: write d5/f3c [841345,52060] 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:3/609: write df/d1e/f37 [3817819,120419] 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:3/610: creat df/d1e/d78/da4/fca x:0 0 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:3/611: chown df/d1e/d2a/f89 7121 1 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:4/778: truncate d17/d1c/d75/d7a/db8/d28/d2b/f34 343376 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:8/760: dwrite d5/d16/d1d/f42 [0,4194304] 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:8/761: truncate d5/d55/d74/db4/dba/fc3 295708 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:2/549: creat db/d15/d49/d82/fac x:0 0 0 2024-06-03T01:20:59.146 INFO:tasks.workunit.client.1.smithi044.stdout:2/550: chown db/d1b/d2a/d2f/d4c/l6d 422775651 1 2024-06-03T01:20:59.147 INFO:tasks.workunit.client.0.smithi032.stdout:2/230: symlink d5/da/d1a/d2a/d36/l48 0 2024-06-03T01:20:59.147 INFO:tasks.workunit.client.0.smithi032.stdout:2/231: fsync d5/fe 0 2024-06-03T01:20:59.147 INFO:tasks.workunit.client.1.smithi044.stdout:7/715: write d0/d3/d30/f73 [4002543,59467] 0 2024-06-03T01:20:59.148 INFO:tasks.workunit.client.1.smithi044.stdout:5/650: unlink d2/d11/d1e/d2f/f6e 0 2024-06-03T01:20:59.149 INFO:tasks.workunit.client.1.smithi044.stdout:0/832: write d3/d9/f13 [1157745,98102] 0 2024-06-03T01:20:59.149 INFO:tasks.workunit.client.1.smithi044.stdout:0/833: truncate d3/d6b/d95/dae/dd1/f12d 1003829 0 2024-06-03T01:20:59.149 INFO:tasks.workunit.client.1.smithi044.stdout:0/834: fsync d3/d6b/de8/d115/f11b 0 2024-06-03T01:20:59.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/551: dread db/dd/f60 [0,4194304] 0 2024-06-03T01:20:59.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/552: stat db/d15/d49/d6c 0 2024-06-03T01:20:59.154 INFO:tasks.workunit.client.1.smithi044.stdout:9/647: dread d6/dd/d51/d16/d5f/f7c [0,4194304] 0 2024-06-03T01:20:59.157 INFO:tasks.workunit.client.1.smithi044.stdout:1/845: creat d0/dd2/dfe/dbe/d9e/d11b/f122 x:0 0 0 2024-06-03T01:20:59.157 INFO:tasks.workunit.client.1.smithi044.stdout:3/612: dread df/d13/d39/d3d/d50/f6f [0,4194304] 0 2024-06-03T01:20:59.157 INFO:tasks.workunit.client.1.smithi044.stdout:1/846: fsync d0/d3/d14/dae/ff9 0 2024-06-03T01:20:59.160 INFO:tasks.workunit.client.1.smithi044.stdout:5/651: dread d2/d11/d1e/d21/d24/d9e/dbb/fcf [0,4194304] 0 2024-06-03T01:20:59.161 INFO:tasks.workunit.client.1.smithi044.stdout:5/652: read - d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f61 zero size 2024-06-03T01:20:59.161 INFO:tasks.workunit.client.1.smithi044.stdout:4/779: unlink d17/f31 0 2024-06-03T01:20:59.162 INFO:tasks.workunit.client.1.smithi044.stdout:8/762: unlink d5/d16/d81/de3/d98/le0 0 2024-06-03T01:20:59.162 INFO:tasks.workunit.client.1.smithi044.stdout:8/763: truncate d5/fe4 210930 0 2024-06-03T01:20:59.163 INFO:tasks.workunit.client.1.smithi044.stdout:0/835: dwrite d3/d6b/d95/fb2 [0,4194304] 0 2024-06-03T01:20:59.163 INFO:tasks.workunit.client.1.smithi044.stdout:7/716: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8 0 2024-06-03T01:20:59.165 INFO:tasks.workunit.client.0.smithi032.stdout:2/232: write d5/da/f39 [4004020,81418] 0 2024-06-03T01:20:59.165 INFO:tasks.workunit.client.0.smithi032.stdout:2/233: chown d5/da/d2e/f3d 1 1 2024-06-03T01:20:59.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/553: mknod db/d15/d49/d6c/d72/d8f/cad 0 2024-06-03T01:20:59.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/554: creat db/d1b/d2a/d67/d53/d9b/fae x:0 0 0 2024-06-03T01:20:59.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/555: stat db/d1b/d3f 0 2024-06-03T01:20:59.166 INFO:tasks.workunit.client.0.smithi032.stdout:2/234: mkdir d5/da/d1a/d2a/d49 0 2024-06-03T01:20:59.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/613: symlink df/d1e/lcb 0 2024-06-03T01:20:59.168 INFO:tasks.workunit.client.0.smithi032.stdout:2/235: symlink d5/da/d1a/d2a/d36/l4a 0 2024-06-03T01:20:59.171 INFO:tasks.workunit.client.1.smithi044.stdout:1/847: link d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/fbd d0/dd2/dfe/dbe/d9e/d10d/dcb/f123 0 2024-06-03T01:20:59.171 INFO:tasks.workunit.client.1.smithi044.stdout:1/848: chown d0/dd2/dfe/d109 1604 1 2024-06-03T01:20:59.171 INFO:tasks.workunit.client.1.smithi044.stdout:1/849: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80 61932149 1 2024-06-03T01:20:59.172 INFO:tasks.workunit.client.0.smithi032.stdout:2/236: dread d5/da/f2c [0,4194304] 0 2024-06-03T01:20:59.172 INFO:tasks.workunit.client.0.smithi032.stdout:2/237: creat d5/da/d1a/f4b x:0 0 0 2024-06-03T01:20:59.173 INFO:tasks.workunit.client.0.smithi032.stdout:2/238: symlink d5/da/d1a/d2a/d49/l4c 0 2024-06-03T01:20:59.174 INFO:tasks.workunit.client.0.smithi032.stdout:2/239: chown d5/da/d1a/d2a/d36/l44 260548 1 2024-06-03T01:20:59.174 INFO:tasks.workunit.client.1.smithi044.stdout:6/857: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/dba to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/d12c 0 2024-06-03T01:20:59.174 INFO:tasks.workunit.client.1.smithi044.stdout:4/780: rename d17 to d17/d1c/d7e/d101 22 2024-06-03T01:20:59.175 INFO:tasks.workunit.client.0.smithi032.stdout:2/240: symlink d5/da/d1a/d2a/d36/l4d 0 2024-06-03T01:20:59.175 INFO:tasks.workunit.client.1.smithi044.stdout:5/653: creat d2/d11/fd8 x:0 0 0 2024-06-03T01:20:59.175 INFO:tasks.workunit.client.1.smithi044.stdout:5/654: write d2/fa [2450783,84342] 0 2024-06-03T01:20:59.175 INFO:tasks.workunit.client.1.smithi044.stdout:5/655: write d2/d11/d12/f2d [5135917,5850] 0 2024-06-03T01:20:59.176 INFO:tasks.workunit.client.1.smithi044.stdout:8/764: read d5/d16/d81/f83 [3068408,72318] 0 2024-06-03T01:20:59.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/241: symlink d5/da/d1a/d2a/l4e 0 2024-06-03T01:20:59.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/242: fdatasync d5/f23 0 2024-06-03T01:20:59.178 INFO:tasks.workunit.client.0.smithi032.stdout:2/243: rename d5/f13 to d5/da/d1a/d2a/d36/d3f/f4f 0 2024-06-03T01:20:59.178 INFO:tasks.workunit.client.0.smithi032.stdout:2/244: readlink d5/l2d 0 2024-06-03T01:20:59.179 INFO:tasks.workunit.client.1.smithi044.stdout:7/717: write d0/d3/d8/d95/d63/d7f/d9c/f50 [1839131,34374] 0 2024-06-03T01:20:59.180 INFO:tasks.workunit.client.1.smithi044.stdout:7/718: chown d0/d3/d8/f14 1409 1 2024-06-03T01:20:59.180 INFO:tasks.workunit.client.0.smithi032.stdout:2/245: rename d5/da/d1a/f31 to d5/da/d1a/d2a/f50 0 2024-06-03T01:20:59.181 INFO:tasks.workunit.client.1.smithi044.stdout:0/836: link d3/d9/d55/c61 d3/da6/d21/d94/da3/dd8/c137 0 2024-06-03T01:20:59.182 INFO:tasks.workunit.client.0.smithi032.stdout:2/246: creat d5/da/d1a/d2a/d36/d3f/f51 x:0 0 0 2024-06-03T01:20:59.184 INFO:tasks.workunit.client.0.smithi032.stdout:2/247: mkdir d5/d38/d52 0 2024-06-03T01:20:59.184 INFO:tasks.workunit.client.1.smithi044.stdout:7/719: dread d0/d3/f7 [0,4194304] 0 2024-06-03T01:20:59.185 INFO:tasks.workunit.client.1.smithi044.stdout:2/556: rename db/dd/l90 to db/d15/d49/d6c/d72/d7a/d4e/laf 0 2024-06-03T01:20:59.186 INFO:tasks.workunit.client.1.smithi044.stdout:4/781: mknod d17/d1c/d75/d7a/db8/d28/c102 0 2024-06-03T01:20:59.187 INFO:tasks.workunit.client.1.smithi044.stdout:8/765: mkdir d5/d20/d33/dfd 0 2024-06-03T01:20:59.189 INFO:tasks.workunit.client.1.smithi044.stdout:6/858: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/f3c [0,4194304] 0 2024-06-03T01:20:59.189 INFO:tasks.workunit.client.1.smithi044.stdout:6/859: truncate d2/d23/d44/fc6 431273 0 2024-06-03T01:20:59.196 INFO:tasks.workunit.client.1.smithi044.stdout:0/837: unlink d3/f18 0 2024-06-03T01:20:59.196 INFO:tasks.workunit.client.1.smithi044.stdout:0/838: chown d3/d9/d102/f11f 229310315 1 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:7/720: symlink d0/d3/ddf/le9 0 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:7/721: dread - d0/d3/da8/fd0 zero size 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:1/850: dwrite d0/d3/d20/d7/fb1 [0,4194304] 0 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:1/851: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d6f 27598 1 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:7/722: creat d0/d16/fea x:0 0 0 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/648: dwrite d6/dd/d51/d16/d27/f39 [0,4194304] 0 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/649: fsync d6/dd/d51/f86 0 2024-06-03T01:20:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/650: chown d6/dd/d51/d16/d27/d5e/d71/fbe 13608017 1 2024-06-03T01:20:59.199 INFO:tasks.workunit.client.0.smithi032.stdout:2/248: dread d5/da/f20 [0,4194304] 0 2024-06-03T01:20:59.199 INFO:tasks.workunit.client.0.smithi032.stdout:2/249: fsync d5/d38/f3b 0 2024-06-03T01:20:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/656: rename d2/d11/d1e/l36 to d2/d11/d3e/dce/ld9 0 2024-06-03T01:20:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/657: readlink d2/d11/d1e/d3b/l5a 0 2024-06-03T01:20:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/658: write d2/f2e [4275294,129324] 0 2024-06-03T01:20:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/659: readlink d2/d11/d12/d2b/d44/lc4 0 2024-06-03T01:20:59.201 INFO:tasks.workunit.client.0.smithi032.stdout:2/250: unlink d5/da/d1a/d2a/l4e 0 2024-06-03T01:20:59.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/839: mknod d3/d9/d36/d82/c138 0 2024-06-03T01:20:59.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/840: creat d3/d9/d55/d5a/dde/dea/f139 x:0 0 0 2024-06-03T01:20:59.206 INFO:tasks.workunit.client.1.smithi044.stdout:3/614: dwrite df/d13/f3f [0,4194304] 0 2024-06-03T01:20:59.207 INFO:tasks.workunit.client.1.smithi044.stdout:7/723: write d0/d3/d8/d95/d63/d89/da3/f9e [965179,39000] 0 2024-06-03T01:20:59.208 INFO:tasks.workunit.client.1.smithi044.stdout:1/852: rmdir d0/dd2/dfe/dc2 39 2024-06-03T01:20:59.213 INFO:tasks.workunit.client.1.smithi044.stdout:0/841: dread d3/da6/d20/d25/f4b [0,4194304] 0 2024-06-03T01:20:59.213 INFO:tasks.workunit.client.1.smithi044.stdout:0/842: truncate d3/d9/d55/d7b/fb0 4248268 0 2024-06-03T01:20:59.218 INFO:tasks.workunit.client.1.smithi044.stdout:2/557: rename db/d15/d49/d6c/d72/d8f/cad to db/d15/d49/cb0 0 2024-06-03T01:20:59.220 INFO:tasks.workunit.client.1.smithi044.stdout:5/660: mkdir d2/d11/d1e/dda 0 2024-06-03T01:20:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:58 smithi044 ceph-mon[36956]: pgmap v4: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:20:59.224 INFO:tasks.workunit.client.1.smithi044.stdout:0/843: write d3/d9/d55/d5a/d29/f4f [628743,51549] 0 2024-06-03T01:20:59.224 INFO:tasks.workunit.client.1.smithi044.stdout:0/844: creat d3/d9/d102/f13a x:0 0 0 2024-06-03T01:20:59.226 INFO:tasks.workunit.client.1.smithi044.stdout:6/860: getdents d2/d23/d55/d5f/d99/dcc 0 2024-06-03T01:20:59.227 INFO:tasks.workunit.client.0.smithi032.stdout:2/251: dwrite d5/da/f29 [4194304,4194304] 0 2024-06-03T01:20:59.229 INFO:tasks.workunit.client.0.smithi032.stdout:2/252: mkdir d5/d38/d46/d53 0 2024-06-03T01:20:59.230 INFO:tasks.workunit.client.0.smithi032.stdout:2/253: readlink d5/da/d1a/d2a/d36/l44 0 2024-06-03T01:20:59.230 INFO:tasks.workunit.client.1.smithi044.stdout:7/724: dwrite d0/d16/f3c [0,4194304] 0 2024-06-03T01:20:59.231 INFO:tasks.workunit.client.0.smithi032.stdout:2/254: unlink d5/da/d1a/l27 0 2024-06-03T01:20:59.233 INFO:tasks.workunit.client.0.smithi032.stdout:2/255: creat d5/d38/d52/f54 x:0 0 0 2024-06-03T01:20:59.234 INFO:tasks.workunit.client.0.smithi032.stdout:2/256: readlink d5/da/d1a/d2a/d36/l4d 0 2024-06-03T01:20:59.234 INFO:tasks.workunit.client.0.smithi032.stdout:2/257: readlink d5/da/d1a/d2a/d36/l44 0 2024-06-03T01:20:59.235 INFO:tasks.workunit.client.1.smithi044.stdout:3/615: unlink df/d13/d1b/d56/d5c/d98/fa6 0 2024-06-03T01:20:59.246 INFO:tasks.workunit.client.1.smithi044.stdout:6/861: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/fdd [0,4194304] 0 2024-06-03T01:20:59.246 INFO:tasks.workunit.client.1.smithi044.stdout:0/845: dwrite d3/da6/d56/d62/fa1 [0,4194304] 0 2024-06-03T01:20:59.252 INFO:tasks.workunit.client.1.smithi044.stdout:3/616: dwrite df/d13/d1b/d56/d5c/f5f [0,4194304] 0 2024-06-03T01:20:59.252 INFO:tasks.workunit.client.1.smithi044.stdout:3/617: chown df/d13/d1b/d4b/d69 330625157 1 2024-06-03T01:20:59.252 INFO:tasks.workunit.client.1.smithi044.stdout:3/618: read - df/d13/d39/d3d/fc5 zero size 2024-06-03T01:20:59.255 INFO:tasks.workunit.client.0.smithi032.stdout:2/258: truncate d5/da/f29 5105222 0 2024-06-03T01:20:59.258 INFO:tasks.workunit.client.1.smithi044.stdout:1/853: symlink d0/d3/d20/d7/l124 0 2024-06-03T01:20:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:6/862: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/d12c/dc0/f10f [4194304,4194304] 0 2024-06-03T01:20:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:6/863: chown d2/d23/d30/d1e/c32 23638358 1 2024-06-03T01:20:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:6/864: readlink d2/d23/d55/d5f/ld2 0 2024-06-03T01:20:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:9/651: rename d6/dd/d45/f81 to d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf 0 2024-06-03T01:20:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:9/652: rename d6/dd/d51 to d6/dd/d51/d16/d4d/dad/dcd/dd0 22 2024-06-03T01:20:59.262 INFO:tasks.workunit.client.1.smithi044.stdout:9/653: fsync d6/dd/d51/d16/d61/f6c 0 2024-06-03T01:20:59.262 INFO:tasks.workunit.client.1.smithi044.stdout:9/654: dread - d6/dd/d51/d16/d5f/d79/f90 zero size 2024-06-03T01:20:59.263 INFO:tasks.workunit.client.1.smithi044.stdout:2/558: creat db/d1b/d5d/fb1 x:0 0 0 2024-06-03T01:20:59.264 INFO:tasks.workunit.client.1.smithi044.stdout:5/661: creat d2/d11/d1e/d3b/fdb x:0 0 0 2024-06-03T01:20:59.265 INFO:tasks.workunit.client.1.smithi044.stdout:9/655: dread d6/dd/d51/f86 [0,4194304] 0 2024-06-03T01:20:59.266 INFO:tasks.workunit.client.0.smithi032.stdout:2/259: write d5/da/d1a/f26 [3245176,76601] 0 2024-06-03T01:20:59.268 INFO:tasks.workunit.client.0.smithi032.stdout:2/260: dread d5/f3c [0,4194304] 0 2024-06-03T01:20:59.270 INFO:tasks.workunit.client.1.smithi044.stdout:7/725: link d0/d16/f4d d0/d3/d8/d95/d63/feb 0 2024-06-03T01:20:59.270 INFO:tasks.workunit.client.1.smithi044.stdout:7/726: fsync d0/d3/d8/d95/d63/d89/da3/d9a/fe1 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/261: mknod d5/da/d1a/c55 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/262: readlink d5/da/d1a/d2a/d36/l4a 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/263: readlink d5/l24 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/264: creat d5/da/d1a/d2a/d36/d3f/f56 x:0 0 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/265: fsync d5/da/d1a/f4b 0 2024-06-03T01:20:59.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/266: write d5/da/d1a/f41 [1003195,88322] 0 2024-06-03T01:20:59.275 INFO:tasks.workunit.client.1.smithi044.stdout:0/846: mknod d3/d9/d55/d7b/dfc/d10c/c13b 0 2024-06-03T01:20:59.279 INFO:tasks.workunit.client.1.smithi044.stdout:3/619: stat df/d13/d1b/d93/fa7 0 2024-06-03T01:20:59.280 INFO:tasks.workunit.client.1.smithi044.stdout:1/854: getdents d0/d3/d20/d7/d13/d2d/dcd/d108/d64 0 2024-06-03T01:20:59.284 INFO:tasks.workunit.client.1.smithi044.stdout:6/865: creat d2/d23/d30/ddf/f12d x:0 0 0 2024-06-03T01:20:59.286 INFO:tasks.workunit.client.1.smithi044.stdout:2/559: symlink db/d1b/d27/lb2 0 2024-06-03T01:20:59.286 INFO:tasks.workunit.client.0.smithi032.stdout:2/267: dwrite d5/da/d2e/f32 [0,4194304] 0 2024-06-03T01:20:59.286 INFO:tasks.workunit.client.0.smithi032.stdout:2/268: readlink d5/da/d1a/d2a/d36/l4d 0 2024-06-03T01:20:59.287 INFO:tasks.workunit.client.1.smithi044.stdout:9/656: mknod d6/dd/d51/d16/d4d/d67/cd1 0 2024-06-03T01:20:59.287 INFO:tasks.workunit.client.1.smithi044.stdout:9/657: dread - d6/dd/d51/d16/d27/d5e/d71/dc1/fc4 zero size 2024-06-03T01:20:59.287 INFO:tasks.workunit.client.1.smithi044.stdout:9/658: write d6/dd/f17 [576971,80109] 0 2024-06-03T01:20:59.288 INFO:tasks.workunit.client.0.smithi032.stdout:2/269: mkdir d5/d38/d46/d53/d57 0 2024-06-03T01:20:59.288 INFO:tasks.workunit.client.0.smithi032.stdout:2/270: creat d5/f58 x:0 0 0 2024-06-03T01:20:59.288 INFO:tasks.workunit.client.0.smithi032.stdout:2/271: write d5/da/d1a/d2a/d36/d3f/f51 [522680,7307] 0 2024-06-03T01:20:59.291 INFO:tasks.workunit.client.0.smithi032.stdout:2/272: creat d5/da/f59 x:0 0 0 2024-06-03T01:20:59.292 INFO:tasks.workunit.client.1.smithi044.stdout:7/727: rename d0/d3/d8/da0/fbe to d0/d3/d8/d95/d63/d89/fec 0 2024-06-03T01:20:59.292 INFO:tasks.workunit.client.1.smithi044.stdout:7/728: fdatasync d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/fd8 0 2024-06-03T01:20:59.292 INFO:tasks.workunit.client.1.smithi044.stdout:7/729: write d0/d16/d23/f24 [1025920,87840] 0 2024-06-03T01:20:59.292 INFO:tasks.workunit.client.1.smithi044.stdout:7/730: write d0/d3/d8/d95/d63/d7f/d9c/fe3 [165363,86043] 0 2024-06-03T01:20:59.293 INFO:tasks.workunit.client.0.smithi032.stdout:2/273: symlink d5/da/d1a/d2a/d49/l5a 0 2024-06-03T01:20:59.293 INFO:tasks.workunit.client.1.smithi044.stdout:0/847: rmdir d3/da6/d21/d94/da3 39 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/848: chown d3/f81 47 1 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/849: creat d3/f13c x:0 0 0 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/850: chown d3/d6b/d95/cdd 15 1 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/851: write d3/f85 [950324,120052] 0 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:3/620: creat df/d13/d6d/d9a/fcc x:0 0 0 2024-06-03T01:20:59.294 INFO:tasks.workunit.client.1.smithi044.stdout:3/621: read fe [107586,1252] 0 2024-06-03T01:20:59.295 INFO:tasks.workunit.client.1.smithi044.stdout:1/855: truncate d0/d3/d20/d7/d13/d2d/dcd/d108/f93 583424 0 2024-06-03T01:20:59.296 INFO:tasks.workunit.client.1.smithi044.stdout:2/560: dread db/d1b/d3f/f4a [0,4194304] 0 2024-06-03T01:20:59.296 INFO:tasks.workunit.client.1.smithi044.stdout:6/866: creat d2/d23/d30/d1e/d38/d77/f12e x:0 0 0 2024-06-03T01:20:59.299 INFO:tasks.workunit.client.1.smithi044.stdout:6/867: dread d2/d23/d30/d1e/d38/d6c/fbf [0,4194304] 0 2024-06-03T01:20:59.299 INFO:tasks.workunit.client.1.smithi044.stdout:9/659: rename d6/dd/d45/cc3 to d6/dd/d51/d16/d27/d5e/d71/d89/cd2 0 2024-06-03T01:20:59.299 INFO:tasks.workunit.client.1.smithi044.stdout:9/660: fdatasync d6/dd/d51/d16/f5a 0 2024-06-03T01:20:59.302 INFO:tasks.workunit.client.1.smithi044.stdout:7/731: mknod d0/d3/d8/d95/d63/d89/d8b/da9/ced 0 2024-06-03T01:20:59.303 INFO:tasks.workunit.client.1.smithi044.stdout:0/852: read d3/d9/d55/d5a/f93 [733453,18617] 0 2024-06-03T01:20:59.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/622: link df/d13/d1b/d56/d5c/c66 df/d13/d39/d3d/d50/ccd 0 2024-06-03T01:20:59.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/623: fdatasync df/d13/d1b/d4b/f4f 0 2024-06-03T01:20:59.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/624: chown df/d13/d6d 164 1 2024-06-03T01:20:59.304 INFO:tasks.workunit.client.1.smithi044.stdout:1/856: creat d0/d3/de1/f125 x:0 0 0 2024-06-03T01:20:59.305 INFO:tasks.workunit.client.1.smithi044.stdout:2/561: symlink db/d15/d49/d6c/d72/d7a/d4e/lb3 0 2024-06-03T01:20:59.308 INFO:tasks.workunit.client.1.smithi044.stdout:9/661: symlink d6/dd/d51/d16/d4d/d55/d68/ld3 0 2024-06-03T01:20:59.311 INFO:tasks.workunit.client.1.smithi044.stdout:2/562: rename db/d1b/d5d/f86 to db/d1b/d2a/d52/fb4 0 2024-06-03T01:20:59.311 INFO:tasks.workunit.client.1.smithi044.stdout:6/868: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fac [0,4194304] 0 2024-06-03T01:20:59.313 INFO:tasks.workunit.client.1.smithi044.stdout:9/662: getdents d6/dd/d51/d16/d4d 0 2024-06-03T01:20:59.313 INFO:tasks.workunit.client.1.smithi044.stdout:9/663: creat d6/dd/d51/d16/d4d/d55/d68/fd4 x:0 0 0 2024-06-03T01:20:59.313 INFO:tasks.workunit.client.1.smithi044.stdout:3/625: write df/d1e/f45 [573205,49659] 0 2024-06-03T01:20:59.313 INFO:tasks.workunit.client.1.smithi044.stdout:3/626: chown fd 427198 1 2024-06-03T01:20:59.314 INFO:tasks.workunit.client.1.smithi044.stdout:3/627: chown df/d1e/d2a/d70/f96 88 1 2024-06-03T01:20:59.314 INFO:tasks.workunit.client.1.smithi044.stdout:2/563: mkdir db/d15/d49/d82/db5 0 2024-06-03T01:20:59.316 INFO:tasks.workunit.client.1.smithi044.stdout:6/869: mknod d2/d23/d55/d5f/d99/dcc/dd4/df3/c12f 0 2024-06-03T01:20:59.317 INFO:tasks.workunit.client.1.smithi044.stdout:1/857: dread d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/fb3 [0,4194304] 0 2024-06-03T01:20:59.319 INFO:tasks.workunit.client.1.smithi044.stdout:2/564: dread db/d1b/d2a/d52/fb4 [0,4194304] 0 2024-06-03T01:20:59.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/870: rmdir d2/d23/d30/ddf/dfb 0 2024-06-03T01:20:59.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/871: truncate d2/d23/d30/ddf/ffd 484892 0 2024-06-03T01:20:59.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/872: chown d2/d23/d30/c26 12346 1 2024-06-03T01:20:59.321 INFO:tasks.workunit.client.1.smithi044.stdout:9/664: mkdir d6/dd/d51/db/dd5 0 2024-06-03T01:20:59.321 INFO:tasks.workunit.client.1.smithi044.stdout:2/565: mkdir db/d1b/d2a/d52/d54/db6 0 2024-06-03T01:20:59.322 INFO:tasks.workunit.client.1.smithi044.stdout:6/873: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/d12c/dc0/f130 x:0 0 0 2024-06-03T01:20:59.323 INFO:tasks.workunit.client.1.smithi044.stdout:9/665: mkdir d6/dd/d51/d16/d27/d5e/db7/dd6 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/566: unlink db/d1b/d27/d79/fa1 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/567: fsync db/d1b/d2a/f43 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/568: truncate db/d15/d49/f8e 155675 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:1/858: dread d0/f32 [0,4194304] 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/569: fdatasync db/d1b/d2a/d67/d53/f93 0 2024-06-03T01:20:59.325 INFO:tasks.workunit.client.1.smithi044.stdout:1/859: stat d0/d3/d20/d7/d13/d2d/d101 0 2024-06-03T01:20:59.327 INFO:tasks.workunit.client.0.smithi032.stdout:2/274: dwrite d5/f19 [0,4194304] 0 2024-06-03T01:20:59.327 INFO:tasks.workunit.client.0.smithi032.stdout:2/275: write d5/f1c [2509557,88046] 0 2024-06-03T01:20:59.331 INFO:tasks.workunit.client.1.smithi044.stdout:9/666: rename d6/dd/d51/d16/d5f/d79/cb3 to d6/dd/d51/d16/d27/d5e/d71/cd7 0 2024-06-03T01:20:59.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/860: creat d0/d3/f126 x:0 0 0 2024-06-03T01:20:59.331 INFO:tasks.workunit.client.1.smithi044.stdout:9/667: stat d6/dd/d51/d46/f9f 0 2024-06-03T01:20:59.333 INFO:tasks.workunit.client.1.smithi044.stdout:7/732: dwrite d0/d3/da8/fd0 [0,4194304] 0 2024-06-03T01:20:59.333 INFO:tasks.workunit.client.0.smithi032.stdout:2/276: symlink d5/da/l5b 0 2024-06-03T01:20:59.334 INFO:tasks.workunit.client.0.smithi032.stdout:2/277: mkdir d5/d38/d46/d5c 0 2024-06-03T01:20:59.335 INFO:tasks.workunit.client.1.smithi044.stdout:1/861: link d0/d3/d20/d7/lf d0/dd2/dfe/d109/l127 0 2024-06-03T01:20:59.336 INFO:tasks.workunit.client.0.smithi032.stdout:2/278: mkdir d5/d5d 0 2024-06-03T01:20:59.338 INFO:tasks.workunit.client.0.smithi032.stdout:2/279: creat d5/d38/d46/d53/f5e x:0 0 0 2024-06-03T01:20:59.342 INFO:tasks.workunit.client.1.smithi044.stdout:1/862: rename d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/ce7 to d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d8c/c128 0 2024-06-03T01:20:59.345 INFO:tasks.workunit.client.1.smithi044.stdout:1/863: creat d0/dd2/dfe/dbe/d9e/d10d/dcb/f129 x:0 0 0 2024-06-03T01:20:59.345 INFO:tasks.workunit.client.1.smithi044.stdout:1/864: dread - d0/d3/d20/d7/d13/d2d/d101/f11c zero size 2024-06-03T01:20:59.347 INFO:tasks.workunit.client.1.smithi044.stdout:6/874: dwrite d2/d23/d55/d5f/db2/fec [0,4194304] 0 2024-06-03T01:20:59.350 INFO:tasks.workunit.client.1.smithi044.stdout:1/865: creat d0/d3/d20/d7/d13/d2d/dcd/d108/f12a x:0 0 0 2024-06-03T01:20:59.351 INFO:tasks.workunit.client.1.smithi044.stdout:2/570: dwrite db/d1b/d5d/f95 [0,4194304] 0 2024-06-03T01:20:59.351 INFO:tasks.workunit.client.1.smithi044.stdout:2/571: write db/d15/d49/f5c [432926,78446] 0 2024-06-03T01:20:59.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/875: dread d2/d23/d30/d1e/d38/f78 [0,4194304] 0 2024-06-03T01:20:59.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/876: chown d2/d23/d30/ddf/f12d 13 1 2024-06-03T01:20:59.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/877: chown d2/d23/d55/d5f/d53/f61 3685 1 2024-06-03T01:20:59.354 INFO:tasks.workunit.client.0.smithi032.stdout:3/215: sync 2024-06-03T01:20:59.355 INFO:tasks.workunit.client.1.smithi044.stdout:6/878: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/f131 x:0 0 0 2024-06-03T01:20:59.355 INFO:tasks.workunit.client.0.smithi032.stdout:3/216: mknod de/c41 0 2024-06-03T01:20:59.356 INFO:tasks.workunit.client.0.smithi032.stdout:3/217: chown de/c41 2036410 1 2024-06-03T01:20:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/572: write db/dd/f60 [697109,32057] 0 2024-06-03T01:20:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/573: read - db/d15/d49/d82/fac zero size 2024-06-03T01:20:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/574: readlink db/l13 0 2024-06-03T01:20:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/575: truncate db/d15/d49/d6c/fa9 225935 0 2024-06-03T01:20:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/576: chown db/d1b/d2a/d67/d53 430889630 1 2024-06-03T01:20:59.359 INFO:tasks.workunit.client.1.smithi044.stdout:2/577: creat db/dd/fb7 x:0 0 0 2024-06-03T01:20:59.363 INFO:tasks.workunit.client.1.smithi044.stdout:2/578: symlink db/d1b/d3f/lb8 0 2024-06-03T01:20:59.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/866: dread d0/dd2/dfe/dbe/f75 [0,4194304] 0 2024-06-03T01:20:59.364 INFO:tasks.workunit.client.1.smithi044.stdout:7/733: dwrite d0/d3/d8/d95/f53 [0,4194304] 0 2024-06-03T01:20:59.365 INFO:tasks.workunit.client.1.smithi044.stdout:2/579: creat db/d15/d49/d6c/d72/fb9 x:0 0 0 2024-06-03T01:20:59.365 INFO:tasks.workunit.client.1.smithi044.stdout:2/580: chown db/f12 505 1 2024-06-03T01:20:59.366 INFO:tasks.workunit.client.1.smithi044.stdout:2/581: write db/d1b/d2a/d67/d53/d9b/f9e [914644,102903] 0 2024-06-03T01:20:59.366 INFO:tasks.workunit.client.0.smithi032.stdout:2/280: dwrite d5/da/f2f [0,4194304] 0 2024-06-03T01:20:59.368 INFO:tasks.workunit.client.0.smithi032.stdout:2/281: symlink d5/da/d1a/d2a/d36/l5f 0 2024-06-03T01:20:59.375 INFO:tasks.workunit.client.1.smithi044.stdout:6/879: rename d2/d23/d55/d5f/d99/dcc/dd4 to d2/d23/d30/dfe/d132 0 2024-06-03T01:20:59.377 INFO:tasks.workunit.client.1.smithi044.stdout:9/668: dwrite d6/dd/d51/d16/d4d/d55/d68/fc8 [0,4194304] 0 2024-06-03T01:20:59.381 INFO:tasks.workunit.client.1.smithi044.stdout:9/669: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dd8 0 2024-06-03T01:20:59.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/282: dread d5/da/d1a/d2a/d36/d3f/f4f [0,4194304] 0 2024-06-03T01:20:59.383 INFO:tasks.workunit.client.1.smithi044.stdout:9/670: mkdir d6/dd/d51/d16/d61/dd9 0 2024-06-03T01:20:59.384 INFO:tasks.workunit.client.0.smithi032.stdout:3/218: dwrite de/d29/d2e/d31/f18 [0,4194304] 0 2024-06-03T01:20:59.384 INFO:tasks.workunit.client.0.smithi032.stdout:2/283: unlink d5/da/d1a/d2a/d36/l4d 0 2024-06-03T01:20:59.384 INFO:tasks.workunit.client.0.smithi032.stdout:2/284: truncate d5/da/d1a/f4b 739088 0 2024-06-03T01:20:59.385 INFO:tasks.workunit.client.1.smithi044.stdout:1/867: dwrite d0/f3f [0,4194304] 0 2024-06-03T01:20:59.389 INFO:tasks.workunit.client.0.smithi032.stdout:2/285: link d5/da/d1a/f41 d5/da/d1a/d2a/d36/f60 0 2024-06-03T01:20:59.391 INFO:tasks.workunit.client.1.smithi044.stdout:1/868: rename d0/d3/d20/d7/d13/d112/c118 to d0/d3/d20/d7/d2f/de0/c12b 0 2024-06-03T01:20:59.391 INFO:tasks.workunit.client.0.smithi032.stdout:3/219: rmdir de/d29/d2e/d31/d24 39 2024-06-03T01:20:59.394 INFO:tasks.workunit.client.0.smithi032.stdout:3/220: symlink de/d29/d2e/d31/d24/l42 0 2024-06-03T01:20:59.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/869: dread d0/dd2/dfe/dbe/d9e/d10d/fdd [0,4194304] 0 2024-06-03T01:20:59.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/870: stat d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/ff5 0 2024-06-03T01:20:59.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/221: rename de/d29/d2e/d31/c27 to de/d29/d2e/d31/d3c/c43 0 2024-06-03T01:20:59.397 INFO:tasks.workunit.client.0.smithi032.stdout:3/222: read de/d29/f3f [9468,73532] 0 2024-06-03T01:20:59.397 INFO:tasks.workunit.client.0.smithi032.stdout:3/223: fsync de/d29/d2e/d31/d24/f25 0 2024-06-03T01:20:59.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/286: write d5/f19 [136024,14479] 0 2024-06-03T01:20:59.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/287: creat d5/da/d2e/f61 x:0 0 0 2024-06-03T01:20:59.402 INFO:tasks.workunit.client.1.smithi044.stdout:9/671: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:20:59.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/224: link de/d29/f3b de/d29/d2e/d31/d3c/f44 0 2024-06-03T01:20:59.405 INFO:tasks.workunit.client.0.smithi032.stdout:3/225: dread de/d29/d2e/f37 [0,4194304] 0 2024-06-03T01:20:59.406 INFO:tasks.workunit.client.1.smithi044.stdout:2/582: truncate db/d15/d49/f5c 1475770 0 2024-06-03T01:20:59.407 INFO:tasks.workunit.client.0.smithi032.stdout:2/288: rename d5/da/d1a/d2a/d36/c43 to d5/da/d2e/c62 0 2024-06-03T01:20:59.407 INFO:tasks.workunit.client.0.smithi032.stdout:2/289: truncate d5/da/d1a/d2a/f50 6191 0 2024-06-03T01:20:59.409 INFO:tasks.workunit.client.1.smithi044.stdout:9/672: creat d6/dd/d51/db/dd5/fda x:0 0 0 2024-06-03T01:20:59.409 INFO:tasks.workunit.client.1.smithi044.stdout:9/673: creat d6/dd/d51/d3b/fdb x:0 0 0 2024-06-03T01:20:59.410 INFO:tasks.workunit.client.1.smithi044.stdout:2/583: dread db/d1b/d2a/d52/fb4 [0,4194304] 0 2024-06-03T01:20:59.412 INFO:tasks.workunit.client.1.smithi044.stdout:9/674: link d6/dd/d51/d16/f34 d6/dd/d51/d16/d27/d5e/db7/dd6/fdc 0 2024-06-03T01:20:59.412 INFO:tasks.workunit.client.1.smithi044.stdout:9/675: stat d6/dd/d51/d46/fb0 0 2024-06-03T01:20:59.417 INFO:tasks.workunit.client.1.smithi044.stdout:9/676: rename d6/dd/d51/d16/d4d to d6/dd/d51/d16/d5f/ddd 0 2024-06-03T01:20:59.417 INFO:tasks.workunit.client.1.smithi044.stdout:9/677: chown d6/dd/d51/d16/d5f/d8b/ca7 171064 1 2024-06-03T01:20:59.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/678: rename d6/dd/d51/d16/d5f/d79/f97 to d6/dd/d51/d16/d27/d5e/fde 0 2024-06-03T01:20:59.433 INFO:tasks.workunit.client.0.smithi032.stdout:0/256: sync 2024-06-03T01:20:59.433 INFO:tasks.workunit.client.0.smithi032.stdout:8/166: sync 2024-06-03T01:20:59.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/679: dread d6/dd/d51/d16/d61/f6c [0,4194304] 0 2024-06-03T01:20:59.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/680: write d6/dd/d51/d46/f48 [438417,12408] 0 2024-06-03T01:20:59.435 INFO:tasks.workunit.client.0.smithi032.stdout:0/257: creat d1/da/d28/f51 x:0 0 0 2024-06-03T01:20:59.436 INFO:tasks.workunit.client.0.smithi032.stdout:0/258: getdents d1/da/d10 0 2024-06-03T01:20:59.437 INFO:tasks.workunit.client.1.smithi044.stdout:9/681: dread d6/dd/d51/d16/d5f/ddd/d67/fb1 [0,4194304] 0 2024-06-03T01:20:59.438 INFO:tasks.workunit.client.1.smithi044.stdout:9/682: symlink d6/dd/d51/d16/d27/d5e/db7/ldf 0 2024-06-03T01:20:59.439 INFO:tasks.workunit.client.1.smithi044.stdout:9/683: truncate d6/dd/d51/d16/d5f/ddd/d55/d68/fd4 55999 0 2024-06-03T01:20:59.439 INFO:tasks.workunit.client.1.smithi044.stdout:8/766: sync 2024-06-03T01:20:59.439 INFO:tasks.workunit.client.1.smithi044.stdout:8/767: fsync d5/d55/d74/db4/dba/fc3 0 2024-06-03T01:20:59.440 INFO:tasks.workunit.client.1.smithi044.stdout:9/684: truncate d6/dd/d51/d16/f37 1340702 0 2024-06-03T01:20:59.440 INFO:tasks.workunit.client.1.smithi044.stdout:9/685: write d6/dd/d51/d46/f5c [152228,85839] 0 2024-06-03T01:20:59.440 INFO:tasks.workunit.client.1.smithi044.stdout:8/768: mknod d5/d16/d81/de3/d58/d89/dbf/dd2/cfe 0 2024-06-03T01:20:59.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/686: mkdir d6/dd/d51/d16/d27/d5e/d71/de0 0 2024-06-03T01:20:59.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/687: chown d6 0 1 2024-06-03T01:20:59.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/688: chown d6/dd/d51/c4c 265 1 2024-06-03T01:20:59.448 INFO:tasks.workunit.client.1.smithi044.stdout:9/689: rename d6/dd/d51/d3b to d6/dd/d51/d16/d27/d5e/db7/de1 0 2024-06-03T01:20:59.448 INFO:tasks.workunit.client.1.smithi044.stdout:9/690: write d6/dd/d51/d16/d5f/ddd/fc5 [470048,79110] 0 2024-06-03T01:20:59.449 INFO:tasks.workunit.client.1.smithi044.stdout:9/691: dread d6/dd/d51/d16/d27/d5e/db7/de1/f5d [0,4194304] 0 2024-06-03T01:20:59.456 INFO:tasks.workunit.client.1.smithi044.stdout:4/782: sync 2024-06-03T01:20:59.461 INFO:tasks.workunit.client.1.smithi044.stdout:2/584: dwrite db/d1b/d2a/f43 [0,4194304] 0 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:8/167: dwrite dc/fd [4194304,4194304] 0 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:3/226: dwrite de/d29/d2e/d31/d24/f39 [0,4194304] 0 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:3/227: truncate de/d29/d2e/d31/d3c/f44 4407396 0 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:3/228: chown de/c41 0 1 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:3/229: dread - de/d29/f35 zero size 2024-06-03T01:20:59.462 INFO:tasks.workunit.client.0.smithi032.stdout:2/290: dwrite d5/f23 [4194304,4194304] 0 2024-06-03T01:20:59.463 INFO:tasks.workunit.client.1.smithi044.stdout:1/871: dwrite d0/d3/d20/d7/d13/d2d/f79 [0,4194304] 0 2024-06-03T01:20:59.470 INFO:tasks.workunit.client.0.smithi032.stdout:3/230: rename de/d29/f3a to de/d29/d2e/f45 0 2024-06-03T01:20:59.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/168: mkdir dc/d25/d3a 0 2024-06-03T01:20:59.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/169: fdatasync f8 0 2024-06-03T01:20:59.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/170: truncate dc/d14/d31/f38 805210 0 2024-06-03T01:20:59.474 INFO:tasks.workunit.client.0.smithi032.stdout:3/231: mknod de/d29/d2e/d31/c46 0 2024-06-03T01:20:59.474 INFO:tasks.workunit.client.0.smithi032.stdout:3/232: write de/d29/f35 [239088,34657] 0 2024-06-03T01:20:59.476 INFO:tasks.workunit.client.0.smithi032.stdout:2/291: rmdir d5/da/d1a/d2a 39 2024-06-03T01:20:59.476 INFO:tasks.workunit.client.0.smithi032.stdout:2/292: fdatasync d5/f35 0 2024-06-03T01:20:59.486 INFO:tasks.workunit.client.1.smithi044.stdout:8/769: dwrite d5/fe4 [0,4194304] 0 2024-06-03T01:20:59.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/233: write de/d29/d2e/d31/f32 [3916765,49926] 0 2024-06-03T01:20:59.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/234: truncate de/d29/d2e/d31/d24/f25 1426195 0 2024-06-03T01:20:59.490 INFO:tasks.workunit.client.1.smithi044.stdout:8/770: rmdir d5/d16/d1d/d32/dfc 0 2024-06-03T01:20:59.490 INFO:tasks.workunit.client.1.smithi044.stdout:8/771: read d5/d20/d33/f63 [277027,129931] 0 2024-06-03T01:20:59.491 INFO:tasks.workunit.client.0.smithi032.stdout:3/235: getdents de/d29/d2e/d31/d23 0 2024-06-03T01:20:59.492 INFO:tasks.workunit.client.1.smithi044.stdout:8/772: dread d5/d55/d56/d5c/f72 [0,4194304] 0 2024-06-03T01:20:59.492 INFO:tasks.workunit.client.1.smithi044.stdout:8/773: chown d5/d55/d74/f94 479 1 2024-06-03T01:20:59.492 INFO:tasks.workunit.client.1.smithi044.stdout:8/774: chown d5/d55/d74/d82/db0 798767 1 2024-06-03T01:20:59.493 INFO:tasks.workunit.client.0.smithi032.stdout:3/236: dread f9 [0,4194304] 0 2024-06-03T01:20:59.493 INFO:tasks.workunit.client.0.smithi032.stdout:8/171: dwrite dc/d14/f22 [0,4194304] 0 2024-06-03T01:20:59.496 INFO:tasks.workunit.client.0.smithi032.stdout:8/172: mkdir dc/d14/d31/d3b 0 2024-06-03T01:20:59.500 INFO:tasks.workunit.client.0.smithi032.stdout:8/173: mkdir dc/d25/d28/d3c 0 2024-06-03T01:20:59.504 INFO:tasks.workunit.client.1.smithi044.stdout:4/783: dwrite d17/d1c/d75/d7a/db8/d28/d2b/f24 [0,4194304] 0 2024-06-03T01:20:59.504 INFO:tasks.workunit.client.1.smithi044.stdout:2/585: dwrite db/d1b/d3f/f66 [0,4194304] 0 2024-06-03T01:20:59.504 INFO:tasks.workunit.client.1.smithi044.stdout:2/586: chown db/f19 121306 1 2024-06-03T01:20:59.504 INFO:tasks.workunit.client.1.smithi044.stdout:1/872: dwrite d0/d3/d14/d81/fcf [0,4194304] 0 2024-06-03T01:20:59.505 INFO:tasks.workunit.client.0.smithi032.stdout:2/293: dwrite d5/da/f1e [0,4194304] 0 2024-06-03T01:20:59.506 INFO:tasks.workunit.client.0.smithi032.stdout:8/174: getdents dc/d25 0 2024-06-03T01:20:59.506 INFO:tasks.workunit.client.1.smithi044.stdout:8/775: dread d5/d20/d47/d64/fa0 [0,4194304] 0 2024-06-03T01:20:59.506 INFO:tasks.workunit.client.1.smithi044.stdout:9/692: dwrite d6/dd/d45/f56 [0,4194304] 0 2024-06-03T01:20:59.508 INFO:tasks.workunit.client.0.smithi032.stdout:2/294: creat d5/da/d1a/d2a/d36/d3f/f63 x:0 0 0 2024-06-03T01:20:59.508 INFO:tasks.workunit.client.0.smithi032.stdout:2/295: creat d5/d38/d52/f64 x:0 0 0 2024-06-03T01:20:59.508 INFO:tasks.workunit.client.0.smithi032.stdout:2/296: stat d5/da/d2e/f61 0 2024-06-03T01:20:59.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/237: rmdir de 39 2024-06-03T01:20:59.509 INFO:tasks.workunit.client.0.smithi032.stdout:8/175: chown dc/d14/c1c 6225987 1 2024-06-03T01:20:59.509 INFO:tasks.workunit.client.0.smithi032.stdout:2/297: dread d5/fe [0,4194304] 0 2024-06-03T01:20:59.511 INFO:tasks.workunit.client.1.smithi044.stdout:4/784: symlink d17/d1c/d70/dfb/dff/l103 0 2024-06-03T01:20:59.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/176: rename dc/l18 to dc/d14/d31/l3d 0 2024-06-03T01:20:59.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/177: stat l3 0 2024-06-03T01:20:59.515 INFO:tasks.workunit.client.1.smithi044.stdout:2/587: truncate db/d1b/d27/f50 1561107 0 2024-06-03T01:20:59.515 INFO:tasks.workunit.client.0.smithi032.stdout:2/298: symlink d5/da/d1a/d2a/d36/d3f/l65 0 2024-06-03T01:20:59.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/238: dread de/d29/f35 [0,4194304] 0 2024-06-03T01:20:59.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/239: write f8 [2154436,112270] 0 2024-06-03T01:20:59.516 INFO:tasks.workunit.client.0.smithi032.stdout:8/178: link dc/cf dc/d25/d28/d32/c3e 0 2024-06-03T01:20:59.516 INFO:tasks.workunit.client.0.smithi032.stdout:8/179: write dc/de/f37 [43494,125269] 0 2024-06-03T01:20:59.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/240: mknod de/d29/d2e/d31/d3c/c47 0 2024-06-03T01:20:59.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/241: write f8 [304589,52251] 0 2024-06-03T01:20:59.519 INFO:tasks.workunit.client.1.smithi044.stdout:1/873: mknod d0/d3/d20/d7/d2f/c12c 0 2024-06-03T01:20:59.519 INFO:tasks.workunit.client.1.smithi044.stdout:1/874: chown d0/dd2/dfe/f11e 12389136 1 2024-06-03T01:20:59.519 INFO:tasks.workunit.client.1.smithi044.stdout:1/875: stat d0/d3/d14/fe5 0 2024-06-03T01:20:59.520 INFO:tasks.workunit.client.1.smithi044.stdout:5/662: sync 2024-06-03T01:20:59.521 INFO:tasks.workunit.client.0.smithi032.stdout:1/146: sync 2024-06-03T01:20:59.521 INFO:tasks.workunit.client.0.smithi032.stdout:6/219: sync 2024-06-03T01:20:59.521 INFO:tasks.workunit.client.0.smithi032.stdout:9/365: sync 2024-06-03T01:20:59.521 INFO:tasks.workunit.client.0.smithi032.stdout:4/245: sync 2024-06-03T01:20:59.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/285: sync 2024-06-03T01:20:59.522 INFO:tasks.workunit.client.1.smithi044.stdout:8/776: rmdir d5/dd4 39 2024-06-03T01:20:59.523 INFO:tasks.workunit.client.0.smithi032.stdout:7/248: sync 2024-06-03T01:20:59.523 INFO:tasks.workunit.client.0.smithi032.stdout:8/180: creat dc/d25/d28/d3c/f3f x:0 0 0 2024-06-03T01:20:59.525 INFO:tasks.workunit.client.1.smithi044.stdout:2/588: dread db/d1b/d3f/f66 [0,4194304] 0 2024-06-03T01:20:59.527 INFO:tasks.workunit.client.1.smithi044.stdout:4/785: link d17/d1c/d75/d7a/db8/d28/d2b/d8e/cb0 d17/d1c/de0/c104 0 2024-06-03T01:20:59.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/876: creat d0/d3/d20/d7/d13/d112/f12d x:0 0 0 2024-06-03T01:20:59.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/877: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f113 [423446,122777] 0 2024-06-03T01:20:59.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/878: fsync d0/dd2/dfe/dbe/d9e/d10d/fdd 0 2024-06-03T01:20:59.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/879: creat d0/d3/d20/d7/d13/d2d/dcd/d108/f12e x:0 0 0 2024-06-03T01:20:59.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/242: rename de/d29/c36 to de/d29/c48 0 2024-06-03T01:20:59.529 INFO:tasks.workunit.client.0.smithi032.stdout:6/220: mknod d8/d11/d20/d36/c47 0 2024-06-03T01:20:59.529 INFO:tasks.workunit.client.0.smithi032.stdout:4/246: mknod d4/db/c4e 0 2024-06-03T01:20:59.529 INFO:tasks.workunit.client.0.smithi032.stdout:4/247: creat d4/db/da/d2d/d27/f4f x:0 0 0 2024-06-03T01:20:59.530 INFO:tasks.workunit.client.0.smithi032.stdout:4/248: write d4/f7 [5337178,86960] 0 2024-06-03T01:20:59.530 INFO:tasks.workunit.client.0.smithi032.stdout:4/249: read - d4/db/f2e zero size 2024-06-03T01:20:59.530 INFO:tasks.workunit.client.1.smithi044.stdout:5/663: unlink d2/c1d 0 2024-06-03T01:20:59.531 INFO:tasks.workunit.client.1.smithi044.stdout:8/777: creat d5/d16/d81/de3/d98/fff x:0 0 0 2024-06-03T01:20:59.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/589: mknod db/d1b/d2a/d52/d54/cba 0 2024-06-03T01:20:59.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/590: fdatasync db/d15/d49/d6c/d72/fb9 0 2024-06-03T01:20:59.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/591: readlink db/d15/d49/d6c/d72/d7a/d4e/laf 0 2024-06-03T01:20:59.533 INFO:tasks.workunit.client.0.smithi032.stdout:5/286: link da/dd/df/l1b da/dd/df/d10/d16/l55 0 2024-06-03T01:20:59.533 INFO:tasks.workunit.client.0.smithi032.stdout:5/287: fdatasync f4 0 2024-06-03T01:20:59.533 INFO:tasks.workunit.client.0.smithi032.stdout:7/249: truncate dc/dd/f20 565345 0 2024-06-03T01:20:59.534 INFO:tasks.workunit.client.0.smithi032.stdout:8/181: truncate fa 4106653 0 2024-06-03T01:20:59.534 INFO:tasks.workunit.client.0.smithi032.stdout:8/182: stat dc/d25 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/786: rename d17/d1c/d7e/ceb to d17/d1c/d75/d7a/db8/c105 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/787: fsync d17/d1c/d9c/ff3 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/693: dwrite d6/dd/d51/f62 [0,4194304] 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/694: fsync d6/dd/d51/d16/d27/f39 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.1.smithi044.stdout:9/695: fsync d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf 0 2024-06-03T01:20:59.535 INFO:tasks.workunit.client.0.smithi032.stdout:4/250: mkdir d4/db/da/d2d/d27/d50 0 2024-06-03T01:20:59.538 INFO:tasks.workunit.client.0.smithi032.stdout:5/288: creat da/dd/df/d10/d16/d25/f56 x:0 0 0 2024-06-03T01:20:59.538 INFO:tasks.workunit.client.0.smithi032.stdout:5/289: readlink l9 0 2024-06-03T01:20:59.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/290: fsync da/dd/f4e 0 2024-06-03T01:20:59.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/291: creat da/dd/df/d10/d16/d18/d1a/d30/d51/f57 x:0 0 0 2024-06-03T01:20:59.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/292: dread - da/dd/df/d10/d16/d18/d1a/d30/d51/f52 zero size 2024-06-03T01:20:59.539 INFO:tasks.workunit.client.1.smithi044.stdout:1/880: link d0/d3/d20/l48 d0/d3/d20/l12f 0 2024-06-03T01:20:59.539 INFO:tasks.workunit.client.1.smithi044.stdout:1/881: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f130 x:0 0 0 2024-06-03T01:20:59.540 INFO:tasks.workunit.client.0.smithi032.stdout:2/299: dwrite d5/da/f2c [4194304,4194304] 0 2024-06-03T01:20:59.540 INFO:tasks.workunit.client.0.smithi032.stdout:2/300: write d5/da/d1a/f4b [1437658,26325] 0 2024-06-03T01:20:59.540 INFO:tasks.workunit.client.0.smithi032.stdout:7/250: creat dc/d12/d22/d42/f50 x:0 0 0 2024-06-03T01:20:59.540 INFO:tasks.workunit.client.0.smithi032.stdout:7/251: chown dc/d12/d22/c3e 32645 1 2024-06-03T01:20:59.540 INFO:tasks.workunit.client.0.smithi032.stdout:4/251: creat d4/db/f51 x:0 0 0 2024-06-03T01:20:59.541 INFO:tasks.workunit.client.1.smithi044.stdout:5/664: mknod d2/d11/d1e/d3b/cdc 0 2024-06-03T01:20:59.543 INFO:tasks.workunit.client.1.smithi044.stdout:2/592: mkdir db/d1b/d3f/dbb 0 2024-06-03T01:20:59.544 INFO:tasks.workunit.client.0.smithi032.stdout:8/183: dread dc/f15 [0,4194304] 0 2024-06-03T01:20:59.546 INFO:tasks.workunit.client.0.smithi032.stdout:1/147: dwrite f17 [0,4194304] 0 2024-06-03T01:20:59.546 INFO:tasks.workunit.client.1.smithi044.stdout:4/788: mknod d17/d1c/d70/c106 0 2024-06-03T01:20:59.546 INFO:tasks.workunit.client.1.smithi044.stdout:4/789: chown d17/d1c/d75/d7a/db8/l1d 4975 1 2024-06-03T01:20:59.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/790: write d17/d1c/d75/d7a/db8/d59/f81 [4185705,3984] 0 2024-06-03T01:20:59.548 INFO:tasks.workunit.client.0.smithi032.stdout:5/293: dread da/dd/df/d10/f39 [0,4194304] 0 2024-06-03T01:20:59.549 INFO:tasks.workunit.client.1.smithi044.stdout:3/628: sync 2024-06-03T01:20:59.549 INFO:tasks.workunit.client.1.smithi044.stdout:7/734: sync 2024-06-03T01:20:59.549 INFO:tasks.workunit.client.1.smithi044.stdout:6/880: sync 2024-06-03T01:20:59.549 INFO:tasks.workunit.client.1.smithi044.stdout:0/853: sync 2024-06-03T01:20:59.549 INFO:tasks.workunit.client.1.smithi044.stdout:7/735: chown d0/d3/d8/d32/cb2 6965253 1 2024-06-03T01:20:59.551 INFO:tasks.workunit.client.1.smithi044.stdout:1/882: unlink d0/d3/d20/d7/d13/d2d/fce 0 2024-06-03T01:20:59.551 INFO:tasks.workunit.client.1.smithi044.stdout:1/883: fdatasync d0/d3/d20/d7/fa7 0 2024-06-03T01:20:59.552 INFO:tasks.workunit.client.0.smithi032.stdout:2/301: dread d5/da/f20 [4194304,4194304] 0 2024-06-03T01:20:59.554 INFO:tasks.workunit.client.0.smithi032.stdout:7/252: unlink dc/l27 0 2024-06-03T01:20:59.554 INFO:tasks.workunit.client.0.smithi032.stdout:7/253: truncate dc/d12/d24/d33/f4c 694397 0 2024-06-03T01:20:59.555 INFO:tasks.workunit.client.1.smithi044.stdout:1/884: dread d0/dd2/dfe/dbe/d9e/d10d/f88 [0,4194304] 0 2024-06-03T01:20:59.556 INFO:tasks.workunit.client.1.smithi044.stdout:4/791: creat d17/d1c/d75/d7a/db8/d28/d2b/d8e/f107 x:0 0 0 2024-06-03T01:20:59.558 INFO:tasks.workunit.client.1.smithi044.stdout:5/665: dread d2/f2e [0,4194304] 0 2024-06-03T01:20:59.559 INFO:tasks.workunit.client.1.smithi044.stdout:6/881: getdents d2/d23/d30/d1e/d38/d6c/d5c/d79 0 2024-06-03T01:20:59.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/629: creat df/d1e/d8a/fce x:0 0 0 2024-06-03T01:20:59.562 INFO:tasks.workunit.client.1.smithi044.stdout:2/593: getdents db/d1b/d2a/d2f 0 2024-06-03T01:20:59.563 INFO:tasks.workunit.client.1.smithi044.stdout:2/594: chown db/d1b/l21 116264516 1 2024-06-03T01:20:59.564 INFO:tasks.workunit.client.1.smithi044.stdout:6/882: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/d12c/dc0/dff/l133 0 2024-06-03T01:20:59.565 INFO:tasks.workunit.client.1.smithi044.stdout:3/630: symlink df/d13/d39/d3d/d50/lcf 0 2024-06-03T01:20:59.567 INFO:tasks.workunit.client.0.smithi032.stdout:9/366: dwrite d0/d11/d1d/d38/d4f/f7a [0,4194304] 0 2024-06-03T01:20:59.568 INFO:tasks.workunit.client.1.smithi044.stdout:6/883: creat d2/f134 x:0 0 0 2024-06-03T01:20:59.569 INFO:tasks.workunit.client.1.smithi044.stdout:6/884: truncate d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/fc9 1484074 0 2024-06-03T01:20:59.570 INFO:tasks.workunit.client.1.smithi044.stdout:6/885: creat d2/d23/d55/d5f/f135 x:0 0 0 2024-06-03T01:20:59.574 INFO:tasks.workunit.client.1.smithi044.stdout:6/886: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/d12c/f136 x:0 0 0 2024-06-03T01:20:59.574 INFO:tasks.workunit.client.1.smithi044.stdout:6/887: fdatasync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/fe9 0 2024-06-03T01:20:59.577 INFO:tasks.workunit.client.0.smithi032.stdout:7/254: dread dc/dd/f11 [4194304,4194304] 0 2024-06-03T01:20:59.577 INFO:tasks.workunit.client.0.smithi032.stdout:7/255: chown dc/f15 3511346 1 2024-06-03T01:20:59.577 INFO:tasks.workunit.client.0.smithi032.stdout:4/252: mknod d4/db/da/d2d/c52 0 2024-06-03T01:20:59.578 INFO:tasks.workunit.client.1.smithi044.stdout:4/792: dread d17/d1c/d75/fe1 [0,4194304] 0 2024-06-03T01:20:59.578 INFO:tasks.workunit.client.1.smithi044.stdout:2/595: dread db/d15/d49/d6c/d72/d7a/f68 [0,4194304] 0 2024-06-03T01:20:59.578 INFO:tasks.workunit.client.1.smithi044.stdout:1/885: dread d0/d3/d14/dae/f110 [0,4194304] 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/886: fsync d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/fd4 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/887: chown d0/d3/d14/fe5 17737344 1 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/888: truncate d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/f72 990832 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/889: chown d0/dd2/dfe/dbe/d9e/d10d/dcb/fe3 22449883 1 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/890: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f90 [35112,101350] 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/891: write d0/d3/d14/d30/f117 [457337,39835] 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/892: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f131 x:0 0 0 2024-06-03T01:20:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:5/666: read d2/d11/d1e/f51 [11438229,108192] 0 2024-06-03T01:20:59.580 INFO:tasks.workunit.client.1.smithi044.stdout:4/793: creat d17/d1c/d75/d7a/db3/de9/f108 x:0 0 0 2024-06-03T01:20:59.580 INFO:tasks.workunit.client.0.smithi032.stdout:8/184: symlink dc/d14/d31/d3b/l40 0 2024-06-03T01:20:59.580 INFO:tasks.workunit.client.0.smithi032.stdout:8/185: write f9 [3405451,123298] 0 2024-06-03T01:20:59.580 INFO:tasks.workunit.client.0.smithi032.stdout:8/186: read dc/d14/d31/f38 [691775,34461] 0 2024-06-03T01:20:59.580 INFO:tasks.workunit.client.0.smithi032.stdout:8/187: stat dc/de/d1e/c23 0 2024-06-03T01:20:59.581 INFO:tasks.workunit.client.0.smithi032.stdout:8/188: dread dc/f13 [0,4194304] 0 2024-06-03T01:20:59.581 INFO:tasks.workunit.client.1.smithi044.stdout:2/596: creat db/d1b/d2a/d52/d54/fbc x:0 0 0 2024-06-03T01:20:59.582 INFO:tasks.workunit.client.1.smithi044.stdout:3/631: write df/d1e/d78/da4/fbf [53190,62555] 0 2024-06-03T01:20:59.583 INFO:tasks.workunit.client.0.smithi032.stdout:5/294: rmdir da/dd/df/d10/d16 39 2024-06-03T01:20:59.586 INFO:tasks.workunit.client.0.smithi032.stdout:9/367: link d0/d23/d24/c59 d0/d11/d79/c80 0 2024-06-03T01:20:59.587 INFO:tasks.workunit.client.0.smithi032.stdout:7/256: truncate dc/dd/f11 7224511 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/189: rename dc/l17 to dc/d25/d28/d33/l41 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/190: creat dc/d25/d28/d3c/f42 x:0 0 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/191: write dc/f15 [1269446,80784] 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/192: fdatasync f1 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/193: write f9 [4001325,2380] 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/194: truncate dc/de/d1e/f2e 153290 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/195: truncate dc/d14/f22 4497715 0 2024-06-03T01:20:59.590 INFO:tasks.workunit.client.0.smithi032.stdout:2/302: dwrite d5/d38/f3b [0,4194304] 0 2024-06-03T01:20:59.591 INFO:tasks.workunit.client.1.smithi044.stdout:7/736: dwrite d0/d3/d8/d95/d63/f7a [4194304,4194304] 0 2024-06-03T01:20:59.591 INFO:tasks.workunit.client.1.smithi044.stdout:0/854: dwrite d3/da6/d56/d8e/f92 [0,4194304] 0 2024-06-03T01:20:59.591 INFO:tasks.workunit.client.0.smithi032.stdout:3/243: dwrite f8 [4194304,4194304] 0 2024-06-03T01:20:59.591 INFO:tasks.workunit.client.0.smithi032.stdout:5/295: mkdir da/dd/df/d10/d16/d18/d1a/d30/d58 0 2024-06-03T01:20:59.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/893: getdents d0/d3/d20/d7/d2f/db7 0 2024-06-03T01:20:59.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/894: dread - d0/d3/d20/d7/d13/fab zero size 2024-06-03T01:20:59.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/895: truncate d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80/fee 216583 0 2024-06-03T01:20:59.594 INFO:tasks.workunit.client.0.smithi032.stdout:8/196: mkdir dc/de/d1e/d43 0 2024-06-03T01:20:59.597 INFO:tasks.workunit.client.1.smithi044.stdout:5/667: rename d2/d11/d1e/d3b/l5a to d2/d11/d12/d2b/d44/ldd 0 2024-06-03T01:20:59.598 INFO:tasks.workunit.client.0.smithi032.stdout:2/303: creat d5/da/d1a/d2a/d36/f66 x:0 0 0 2024-06-03T01:20:59.599 INFO:tasks.workunit.client.1.smithi044.stdout:4/794: mknod d17/d1c/d75/d7a/db8/c109 0 2024-06-03T01:20:59.602 INFO:tasks.workunit.client.0.smithi032.stdout:2/304: dread d5/f25 [0,4194304] 0 2024-06-03T01:20:59.607 INFO:tasks.workunit.client.0.smithi032.stdout:3/244: mknod de/d29/d2e/d31/d23/c49 0 2024-06-03T01:20:59.607 INFO:tasks.workunit.client.1.smithi044.stdout:2/597: symlink db/d1b/d5d/da7/lbd 0 2024-06-03T01:20:59.612 INFO:tasks.workunit.client.0.smithi032.stdout:5/296: unlink da/dd/df/d10/d16/f36 0 2024-06-03T01:20:59.613 INFO:tasks.workunit.client.0.smithi032.stdout:3/245: creat de/d29/d2e/d31/d23/f4a x:0 0 0 2024-06-03T01:20:59.613 INFO:tasks.workunit.client.0.smithi032.stdout:3/246: read - de/d29/d2e/d31/f1c zero size 2024-06-03T01:20:59.618 INFO:tasks.workunit.client.1.smithi044.stdout:7/737: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee 0 2024-06-03T01:20:59.618 INFO:tasks.workunit.client.1.smithi044.stdout:6/888: rmdir d2/d23/d55/d5f 39 2024-06-03T01:20:59.621 INFO:tasks.workunit.client.0.smithi032.stdout:5/297: dread da/dd/df/d10/f39 [0,4194304] 0 2024-06-03T01:20:59.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/855: mkdir d3/d9/d55/d7b/dfc/d10c/d13d 0 2024-06-03T01:20:59.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/856: chown d3/da6/d20/d25/ld5 238 1 2024-06-03T01:20:59.624 INFO:tasks.workunit.client.1.smithi044.stdout:0/857: fdatasync d3/d6b/d95/db3/f12b 0 2024-06-03T01:20:59.625 INFO:tasks.workunit.client.0.smithi032.stdout:5/298: dread da/dd/df/d10/f42 [0,4194304] 0 2024-06-03T01:20:59.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/368: dwrite d0/d11/d76/f7c [0,4194304] 0 2024-06-03T01:20:59.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/369: chown d0/d23/d25/f4c 7526688 1 2024-06-03T01:20:59.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/370: write d0/d23/d24/f72 [333977,101868] 0 2024-06-03T01:20:59.626 INFO:tasks.workunit.client.0.smithi032.stdout:1/148: dwrite d18/f1e [0,4194304] 0 2024-06-03T01:20:59.627 INFO:tasks.workunit.client.1.smithi044.stdout:0/858: dread d3/f85 [0,4194304] 0 2024-06-03T01:20:59.628 INFO:tasks.workunit.client.0.smithi032.stdout:8/197: dwrite dc/d14/f22 [0,4194304] 0 2024-06-03T01:20:59.628 INFO:tasks.workunit.client.0.smithi032.stdout:7/257: dwrite dc/d12/d1b/f34 [0,4194304] 0 2024-06-03T01:20:59.628 INFO:tasks.workunit.client.0.smithi032.stdout:8/198: chown dc/d25/f35 49432482 1 2024-06-03T01:20:59.629 INFO:tasks.workunit.client.0.smithi032.stdout:7/258: dread - dc/f3d zero size 2024-06-03T01:20:59.630 INFO:tasks.workunit.client.0.smithi032.stdout:2/305: dwrite d5/da/d1a/d2a/d36/f60 [0,4194304] 0 2024-06-03T01:20:59.630 INFO:tasks.workunit.client.0.smithi032.stdout:2/306: truncate d5/da/d1a/f4b 1573153 0 2024-06-03T01:20:59.630 INFO:tasks.workunit.client.0.smithi032.stdout:2/307: chown d5/f25 3797122 1 2024-06-03T01:20:59.632 INFO:tasks.workunit.client.1.smithi044.stdout:1/896: rmdir d0/dd2/dfe/d109/d115 0 2024-06-03T01:20:59.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/299: mknod da/c59 0 2024-06-03T01:20:59.634 INFO:tasks.workunit.client.1.smithi044.stdout:5/668: truncate d2/d11/d12/d2b/d4a/f5d 3734397 0 2024-06-03T01:20:59.634 INFO:tasks.workunit.client.1.smithi044.stdout:5/669: readlink d2/d11/d12/d2b/l6f 0 2024-06-03T01:20:59.635 INFO:tasks.workunit.client.1.smithi044.stdout:3/632: dread df/d13/d1b/fae [8388608,4194304] 0 2024-06-03T01:20:59.637 INFO:tasks.workunit.client.0.smithi032.stdout:3/247: dwrite de/d29/d2e/f37 [0,4194304] 0 2024-06-03T01:20:59.638 INFO:tasks.workunit.client.1.smithi044.stdout:2/598: mkdir db/d1b/d2a/d2f/dbe 0 2024-06-03T01:20:59.638 INFO:tasks.workunit.client.1.smithi044.stdout:4/795: dwrite d17/d1c/d75/d7a/db8/f9b [0,4194304] 0 2024-06-03T01:20:59.640 INFO:tasks.workunit.client.1.smithi044.stdout:6/889: rmdir d2/d23/d55/d5f/d85 39 2024-06-03T01:20:59.641 INFO:tasks.workunit.client.0.smithi032.stdout:1/149: getdents d18/d27 0 2024-06-03T01:20:59.643 INFO:tasks.workunit.client.0.smithi032.stdout:9/371: write d0/d11/d1d/d38/d4f/f7a [2404829,30864] 0 2024-06-03T01:20:59.644 INFO:tasks.workunit.client.0.smithi032.stdout:9/372: fsync d0/d23/f3d 0 2024-06-03T01:20:59.646 INFO:tasks.workunit.client.1.smithi044.stdout:0/859: unlink d3/d6b/de8/d115/f11b 0 2024-06-03T01:20:59.647 INFO:tasks.workunit.client.1.smithi044.stdout:0/860: dread - d3/d5c/fbd zero size 2024-06-03T01:20:59.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/199: symlink dc/l44 0 2024-06-03T01:20:59.650 INFO:tasks.workunit.client.1.smithi044.stdout:7/738: write d0/d16/f3c [2984001,125941] 0 2024-06-03T01:20:59.651 INFO:tasks.workunit.client.1.smithi044.stdout:3/633: getdents df/d13/d39/d3d/d43 0 2024-06-03T01:20:59.651 INFO:tasks.workunit.client.1.smithi044.stdout:3/634: stat df/d1e/d3a 0 2024-06-03T01:20:59.654 INFO:tasks.workunit.client.0.smithi032.stdout:5/300: truncate da/dd/df/d10/f39 1975430 0 2024-06-03T01:20:59.654 INFO:tasks.workunit.client.0.smithi032.stdout:5/301: fsync da/dd/df/f46 0 2024-06-03T01:20:59.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/670: dread d2/d11/d3e/f59 [0,4194304] 0 2024-06-03T01:20:59.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/671: stat d2/d11/d12/dad/fbf 0 2024-06-03T01:20:59.657 INFO:tasks.workunit.client.1.smithi044.stdout:2/599: mknod db/d1b/d2a/d52/d54/db6/cbf 0 2024-06-03T01:20:59.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/672: fdatasync d2/d11/d1e/d21/d24/d93/d9d/f62 0 2024-06-03T01:20:59.658 INFO:tasks.workunit.client.1.smithi044.stdout:2/600: dread - db/d15/d49/d7b/f88 zero size 2024-06-03T01:20:59.658 INFO:tasks.workunit.client.1.smithi044.stdout:2/601: write db/d1b/d2a/f43 [4547025,128077] 0 2024-06-03T01:20:59.658 INFO:tasks.workunit.client.1.smithi044.stdout:9/696: sync 2024-06-03T01:20:59.659 INFO:tasks.workunit.client.1.smithi044.stdout:8/778: sync 2024-06-03T01:20:59.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/697: read d6/dd/f21 [3304278,105703] 0 2024-06-03T01:20:59.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/698: fsync d6/dd/f17 0 2024-06-03T01:20:59.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/699: read - d6/dd/d51/db/dd5/fda zero size 2024-06-03T01:20:59.659 INFO:tasks.workunit.client.1.smithi044.stdout:4/796: dwrite d17/d1c/d75/d7a/db8/fa5 [0,4194304] 0 2024-06-03T01:20:59.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/308: dwrite d5/f25 [0,4194304] 0 2024-06-03T01:20:59.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/309: dread - d5/d38/d52/f54 zero size 2024-06-03T01:20:59.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/310: write d5/da/d1a/d2a/d36/d3f/f51 [1133828,49523] 0 2024-06-03T01:20:59.661 INFO:tasks.workunit.client.1.smithi044.stdout:6/890: creat d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/f137 x:0 0 0 2024-06-03T01:20:59.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/700: dread d6/dd/d51/d16/f37 [0,4194304] 0 2024-06-03T01:20:59.661 INFO:tasks.workunit.client.1.smithi044.stdout:2/602: read db/d1b/d3f/f66 [1951287,75484] 0 2024-06-03T01:20:59.663 INFO:tasks.workunit.client.1.smithi044.stdout:7/739: rmdir d0/d3/d8/d95 39 2024-06-03T01:20:59.663 INFO:tasks.workunit.client.1.smithi044.stdout:7/740: stat d0/d3/ddf 0 2024-06-03T01:20:59.664 INFO:tasks.workunit.client.0.smithi032.stdout:1/150: write d18/f1e [339415,58911] 0 2024-06-03T01:20:59.668 INFO:tasks.workunit.client.0.smithi032.stdout:3/248: creat de/d29/f4b x:0 0 0 2024-06-03T01:20:59.668 INFO:tasks.workunit.client.1.smithi044.stdout:1/897: rename d0/dd2/dfe/dc2/f10f to d0/d3/d20/d7/d13/d2d/d101/f132 0 2024-06-03T01:20:59.670 INFO:tasks.workunit.client.0.smithi032.stdout:7/259: truncate dc/d12/d1b/f2c 52715 0 2024-06-03T01:20:59.670 INFO:tasks.workunit.client.0.smithi032.stdout:8/200: rename dc/de/l34 to dc/d14/d31/l45 0 2024-06-03T01:20:59.670 INFO:tasks.workunit.client.0.smithi032.stdout:7/260: chown dc/l4d 122461 1 2024-06-03T01:20:59.672 INFO:tasks.workunit.client.1.smithi044.stdout:5/673: symlink d2/d11/d12/d2b/lde 0 2024-06-03T01:20:59.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/779: creat d5/d55/d56/d5c/d68/de5/dea/d79/f100 x:0 0 0 2024-06-03T01:20:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:4/797: truncate d17/d1c/d75/d7a/db3/f95 646808 0 2024-06-03T01:20:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:6/891: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/l138 0 2024-06-03T01:20:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:6/892: fsync d2/d23/f35 0 2024-06-03T01:20:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:4/798: creat d17/d1c/d75/d7a/db3/f10a x:0 0 0 2024-06-03T01:20:59.679 INFO:tasks.workunit.client.0.smithi032.stdout:1/151: mkdir d18/d27/d2b 0 2024-06-03T01:20:59.681 INFO:tasks.workunit.client.1.smithi044.stdout:9/701: mknod d6/dd/d51/d16/d5f/d8b/d8d/ce2 0 2024-06-03T01:20:59.681 INFO:tasks.workunit.client.1.smithi044.stdout:9/702: read d6/dd/d51/d46/f5c [147995,99388] 0 2024-06-03T01:20:59.683 INFO:tasks.workunit.client.1.smithi044.stdout:2/603: unlink db/d1b/d27/f63 0 2024-06-03T01:20:59.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/261: link dc/d12/d24/d33/f4b dc/d12/d22/d42/f51 0 2024-06-03T01:20:59.684 INFO:tasks.workunit.client.1.smithi044.stdout:5/674: write d2/d11/d12/d2b/d4a/d57/f76 [3843724,64653] 0 2024-06-03T01:20:59.686 INFO:tasks.workunit.client.0.smithi032.stdout:7/262: getdents dc/d12/d1b 0 2024-06-03T01:20:59.686 INFO:tasks.workunit.client.1.smithi044.stdout:3/635: rename df/d1e/d3a/lac to df/d13/d6d/ld0 0 2024-06-03T01:20:59.689 INFO:tasks.workunit.client.0.smithi032.stdout:5/302: dwrite da/dd/df/d10/d16/d18/d35/f3a [0,4194304] 0 2024-06-03T01:20:59.690 INFO:tasks.workunit.client.1.smithi044.stdout:1/898: unlink d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/ff5 0 2024-06-03T01:20:59.690 INFO:tasks.workunit.client.1.smithi044.stdout:1/899: chown d0/d3/de1/f125 1524 1 2024-06-03T01:20:59.694 INFO:tasks.workunit.client.1.smithi044.stdout:8/780: creat d5/d55/d56/d5c/d68/de5/dea/d79/d88/f101 x:0 0 0 2024-06-03T01:20:59.695 INFO:tasks.workunit.client.0.smithi032.stdout:5/303: rmdir da/dd 39 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:6/893: rmdir d2/d23/d30 39 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:6/894: creat d2/d23/d30/d1e/d38/f139 x:0 0 0 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/861: dwrite d3/f69 [0,4194304] 0 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/862: fdatasync d3/da6/d20/d25/f40 0 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/863: truncate d3/d9/d102/f13a 705916 0 2024-06-03T01:20:59.696 INFO:tasks.workunit.client.1.smithi044.stdout:0/864: readlink d3/da6/d20/d25/lbe 0 2024-06-03T01:20:59.697 INFO:tasks.workunit.client.0.smithi032.stdout:3/249: dwrite de/f1a [4194304,4194304] 0 2024-06-03T01:20:59.697 INFO:tasks.workunit.client.0.smithi032.stdout:8/201: dwrite f9 [4194304,4194304] 0 2024-06-03T01:20:59.699 INFO:tasks.workunit.client.1.smithi044.stdout:4/799: creat d17/d1c/d75/d7a/db8/d8d/ded/f10b x:0 0 0 2024-06-03T01:20:59.700 INFO:tasks.workunit.client.1.smithi044.stdout:4/800: chown d17/d1c/d75/d7a/db8/db9/fc8 87 1 2024-06-03T01:20:59.700 INFO:tasks.workunit.client.0.smithi032.stdout:7/263: dread dc/d12/d1b/f34 [0,4194304] 0 2024-06-03T01:20:59.702 INFO:tasks.workunit.client.0.smithi032.stdout:5/304: rename da/dd/f4e to da/dd/df/d10/d16/d18/d1a/d30/f5a 0 2024-06-03T01:20:59.702 INFO:tasks.workunit.client.0.smithi032.stdout:5/305: write da/dd/df/d10/f14 [403606,127201] 0 2024-06-03T01:20:59.702 INFO:tasks.workunit.client.0.smithi032.stdout:2/311: dwrite d5/da/f1e [0,4194304] 0 2024-06-03T01:20:59.702 INFO:tasks.workunit.client.1.smithi044.stdout:9/703: mkdir d6/dd/d51/d16/d27/d5e/db7/de3 0 2024-06-03T01:20:59.704 INFO:tasks.workunit.client.0.smithi032.stdout:3/250: getdents de/d29/d2e/d31 0 2024-06-03T01:20:59.704 INFO:tasks.workunit.client.0.smithi032.stdout:3/251: chown de/c41 10237 1 2024-06-03T01:20:59.704 INFO:tasks.workunit.client.1.smithi044.stdout:0/865: write d3/f69 [545041,15433] 0 2024-06-03T01:20:59.705 INFO:tasks.workunit.client.1.smithi044.stdout:0/866: write d3/da6/d21/f26 [995934,76456] 0 2024-06-03T01:20:59.707 INFO:tasks.workunit.client.1.smithi044.stdout:2/604: symlink db/lc0 0 2024-06-03T01:20:59.707 INFO:tasks.workunit.client.0.smithi032.stdout:8/202: mknod dc/d25/d28/d32/c46 0 2024-06-03T01:20:59.710 INFO:tasks.workunit.client.1.smithi044.stdout:5/675: link d2/d11/d12/la8 d2/d11/d12/dad/db3/ldf 0 2024-06-03T01:20:59.711 INFO:tasks.workunit.client.0.smithi032.stdout:5/306: creat da/dd/df/d10/d16/d18/d1a/d30/f5b x:0 0 0 2024-06-03T01:20:59.711 INFO:tasks.workunit.client.0.smithi032.stdout:5/307: write da/dd/df/d10/d16/d18/d1a/d30/f5a [642056,43154] 0 2024-06-03T01:20:59.712 INFO:tasks.workunit.client.1.smithi044.stdout:8/781: dwrite d5/d55/d56/d5c/f72 [0,4194304] 0 2024-06-03T01:20:59.712 INFO:tasks.workunit.client.1.smithi044.stdout:8/782: truncate d5/d16/d1d/f31 4907914 0 2024-06-03T01:20:59.712 INFO:tasks.workunit.client.1.smithi044.stdout:8/783: getdents d5/d55/d56/d5c/d68/de5/dea/de2 0 2024-06-03T01:20:59.712 INFO:tasks.workunit.client.1.smithi044.stdout:8/784: fdatasync d5/d16/d81/d9b/fc1 0 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.0.smithi032.stdout:1/152: dwrite d18/f22 [0,4194304] 0 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.0.smithi032.stdout:2/312: mkdir d5/da/d1a/d2a/d49/d67 0 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.0.smithi032.stdout:2/313: chown d5/da/d1a/d2a/d49/l5a 1288243 1 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.0.smithi032.stdout:2/314: creat d5/da/d1a/f68 x:0 0 0 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/636: mkdir df/d13/d1b/d4b/d69/dd1 0 2024-06-03T01:20:59.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/637: fdatasync f9 0 2024-06-03T01:20:59.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/638: creat df/d1e/d2a/d70/fd2 x:0 0 0 2024-06-03T01:20:59.714 INFO:tasks.workunit.client.0.smithi032.stdout:3/252: mknod de/d29/c4c 0 2024-06-03T01:20:59.715 INFO:tasks.workunit.client.1.smithi044.stdout:1/900: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/f133 x:0 0 0 2024-06-03T01:20:59.718 INFO:tasks.workunit.client.1.smithi044.stdout:6/895: creat d2/d23/d55/d5f/d99/dcc/de0/f13a x:0 0 0 2024-06-03T01:20:59.721 INFO:tasks.workunit.client.0.smithi032.stdout:3/253: rmdir de/d29/d2e/d31/d3c 39 2024-06-03T01:20:59.721 INFO:tasks.workunit.client.0.smithi032.stdout:3/254: write f9 [5158427,93242] 0 2024-06-03T01:20:59.723 INFO:tasks.workunit.client.0.smithi032.stdout:5/308: rename da/dd/df/d10/d16/d18/d35/f3a to da/dd/df/f5c 0 2024-06-03T01:20:59.723 INFO:tasks.workunit.client.1.smithi044.stdout:4/801: rmdir d17/d63/de3 0 2024-06-03T01:20:59.726 INFO:tasks.workunit.client.1.smithi044.stdout:0/867: creat d3/d5c/de3/f13e x:0 0 0 2024-06-03T01:20:59.726 INFO:tasks.workunit.client.1.smithi044.stdout:0/868: dread - d3/da6/d20/d25/f112 zero size 2024-06-03T01:20:59.726 INFO:tasks.workunit.client.0.smithi032.stdout:3/255: creat de/d29/d2e/f4d x:0 0 0 2024-06-03T01:20:59.728 INFO:tasks.workunit.client.0.smithi032.stdout:1/153: dread d18/f25 [0,4194304] 0 2024-06-03T01:20:59.729 INFO:tasks.workunit.client.1.smithi044.stdout:2/605: rename db/l37 to db/d15/d49/lc1 0 2024-06-03T01:20:59.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/901: read d0/f3f [2035426,79127] 0 2024-06-03T01:20:59.730 INFO:tasks.workunit.client.0.smithi032.stdout:3/256: dread de/d29/d2e/d31/f38 [0,4194304] 0 2024-06-03T01:20:59.730 INFO:tasks.workunit.client.1.smithi044.stdout:5/676: mkdir d2/d11/d12/de0 0 2024-06-03T01:20:59.731 INFO:tasks.workunit.client.1.smithi044.stdout:1/902: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/fb0 [239227,126720] 0 2024-06-03T01:20:59.734 INFO:tasks.workunit.client.0.smithi032.stdout:5/309: dread da/dd/df/f46 [0,4194304] 0 2024-06-03T01:20:59.734 INFO:tasks.workunit.client.0.smithi032.stdout:5/310: fdatasync da/f15 0 2024-06-03T01:20:59.734 INFO:tasks.workunit.client.0.smithi032.stdout:3/257: rename fd to de/f4e 0 2024-06-03T01:20:59.735 INFO:tasks.workunit.client.1.smithi044.stdout:3/639: mknod df/d13/d39/d3d/d43/d72/cd3 0 2024-06-03T01:20:59.735 INFO:tasks.workunit.client.1.smithi044.stdout:3/640: creat df/d13/d1b/d93/fd4 x:0 0 0 2024-06-03T01:20:59.735 INFO:tasks.workunit.client.1.smithi044.stdout:1/903: dread d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/f77 [0,4194304] 0 2024-06-03T01:20:59.736 INFO:tasks.workunit.client.0.smithi032.stdout:8/203: dwrite dc/d25/d28/d32/f36 [0,4194304] 0 2024-06-03T01:20:59.736 INFO:tasks.workunit.client.0.smithi032.stdout:8/204: chown f9 0 1 2024-06-03T01:20:59.736 INFO:tasks.workunit.client.0.smithi032.stdout:8/205: fsync dc/d14/f24 0 2024-06-03T01:20:59.736 INFO:tasks.workunit.client.0.smithi032.stdout:5/311: creat da/dd/df/d10/d16/d18/d35/f5d x:0 0 0 2024-06-03T01:20:59.736 INFO:tasks.workunit.client.0.smithi032.stdout:7/264: dwrite dc/d12/d1b/f34 [0,4194304] 0 2024-06-03T01:20:59.737 INFO:tasks.workunit.client.1.smithi044.stdout:2/606: symlink db/dd/lc2 0 2024-06-03T01:20:59.737 INFO:tasks.workunit.client.1.smithi044.stdout:2/607: chown db/d15/d49/d6c/d72/d7a/d4e/laf 3331916 1 2024-06-03T01:20:59.737 INFO:tasks.workunit.client.0.smithi032.stdout:3/258: truncate de/d29/d2e/d31/f18 3059184 0 2024-06-03T01:20:59.741 INFO:tasks.workunit.client.1.smithi044.stdout:0/869: rename d3/da6/d21/f10a to d3/d6b/de8/f13f 0 2024-06-03T01:20:59.741 INFO:tasks.workunit.client.1.smithi044.stdout:0/870: write d3/da6/fb1 [3555,13042] 0 2024-06-03T01:20:59.742 INFO:tasks.workunit.client.1.smithi044.stdout:5/677: mknod d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/ce1 0 2024-06-03T01:20:59.744 INFO:tasks.workunit.client.1.smithi044.stdout:1/904: chown d0/dd2/dfe/dc2/lc9 88420 1 2024-06-03T01:20:59.745 INFO:tasks.workunit.client.0.smithi032.stdout:1/154: dread d18/f22 [0,4194304] 0 2024-06-03T01:20:59.746 INFO:tasks.workunit.client.0.smithi032.stdout:5/312: creat da/dd/f5e x:0 0 0 2024-06-03T01:20:59.748 INFO:tasks.workunit.client.0.smithi032.stdout:7/265: link dc/d12/d1b/f2c dc/d12/d22/f52 0 2024-06-03T01:20:59.751 INFO:tasks.workunit.client.1.smithi044.stdout:6/896: dwrite d2/d23/d55/d5f/db2/fec [0,4194304] 0 2024-06-03T01:20:59.754 INFO:tasks.workunit.client.1.smithi044.stdout:0/871: dread d3/d6b/d95/dae/dd1/f12c [0,4194304] 0 2024-06-03T01:20:59.754 INFO:tasks.workunit.client.1.smithi044.stdout:4/802: dwrite d17/d1c/d75/d7a/db8/d59/fb5 [4194304,4194304] 0 2024-06-03T01:20:59.754 INFO:tasks.workunit.client.1.smithi044.stdout:4/803: write d17/d1c/d75/d7a/db8/d8d/fee [543441,32824] 0 2024-06-03T01:20:59.755 INFO:tasks.workunit.client.0.smithi032.stdout:7/266: symlink dc/l53 0 2024-06-03T01:20:59.755 INFO:tasks.workunit.client.0.smithi032.stdout:7/267: write dc/d12/d2a/f40 [853978,1951] 0 2024-06-03T01:20:59.755 INFO:tasks.workunit.client.0.smithi032.stdout:7/268: creat dc/f54 x:0 0 0 2024-06-03T01:20:59.755 INFO:tasks.workunit.client.0.smithi032.stdout:8/206: getdents dc/d25/d28/d32 0 2024-06-03T01:20:59.755 INFO:tasks.workunit.client.1.smithi044.stdout:2/608: mknod db/d1b/d3f/cc3 0 2024-06-03T01:20:59.756 INFO:tasks.workunit.client.1.smithi044.stdout:5/678: dread d2/f2e [0,4194304] 0 2024-06-03T01:20:59.759 INFO:tasks.workunit.client.0.smithi032.stdout:8/207: creat dc/d25/f47 x:0 0 0 2024-06-03T01:20:59.759 INFO:tasks.workunit.client.1.smithi044.stdout:4/804: dread d17/f7c [0,4194304] 0 2024-06-03T01:20:59.760 INFO:tasks.workunit.client.1.smithi044.stdout:6/897: creat d2/d23/d55/d5f/db2/dca/f13b x:0 0 0 2024-06-03T01:20:59.760 INFO:tasks.workunit.client.1.smithi044.stdout:6/898: chown d2/d23/df4/ff9 1 1 2024-06-03T01:20:59.760 INFO:tasks.workunit.client.1.smithi044.stdout:6/899: stat d2/f134 0 2024-06-03T01:20:59.763 INFO:tasks.workunit.client.0.smithi032.stdout:3/259: getdents de/d29/d2e 0 2024-06-03T01:20:59.767 INFO:tasks.workunit.client.1.smithi044.stdout:0/872: dread d3/da6/f3e [0,4194304] 0 2024-06-03T01:20:59.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/609: unlink db/d15/fa4 0 2024-06-03T01:20:59.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/610: truncate db/d15/d49/f8e 380469 0 2024-06-03T01:20:59.772 INFO:tasks.workunit.client.0.smithi032.stdout:3/260: symlink de/d29/d2e/d31/d3c/l4f 0 2024-06-03T01:20:59.773 INFO:tasks.workunit.client.1.smithi044.stdout:3/641: dwrite df/d1e/d78/da4/fbf [0,4194304] 0 2024-06-03T01:20:59.773 INFO:tasks.workunit.client.1.smithi044.stdout:3/642: write df/d13/d6d/d9a/fcc [731829,43699] 0 2024-06-03T01:20:59.774 INFO:tasks.workunit.client.0.smithi032.stdout:3/261: rmdir de/d29/d2e/d31/d3c 39 2024-06-03T01:20:59.778 INFO:tasks.workunit.client.0.smithi032.stdout:8/208: write dc/fd [3894388,111159] 0 2024-06-03T01:20:59.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/873: dread d3/d9/d36/f43 [0,4194304] 0 2024-06-03T01:20:59.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/874: chown d3/d6b/d95/dae/cb9 950311 1 2024-06-03T01:20:59.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/875: chown d3/da6/d21/ce1 436 1 2024-06-03T01:20:59.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/876: readlink d3/da6/d21/d94/l9f 0 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/313: dwrite da/f15 [0,4194304] 0 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/314: chown da/dd/df/d10/d16/d18/f1d 375480 1 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/315: chown da/f15 871413057 1 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/316: fsync da/dd/df/d10/d16/d18/f27 0 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/317: chown da/dd/df/d10/d16/d18/d1a/c54 0 1 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:8/209: truncate dc/d14/f22 3015125 0 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:8/210: dread - dc/d14/f2d zero size 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:8/211: dread - dc/d14/f2d zero size 2024-06-03T01:20:59.781 INFO:tasks.workunit.client.0.smithi032.stdout:8/212: dread - dc/d25/d28/d3c/f42 zero size 2024-06-03T01:20:59.784 INFO:tasks.workunit.client.0.smithi032.stdout:0/259: sync 2024-06-03T01:20:59.784 INFO:tasks.workunit.client.0.smithi032.stdout:5/318: truncate da/dd/df/f5c 490314 0 2024-06-03T01:20:59.784 INFO:tasks.workunit.client.0.smithi032.stdout:5/319: readlink da/dd/df/d10/l1f 0 2024-06-03T01:20:59.784 INFO:tasks.workunit.client.0.smithi032.stdout:5/320: creat da/dd/df/d10/d16/d18/d1a/d30/f5f x:0 0 0 2024-06-03T01:20:59.785 INFO:tasks.workunit.client.0.smithi032.stdout:5/321: stat da/dd/df/d10/d16/d18/d35/d44/f4f 0 2024-06-03T01:20:59.786 INFO:tasks.workunit.client.1.smithi044.stdout:2/611: unlink db/d1b/d2a/d52/d54/faa 0 2024-06-03T01:20:59.786 INFO:tasks.workunit.client.1.smithi044.stdout:2/612: readlink db/d1b/d5d/l65 0 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.1.smithi044.stdout:3/643: symlink df/d1e/d2a/ld5 0 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/213: creat dc/d25/d28/d33/f48 x:0 0 0 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/214: fsync dc/d25/d28/d32/f36 0 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/215: fdatasync dc/de/f11 0 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/216: chown dc/d25/d28/d32/c3e 0 1 2024-06-03T01:20:59.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/217: readlink dc/l30 0 2024-06-03T01:20:59.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/269: dwrite fa [0,4194304] 0 2024-06-03T01:20:59.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/270: fsync dc/d12/d22/f47 0 2024-06-03T01:20:59.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/271: chown dc/d12/d22/c23 16 1 2024-06-03T01:20:59.790 INFO:tasks.workunit.client.1.smithi044.stdout:0/877: dread d3/d9/f1d [0,4194304] 0 2024-06-03T01:20:59.790 INFO:tasks.workunit.client.1.smithi044.stdout:5/679: dwrite d2/d11/d12/dad/fbf [0,4194304] 0 2024-06-03T01:20:59.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/900: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/f81 [0,4194304] 0 2024-06-03T01:20:59.791 INFO:tasks.workunit.client.0.smithi032.stdout:0/260: rename d1/da/d10/f32 to d1/d46/f52 0 2024-06-03T01:20:59.792 INFO:tasks.workunit.client.1.smithi044.stdout:8/785: truncate d5/d55/d56/d5c/f72 2363454 0 2024-06-03T01:20:59.794 INFO:tasks.workunit.client.0.smithi032.stdout:3/262: dwrite de/d29/d2e/d31/d3c/f44 [0,4194304] 0 2024-06-03T01:20:59.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/878: rmdir d3/d6b/d95/db3 39 2024-06-03T01:20:59.797 INFO:tasks.workunit.client.0.smithi032.stdout:8/218: link l3 dc/d25/l49 0 2024-06-03T01:20:59.800 INFO:tasks.workunit.client.1.smithi044.stdout:6/901: mknod d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/c13c 0 2024-06-03T01:20:59.801 INFO:tasks.workunit.client.0.smithi032.stdout:1/155: dwrite d18/f22 [0,4194304] 0 2024-06-03T01:20:59.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/272: mkdir dc/d12/d55 0 2024-06-03T01:20:59.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/273: creat dc/d12/d2a/d2f/f56 x:0 0 0 2024-06-03T01:20:59.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/274: stat dc 0 2024-06-03T01:20:59.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/275: creat dc/d12/d24/f57 x:0 0 0 2024-06-03T01:20:59.805 INFO:tasks.workunit.client.1.smithi044.stdout:0/879: link d3/d9/d36/cb8 d3/d9/d36/d82/c140 0 2024-06-03T01:20:59.807 INFO:tasks.workunit.client.1.smithi044.stdout:8/786: getdents d5/d16/d81/de3/d58/d89 0 2024-06-03T01:20:59.807 INFO:tasks.workunit.client.1.smithi044.stdout:5/680: dread d2/d11/d1e/d21/d24/d93/d9d/d28/d66/f8e [0,4194304] 0 2024-06-03T01:20:59.808 INFO:tasks.workunit.client.1.smithi044.stdout:0/880: mkdir d3/d6b/d111/d141 0 2024-06-03T01:20:59.809 INFO:tasks.workunit.client.0.smithi032.stdout:0/261: rename d1/d47 to d1/da/d10/d53 0 2024-06-03T01:20:59.809 INFO:tasks.workunit.client.0.smithi032.stdout:0/262: dread - d1/da/d10/d29/f45 zero size 2024-06-03T01:20:59.809 INFO:tasks.workunit.client.0.smithi032.stdout:0/263: chown d1/da/d28/c3f 951354491 1 2024-06-03T01:20:59.812 INFO:tasks.workunit.client.0.smithi032.stdout:3/263: mkdir de/d29/d2e/d31/d3c/d50 0 2024-06-03T01:20:59.815 INFO:tasks.workunit.client.1.smithi044.stdout:8/787: unlink d5/d16/d81/d9b/ld6 0 2024-06-03T01:20:59.817 INFO:tasks.workunit.client.1.smithi044.stdout:5/681: rename d2/d11/d12/l1f to d2/d11/d1e/d21/d24/d93/d9d/d28/le2 0 2024-06-03T01:20:59.819 INFO:tasks.workunit.client.1.smithi044.stdout:8/788: mknod d5/d16/d81/de3/d58/d89/dbf/c102 0 2024-06-03T01:20:59.821 INFO:tasks.workunit.client.1.smithi044.stdout:5/682: mknod d2/d11/d12/d2b/d4a/ce3 0 2024-06-03T01:20:59.821 INFO:tasks.workunit.client.1.smithi044.stdout:5/683: dread - d2/d11/d12/d2b/d44/f82 zero size 2024-06-03T01:20:59.821 INFO:tasks.workunit.client.1.smithi044.stdout:5/684: rename d2 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/de4 22 2024-06-03T01:20:59.822 INFO:tasks.workunit.client.1.smithi044.stdout:8/789: dread d5/d16/d81/de3/f30 [0,4194304] 0 2024-06-03T01:20:59.822 INFO:tasks.workunit.client.1.smithi044.stdout:8/790: read - d5/d16/d81/d9b/fc1 zero size 2024-06-03T01:20:59.823 INFO:tasks.workunit.client.0.smithi032.stdout:8/219: write dc/fd [4993575,71256] 0 2024-06-03T01:20:59.823 INFO:tasks.workunit.client.0.smithi032.stdout:8/220: dread - dc/d14/f2d zero size 2024-06-03T01:20:59.823 INFO:tasks.workunit.client.0.smithi032.stdout:5/322: dwrite da/dd/df/f33 [0,4194304] 0 2024-06-03T01:20:59.825 INFO:tasks.workunit.client.0.smithi032.stdout:0/264: dread d1/f18 [4194304,4194304] 0 2024-06-03T01:20:59.826 INFO:tasks.workunit.client.0.smithi032.stdout:5/323: symlink da/dd/df/d10/d16/d18/d1a/d30/d58/l60 0 2024-06-03T01:20:59.827 INFO:tasks.workunit.client.0.smithi032.stdout:5/324: chown da/dd/df/f46 120381272 1 2024-06-03T01:20:59.827 INFO:tasks.workunit.client.0.smithi032.stdout:5/325: truncate da/dd/df/d10/f39 2704057 0 2024-06-03T01:20:59.827 INFO:tasks.workunit.client.1.smithi044.stdout:0/881: dread d3/da6/d21/f76 [0,4194304] 0 2024-06-03T01:20:59.828 INFO:tasks.workunit.client.1.smithi044.stdout:8/791: creat d5/d55/d56/d5c/dee/f103 x:0 0 0 2024-06-03T01:20:59.828 INFO:tasks.workunit.client.1.smithi044.stdout:5/685: truncate d2/d11/d12/d2b/d4a/d57/f76 2577237 0 2024-06-03T01:20:59.828 INFO:tasks.workunit.client.0.smithi032.stdout:5/326: mknod da/dd/df/d10/d16/d18/d35/c61 0 2024-06-03T01:20:59.830 INFO:tasks.workunit.client.0.smithi032.stdout:5/327: symlink da/dd/df/d10/d16/d18/d1a/d30/d58/l62 0 2024-06-03T01:20:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:8/792: getdents d5/d55/d56/d5c/d68/de5/dea/d79 0 2024-06-03T01:20:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:8/793: fsync d5/d16/d1d/fdf 0 2024-06-03T01:20:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:8/794: truncate d5/f25 4780338 0 2024-06-03T01:20:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:8/795: creat d5/d55/d56/d5c/d68/de5/f104 x:0 0 0 2024-06-03T01:20:59.836 INFO:tasks.workunit.client.0.smithi032.stdout:0/265: write d1/da/d28/f33 [1952860,47196] 0 2024-06-03T01:20:59.837 INFO:tasks.workunit.client.0.smithi032.stdout:0/266: rmdir d1/da/d10 39 2024-06-03T01:20:59.838 INFO:tasks.workunit.client.1.smithi044.stdout:8/796: creat d5/d16/d1d/d59/f105 x:0 0 0 2024-06-03T01:20:59.838 INFO:tasks.workunit.client.0.smithi032.stdout:0/267: symlink d1/da/d28/l54 0 2024-06-03T01:20:59.840 INFO:tasks.workunit.client.1.smithi044.stdout:8/797: creat d5/d16/d81/de3/d58/f106 x:0 0 0 2024-06-03T01:20:59.840 INFO:tasks.workunit.client.1.smithi044.stdout:8/798: creat d5/d16/d81/de3/d58/f107 x:0 0 0 2024-06-03T01:20:59.841 INFO:tasks.workunit.client.0.smithi032.stdout:0/268: dread d1/da/d10/f2d [0,4194304] 0 2024-06-03T01:20:59.842 INFO:tasks.workunit.client.1.smithi044.stdout:0/882: dread d3/d9/d55/fd4 [0,4194304] 0 2024-06-03T01:20:59.843 INFO:tasks.workunit.client.0.smithi032.stdout:0/269: rename d1/da/d10/d29/f44 to d1/d46/f55 0 2024-06-03T01:20:59.843 INFO:tasks.workunit.client.0.smithi032.stdout:8/221: dread dc/de/f11 [0,4194304] 0 2024-06-03T01:20:59.850 INFO:tasks.workunit.client.1.smithi044.stdout:6/902: dwrite d2/d23/d30/f3a [0,4194304] 0 2024-06-03T01:20:59.851 INFO:tasks.workunit.client.1.smithi044.stdout:0/883: dread d3/da6/d20/d25/f64 [0,4194304] 0 2024-06-03T01:20:59.852 INFO:tasks.workunit.client.1.smithi044.stdout:6/903: write d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/f105 [725465,124669] 0 2024-06-03T01:20:59.861 INFO:tasks.workunit.client.0.smithi032.stdout:8/222: dread fa [0,4194304] 0 2024-06-03T01:20:59.862 INFO:tasks.workunit.client.1.smithi044.stdout:0/884: read d3/d9/d36/fd7 [3963698,34599] 0 2024-06-03T01:20:59.862 INFO:tasks.workunit.client.1.smithi044.stdout:0/885: readlink d3/d9/d55/d5a/d29/l31 0 2024-06-03T01:20:59.862 INFO:tasks.workunit.client.1.smithi044.stdout:0/886: fdatasync d3/da6/d56/d62/fa1 0 2024-06-03T01:20:59.862 INFO:tasks.workunit.client.1.smithi044.stdout:0/887: dread - d3/da6/d21/d94/da3/dd8/ff1 zero size 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/223: getdents dc/d14/d31 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/224: stat dc/l30 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/225: symlink dc/de/d1e/l4a 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/226: symlink dc/d25/l4b 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/227: fsync dc/d25/d28/d3c/f3f 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/228: write dc/fd [1390704,111573] 0 2024-06-03T01:20:59.863 INFO:tasks.workunit.client.1.smithi044.stdout:2/613: dwrite db/d1b/d2a/d67/d53/d9b/f9e [0,4194304] 0 2024-06-03T01:20:59.864 INFO:tasks.workunit.client.1.smithi044.stdout:2/614: chown db/d1b/d2a/d2f/d4c/l6d 100 1 2024-06-03T01:20:59.868 INFO:tasks.workunit.client.0.smithi032.stdout:0/270: rename d1/d46/f52 to d1/da/d28/f56 0 2024-06-03T01:20:59.871 INFO:tasks.workunit.client.0.smithi032.stdout:8/229: write f9 [7218135,71399] 0 2024-06-03T01:20:59.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/615: write db/d1b/d27/f39 [3363719,126602] 0 2024-06-03T01:20:59.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/616: stat db/d1b/d2a/d52/d54/db6/cbf 0 2024-06-03T01:20:59.879 INFO:tasks.workunit.client.0.smithi032.stdout:3/264: dwrite de/f1a [0,4194304] 0 2024-06-03T01:20:59.879 INFO:tasks.workunit.client.0.smithi032.stdout:1/156: dwrite f17 [4194304,4194304] 0 2024-06-03T01:20:59.879 INFO:tasks.workunit.client.0.smithi032.stdout:7/276: dwrite dc/d12/d1b/f21 [0,4194304] 0 2024-06-03T01:20:59.881 INFO:tasks.workunit.client.1.smithi044.stdout:2/617: link l4 db/d1b/d2a/d52/lc4 0 2024-06-03T01:20:59.881 INFO:tasks.workunit.client.1.smithi044.stdout:2/618: chown db/dd 2011 1 2024-06-03T01:20:59.886 INFO:tasks.workunit.client.0.smithi032.stdout:1/157: symlink d18/l2c 0 2024-06-03T01:20:59.891 INFO:tasks.workunit.client.1.smithi044.stdout:6/904: dwrite d2/d23/d55/d5f/d99/f11f [4194304,4194304] 0 2024-06-03T01:20:59.893 INFO:tasks.workunit.client.0.smithi032.stdout:5/328: dwrite da/dd/f48 [0,4194304] 0 2024-06-03T01:20:59.895 INFO:tasks.workunit.client.1.smithi044.stdout:6/905: unlink d2/d23/d44/f94 0 2024-06-03T01:20:59.895 INFO:tasks.workunit.client.1.smithi044.stdout:8/799: dwrite d5/d20/f9e [0,4194304] 0 2024-06-03T01:20:59.895 INFO:tasks.workunit.client.1.smithi044.stdout:5/686: write d2/d11/d12/d2b/d4a/d57/f6b [129200,125893] 0 2024-06-03T01:20:59.897 INFO:tasks.workunit.client.0.smithi032.stdout:5/329: mknod da/dd/df/d10/d16/d18/d1a/d30/c63 0 2024-06-03T01:20:59.898 INFO:tasks.workunit.client.0.smithi032.stdout:5/330: mkdir da/dd/df/d10/d16/d18/d1a/d30/d51/d64 0 2024-06-03T01:20:59.901 INFO:tasks.workunit.client.0.smithi032.stdout:1/158: dread d18/f24 [0,4194304] 0 2024-06-03T01:20:59.901 INFO:tasks.workunit.client.1.smithi044.stdout:5/687: creat d2/d11/d1e/d2f/fe5 x:0 0 0 2024-06-03T01:20:59.903 INFO:tasks.workunit.client.0.smithi032.stdout:5/331: unlink da/dd/df/d10/d16/d18/d35/l47 0 2024-06-03T01:20:59.904 INFO:tasks.workunit.client.0.smithi032.stdout:1/159: chown d18/l28 3791310 1 2024-06-03T01:20:59.904 INFO:tasks.workunit.client.1.smithi044.stdout:5/688: creat d2/d11/d1e/d21/fe6 x:0 0 0 2024-06-03T01:20:59.904 INFO:tasks.workunit.client.1.smithi044.stdout:5/689: dread - d2/d11/d1e/d21/fe6 zero size 2024-06-03T01:20:59.905 INFO:tasks.workunit.client.0.smithi032.stdout:7/277: dwrite dc/f15 [0,4194304] 0 2024-06-03T01:20:59.908 INFO:tasks.workunit.client.1.smithi044.stdout:0/888: dwrite d3/d9/f86 [0,4194304] 0 2024-06-03T01:20:59.909 INFO:tasks.workunit.client.1.smithi044.stdout:5/690: getdents d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a 0 2024-06-03T01:20:59.909 INFO:tasks.workunit.client.0.smithi032.stdout:0/271: dwrite d1/f2b [0,4194304] 0 2024-06-03T01:20:59.909 INFO:tasks.workunit.client.0.smithi032.stdout:3/265: dwrite de/d29/d2e/d31/d24/f2f [0,4194304] 0 2024-06-03T01:20:59.909 INFO:tasks.workunit.client.0.smithi032.stdout:3/266: fdatasync de/f15 0 2024-06-03T01:20:59.910 INFO:tasks.workunit.client.0.smithi032.stdout:5/332: symlink da/dd/df/d10/d16/l65 0 2024-06-03T01:20:59.911 INFO:tasks.workunit.client.0.smithi032.stdout:8/230: dread f9 [4194304,4194304] 0 2024-06-03T01:20:59.913 INFO:tasks.workunit.client.1.smithi044.stdout:0/889: mkdir d3/d9/d55/d5a/d29/d58/d142 0 2024-06-03T01:20:59.914 INFO:tasks.workunit.client.1.smithi044.stdout:2/619: dwrite db/f73 [0,4194304] 0 2024-06-03T01:20:59.914 INFO:tasks.workunit.client.1.smithi044.stdout:2/620: chown db/d15/d49/d6c/d72/d7a/l3d 5792589 1 2024-06-03T01:20:59.917 INFO:tasks.workunit.client.1.smithi044.stdout:5/691: rename d2/d11/d12/l23 to d2/d11/d12/le7 0 2024-06-03T01:20:59.920 INFO:tasks.workunit.client.1.smithi044.stdout:0/890: write d3/da6/f3b [8504,70935] 0 2024-06-03T01:20:59.920 INFO:tasks.workunit.client.1.smithi044.stdout:8/800: dwrite d5/d55/d56/f8a [0,4194304] 0 2024-06-03T01:20:59.922 INFO:tasks.workunit.client.1.smithi044.stdout:2/621: mknod db/d1b/d2a/d52/d54/cc5 0 2024-06-03T01:20:59.924 INFO:tasks.workunit.client.1.smithi044.stdout:6/906: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/fe9 [0,4194304] 0 2024-06-03T01:20:59.929 INFO:tasks.workunit.client.1.smithi044.stdout:0/891: mknod d3/da6/d21/d94/da3/c143 0 2024-06-03T01:20:59.929 INFO:tasks.workunit.client.1.smithi044.stdout:0/892: read d3/f85 [761196,47403] 0 2024-06-03T01:20:59.931 INFO:tasks.workunit.client.1.smithi044.stdout:8/801: rename d5/d55/d56/d5c/d68/de5/dea/d79/f100 to d5/d55/d56/d5c/d68/de5/dea/de2/f108 0 2024-06-03T01:20:59.934 INFO:tasks.workunit.client.1.smithi044.stdout:2/622: creat db/d1b/d2a/d52/d54/db6/fc6 x:0 0 0 2024-06-03T01:20:59.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/267: symlink de/d29/l51 0 2024-06-03T01:20:59.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/268: creat de/d29/f52 x:0 0 0 2024-06-03T01:20:59.937 INFO:tasks.workunit.client.0.smithi032.stdout:0/272: mknod d1/da/d28/d3b/d50/c57 0 2024-06-03T01:20:59.938 INFO:tasks.workunit.client.0.smithi032.stdout:1/160: mkdir d18/d2d 0 2024-06-03T01:20:59.940 INFO:tasks.workunit.client.1.smithi044.stdout:7/741: sync 2024-06-03T01:20:59.940 INFO:tasks.workunit.client.1.smithi044.stdout:9/704: sync 2024-06-03T01:20:59.940 INFO:tasks.workunit.client.0.smithi032.stdout:3/269: mkdir de/d29/d2e/d31/d23/d53 0 2024-06-03T01:20:59.942 INFO:tasks.workunit.client.0.smithi032.stdout:0/273: truncate d1/d46/f48 1454011 0 2024-06-03T01:20:59.945 INFO:tasks.workunit.client.1.smithi044.stdout:0/893: creat d3/d9/d55/d7b/dfc/d10c/d13d/f144 x:0 0 0 2024-06-03T01:20:59.947 INFO:tasks.workunit.client.1.smithi044.stdout:8/802: mkdir d5/d16/d1d/d32/d109 0 2024-06-03T01:20:59.947 INFO:tasks.workunit.client.1.smithi044.stdout:8/803: chown d5/fe4 1 1 2024-06-03T01:20:59.947 INFO:tasks.workunit.client.0.smithi032.stdout:3/270: creat de/d29/d2e/d31/d3c/f54 x:0 0 0 2024-06-03T01:20:59.947 INFO:tasks.workunit.client.0.smithi032.stdout:3/271: write de/d29/d2e/d31/f32 [1518616,27617] 0 2024-06-03T01:20:59.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/272: write de/d29/f52 [430501,51657] 0 2024-06-03T01:20:59.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/273: fsync de/d29/f3f 0 2024-06-03T01:20:59.948 INFO:tasks.workunit.client.0.smithi032.stdout:5/333: dwrite da/dd/df/d10/d16/d18/f27 [0,4194304] 0 2024-06-03T01:20:59.948 INFO:tasks.workunit.client.0.smithi032.stdout:0/274: rmdir d1/da/d10/d53 39 2024-06-03T01:20:59.949 INFO:tasks.workunit.client.1.smithi044.stdout:2/623: symlink db/d1b/d2a/lc7 0 2024-06-03T01:20:59.952 INFO:tasks.workunit.client.0.smithi032.stdout:1/161: rename d18/f24 to d18/d29/f2e 0 2024-06-03T01:20:59.952 INFO:tasks.workunit.client.1.smithi044.stdout:9/705: creat d6/dd/d51/d16/d5f/ddd/d55/dae/fe4 x:0 0 0 2024-06-03T01:20:59.952 INFO:tasks.workunit.client.1.smithi044.stdout:9/706: chown d6/dd/d51/d16/d27/d5e/d71/d89 8 1 2024-06-03T01:20:59.953 INFO:tasks.workunit.client.1.smithi044.stdout:2/624: mkdir db/d15/d49/d82/dc8 0 2024-06-03T01:20:59.954 INFO:tasks.workunit.client.1.smithi044.stdout:6/907: dwrite d2/d23/d30/d1e/d38/f139 [0,4194304] 0 2024-06-03T01:20:59.955 INFO:tasks.workunit.client.1.smithi044.stdout:2/625: mkdir db/d1b/d5d/da7/dc9 0 2024-06-03T01:20:59.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/275: rename d1/f40 to d1/da/d28/d3b/d50/f58 0 2024-06-03T01:20:59.958 INFO:tasks.workunit.client.1.smithi044.stdout:2/626: mknod db/d15/d49/d6c/d72/cca 0 2024-06-03T01:20:59.959 INFO:tasks.workunit.client.1.smithi044.stdout:2/627: chown db/d15/f70 8598636 1 2024-06-03T01:20:59.959 INFO:tasks.workunit.client.1.smithi044.stdout:2/628: read db/d15/d49/d6c/fa3 [850558,55250] 0 2024-06-03T01:20:59.960 INFO:tasks.workunit.client.1.smithi044.stdout:2/629: fdatasync db/d1b/d2a/d2f/d4c/f77 0 2024-06-03T01:20:59.960 INFO:tasks.workunit.client.1.smithi044.stdout:2/630: truncate db/d1b/d27/f32 2340346 0 2024-06-03T01:20:59.960 INFO:tasks.workunit.client.1.smithi044.stdout:2/631: dread - db/dd/fb7 zero size 2024-06-03T01:20:59.960 INFO:tasks.workunit.client.1.smithi044.stdout:2/632: truncate db/d1b/d2a/d67/d53/f93 700270 0 2024-06-03T01:20:59.961 INFO:tasks.workunit.client.0.smithi032.stdout:7/278: dwrite dc/d12/d1b/f34 [0,4194304] 0 2024-06-03T01:20:59.961 INFO:tasks.workunit.client.1.smithi044.stdout:0/894: write d3/d9/d36/fb4 [301850,65714] 0 2024-06-03T01:20:59.965 INFO:tasks.workunit.client.1.smithi044.stdout:2/633: getdents db/d15/d49/d6c/d72/d7a/d4e 0 2024-06-03T01:20:59.970 INFO:tasks.workunit.client.1.smithi044.stdout:0/895: creat d3/d6b/d95/dae/dd1/f145 x:0 0 0 2024-06-03T01:20:59.970 INFO:tasks.workunit.client.1.smithi044.stdout:0/896: fdatasync d3/da6/fb6 0 2024-06-03T01:20:59.971 INFO:tasks.workunit.client.0.smithi032.stdout:3/274: write de/f4e [3080164,82828] 0 2024-06-03T01:20:59.972 INFO:tasks.workunit.client.1.smithi044.stdout:0/897: mkdir d3/d9/d55/d5a/d146 0 2024-06-03T01:20:59.973 INFO:tasks.workunit.client.0.smithi032.stdout:6/221: sync 2024-06-03T01:20:59.973 INFO:tasks.workunit.client.0.smithi032.stdout:4/253: sync 2024-06-03T01:20:59.975 INFO:tasks.workunit.client.0.smithi032.stdout:8/231: dwrite dc/fd [0,4194304] 0 2024-06-03T01:20:59.975 INFO:tasks.workunit.client.1.smithi044.stdout:5/692: dwrite d2/d11/d1e/d2f/f42 [0,4194304] 0 2024-06-03T01:20:59.976 INFO:tasks.workunit.client.0.smithi032.stdout:6/222: rename d8/d11/l13 to d8/d11/l48 0 2024-06-03T01:20:59.976 INFO:tasks.workunit.client.0.smithi032.stdout:6/223: chown f4 425387 1 2024-06-03T01:20:59.976 INFO:tasks.workunit.client.0.smithi032.stdout:6/224: fdatasync f4 0 2024-06-03T01:20:59.977 INFO:tasks.workunit.client.0.smithi032.stdout:4/254: mkdir d4/db/da/d2d/d27/d53 0 2024-06-03T01:20:59.977 INFO:tasks.workunit.client.0.smithi032.stdout:4/255: chown d4/f43 815571726 1 2024-06-03T01:20:59.977 INFO:tasks.workunit.client.0.smithi032.stdout:4/256: creat d4/db/da/f54 x:0 0 0 2024-06-03T01:20:59.977 INFO:tasks.workunit.client.1.smithi044.stdout:0/898: mkdir d3/d6b/de8/d147 0 2024-06-03T01:20:59.980 INFO:tasks.workunit.client.1.smithi044.stdout:5/693: rmdir d2/d11/d12/d2b 39 2024-06-03T01:20:59.980 INFO:tasks.workunit.client.1.smithi044.stdout:5/694: readlink d2/d11/d12/le7 0 2024-06-03T01:20:59.982 INFO:tasks.workunit.client.0.smithi032.stdout:8/232: unlink dc/d14/l29 0 2024-06-03T01:20:59.983 INFO:tasks.workunit.client.1.smithi044.stdout:1/905: sync 2024-06-03T01:20:59.983 INFO:tasks.workunit.client.1.smithi044.stdout:1/906: fdatasync d0/d3/d20/d7/d13/f3d 0 2024-06-03T01:20:59.983 INFO:tasks.workunit.client.1.smithi044.stdout:3/644: sync 2024-06-03T01:20:59.983 INFO:tasks.workunit.client.1.smithi044.stdout:4/805: sync 2024-06-03T01:20:59.984 INFO:tasks.workunit.client.1.smithi044.stdout:7/742: dwrite d0/d3/d8/d95/d63/d7f/fa1 [0,4194304] 0 2024-06-03T01:20:59.985 INFO:tasks.workunit.client.0.smithi032.stdout:4/257: creat d4/db/da/d2d/f55 x:0 0 0 2024-06-03T01:20:59.986 INFO:tasks.workunit.client.0.smithi032.stdout:4/258: symlink d4/db/da/l56 0 2024-06-03T01:20:59.987 INFO:tasks.workunit.client.0.smithi032.stdout:4/259: write d4/db/ff [639114,54666] 0 2024-06-03T01:20:59.987 INFO:tasks.workunit.client.0.smithi032.stdout:4/260: fdatasync d4/db/da/d37/f48 0 2024-06-03T01:20:59.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/806: mkdir d17/d63/d10c 0 2024-06-03T01:20:59.989 INFO:tasks.workunit.client.1.smithi044.stdout:7/743: mknod d0/d3/d8/d95/d63/d89/d8b/cef 0 2024-06-03T01:20:59.989 INFO:tasks.workunit.client.0.smithi032.stdout:4/261: mkdir d4/d57 0 2024-06-03T01:20:59.990 INFO:tasks.workunit.client.0.smithi032.stdout:6/225: dread f4 [0,4194304] 0 2024-06-03T01:20:59.990 INFO:tasks.workunit.client.0.smithi032.stdout:6/226: chown d8/la 85 1 2024-06-03T01:20:59.991 INFO:tasks.workunit.client.1.smithi044.stdout:1/907: rename d0/d3/d14 to d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134 0 2024-06-03T01:20:59.992 INFO:tasks.workunit.client.1.smithi044.stdout:0/899: dread d3/d9/f22 [0,4194304] 0 2024-06-03T01:20:59.993 INFO:tasks.workunit.client.1.smithi044.stdout:1/908: truncate d0/f5 1324583 0 2024-06-03T01:20:59.994 INFO:tasks.workunit.client.0.smithi032.stdout:8/233: dread dc/de/f11 [0,4194304] 0 2024-06-03T01:20:59.994 INFO:tasks.workunit.client.0.smithi032.stdout:8/234: dread - dc/d25/d28/d3c/f42 zero size 2024-06-03T01:20:59.994 INFO:tasks.workunit.client.0.smithi032.stdout:6/227: unlink d8/f9 0 2024-06-03T01:20:59.996 INFO:tasks.workunit.client.1.smithi044.stdout:0/900: rename d3/da6/d20/d25/ld5 to d3/da6/d56/l148 0 2024-06-03T01:20:59.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/645: write df/d13/d1b/d4b/f4f [3860580,129474] 0 2024-06-03T01:20:59.997 INFO:tasks.workunit.client.0.smithi032.stdout:4/262: mkdir d4/db/da/d1d/d58 0 2024-06-03T01:20:59.997 INFO:tasks.workunit.client.0.smithi032.stdout:4/263: creat d4/db/da/f59 x:0 0 0 2024-06-03T01:20:59.998 INFO:tasks.workunit.client.1.smithi044.stdout:0/901: getdents d3/da6/d21/d94 0 2024-06-03T01:21:00.001 INFO:tasks.workunit.client.0.smithi032.stdout:8/235: mkdir dc/d25/d28/d32/d4c 0 2024-06-03T01:21:00.003 INFO:tasks.workunit.client.0.smithi032.stdout:4/264: mknod d4/db/da/c5a 0 2024-06-03T01:21:00.003 INFO:tasks.workunit.client.0.smithi032.stdout:4/265: readlink d4/l5 0 2024-06-03T01:21:00.006 INFO:tasks.workunit.client.1.smithi044.stdout:4/807: write d17/d1c/d75/d7a/db3/fdc [64333,120247] 0 2024-06-03T01:21:00.006 INFO:tasks.workunit.client.1.smithi044.stdout:4/808: creat d17/d1c/d75/d7a/db8/d8d/f10d x:0 0 0 2024-06-03T01:21:00.007 INFO:tasks.workunit.client.0.smithi032.stdout:8/236: rmdir dc/d25/d28/d32/d4c 0 2024-06-03T01:21:00.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/909: dread d0/d3/d20/d7/d13/d2d/f79 [0,4194304] 0 2024-06-03T01:21:00.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/910: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/f50 73145489 1 2024-06-03T01:21:00.009 INFO:tasks.workunit.client.0.smithi032.stdout:4/266: mknod d4/db/da/d37/c5b 0 2024-06-03T01:21:00.009 INFO:tasks.workunit.client.0.smithi032.stdout:5/334: dwrite da/dd/f48 [0,4194304] 0 2024-06-03T01:21:00.010 INFO:tasks.workunit.client.1.smithi044.stdout:4/809: symlink d17/d1c/d75/d7a/db8/d35/l10e 0 2024-06-03T01:21:00.011 INFO:tasks.workunit.client.0.smithi032.stdout:7/279: dwrite dc/d12/d2a/f43 [0,4194304] 0 2024-06-03T01:21:00.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/237: symlink dc/d25/d28/l4d 0 2024-06-03T01:21:00.011 INFO:tasks.workunit.client.0.smithi032.stdout:0/276: dwrite d1/fc [0,4194304] 0 2024-06-03T01:21:00.011 INFO:tasks.workunit.client.0.smithi032.stdout:3/275: dwrite f8 [0,4194304] 0 2024-06-03T01:21:00.014 INFO:tasks.workunit.client.0.smithi032.stdout:7/280: dread dc/dd/f20 [0,4194304] 0 2024-06-03T01:21:00.014 INFO:tasks.workunit.client.0.smithi032.stdout:7/281: fsync dc/f30 0 2024-06-03T01:21:00.014 INFO:tasks.workunit.client.0.smithi032.stdout:7/282: chown dc/d12/d55 1430 1 2024-06-03T01:21:00.015 INFO:tasks.workunit.client.1.smithi044.stdout:8/804: dwrite d5/d16/d1d/d59/f6f [0,4194304] 0 2024-06-03T01:21:00.016 INFO:tasks.workunit.client.1.smithi044.stdout:1/911: getdents d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80 0 2024-06-03T01:21:00.016 INFO:tasks.workunit.client.0.smithi032.stdout:1/162: dwrite d18/d29/f2e [0,4194304] 0 2024-06-03T01:21:00.018 INFO:tasks.workunit.client.1.smithi044.stdout:4/810: creat d17/d63/d10c/f10f x:0 0 0 2024-06-03T01:21:00.019 INFO:tasks.workunit.client.1.smithi044.stdout:4/811: creat d17/d1c/d75/d7a/db3/de9/f110 x:0 0 0 2024-06-03T01:21:00.022 INFO:tasks.workunit.client.1.smithi044.stdout:8/805: mkdir d5/d16/d81/d10a 0 2024-06-03T01:21:00.027 INFO:tasks.workunit.client.1.smithi044.stdout:1/912: mknod d0/d3/d20/d7/c135 0 2024-06-03T01:21:00.029 INFO:tasks.workunit.client.1.smithi044.stdout:4/812: rename d17/d1c/d75/d7a/db8/d6a/lfc to d17/d63/d10c/l111 0 2024-06-03T01:21:00.031 INFO:tasks.workunit.client.1.smithi044.stdout:1/913: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d80/f136 x:0 0 0 2024-06-03T01:21:00.032 INFO:tasks.workunit.client.0.smithi032.stdout:3/276: dread de/d29/d2e/d31/d24/f2f [0,4194304] 0 2024-06-03T01:21:00.032 INFO:tasks.workunit.client.0.smithi032.stdout:3/277: write de/d29/f52 [650341,69601] 0 2024-06-03T01:21:00.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:20:59 smithi032 ceph-mon[29360]: mgrmap e23: smithi044.wszjlw(active, since 3s) 2024-06-03T01:21:00.035 INFO:tasks.workunit.client.1.smithi044.stdout:5/695: dwrite d2/d11/d12/d2b/d44/f5c [0,4194304] 0 2024-06-03T01:21:00.035 INFO:tasks.workunit.client.1.smithi044.stdout:5/696: chown d2/d11/d1e/d3b/da6/fd6 8878535 1 2024-06-03T01:21:00.035 INFO:tasks.workunit.client.1.smithi044.stdout:7/744: dwrite d0/d3/f47 [0,4194304] 0 2024-06-03T01:21:00.037 INFO:tasks.workunit.client.1.smithi044.stdout:7/745: truncate d0/d3/d8/d95/d63/d7f/d9c/d6e/faa 2808962 0 2024-06-03T01:21:00.038 INFO:tasks.workunit.client.1.smithi044.stdout:3/646: dwrite df/d13/d39/d3d/f8b [0,4194304] 0 2024-06-03T01:21:00.039 INFO:tasks.workunit.client.0.smithi032.stdout:6/228: dwrite d8/d11/d20/d31/f45 [0,4194304] 0 2024-06-03T01:21:00.039 INFO:tasks.workunit.client.0.smithi032.stdout:6/229: stat d8/d11/d20/d2b/d39/d19/f1d 0 2024-06-03T01:21:00.040 INFO:tasks.workunit.client.1.smithi044.stdout:3/647: write df/d13/d1b/d4b/d69/f76 [914249,87890] 0 2024-06-03T01:21:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:3/648: write df/f11 [602306,38353] 0 2024-06-03T01:21:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:3/649: readlink df/d1e/d78/da4/lbb 0 2024-06-03T01:21:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:3/650: write df/d1e/d3a/f87 [18480,40159] 0 2024-06-03T01:21:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:0/902: dwrite d3/f91 [4194304,4194304] 0 2024-06-03T01:21:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:0/903: dread - d3/d6b/d95/dae/dd1/f145 zero size 2024-06-03T01:21:00.046 INFO:tasks.workunit.client.1.smithi044.stdout:0/904: symlink d3/da6/d56/d62/de7/l149 0 2024-06-03T01:21:00.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/651: dread df/d13/d1b/f73 [0,4194304] 0 2024-06-03T01:21:00.049 INFO:tasks.workunit.client.1.smithi044.stdout:3/652: creat df/d13/d6d/d9a/dad/fd6 x:0 0 0 2024-06-03T01:21:00.049 INFO:tasks.workunit.client.1.smithi044.stdout:3/653: dread - df/d13/d6d/d9a/dad/fd6 zero size 2024-06-03T01:21:00.050 INFO:tasks.workunit.client.1.smithi044.stdout:8/806: dwrite d5/d16/f6e [0,4194304] 0 2024-06-03T01:21:00.051 INFO:tasks.workunit.client.1.smithi044.stdout:3/654: mkdir df/d13/d39/d3d/d43/d71/dd7 0 2024-06-03T01:21:00.051 INFO:tasks.workunit.client.1.smithi044.stdout:3/655: fsync df/d13/d39/d3d/d6a/d9c/fbd 0 2024-06-03T01:21:00.051 INFO:tasks.workunit.client.1.smithi044.stdout:0/905: dread d3/da6/fb1 [0,4194304] 0 2024-06-03T01:21:00.051 INFO:tasks.workunit.client.1.smithi044.stdout:0/906: fdatasync d3/da6/d20/d25/f7d 0 2024-06-03T01:21:00.052 INFO:tasks.workunit.client.1.smithi044.stdout:7/746: write d0/d3/d8/d95/d63/d7f/d9c/f68 [2796087,15676] 0 2024-06-03T01:21:00.052 INFO:tasks.workunit.client.1.smithi044.stdout:8/807: truncate d5/fe4 1168932 0 2024-06-03T01:21:00.052 INFO:tasks.workunit.client.0.smithi032.stdout:3/278: dread de/f4e [0,4194304] 0 2024-06-03T01:21:00.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/813: dwrite d17/d1c/d75/d7a/db8/f57 [0,4194304] 0 2024-06-03T01:21:00.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/814: write d17/d1c/d75/f4b [8042709,75891] 0 2024-06-03T01:21:00.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/815: write d17/d1c/d75/d7a/db8/f68 [1188561,87250] 0 2024-06-03T01:21:00.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/816: chown d17/d63/dae/cd4 20511 1 2024-06-03T01:21:00.054 INFO:tasks.workunit.client.1.smithi044.stdout:4/817: chown d17/d1c/d75/d7a/db8/d8d/l9f 205508 1 2024-06-03T01:21:00.054 INFO:tasks.workunit.client.0.smithi032.stdout:8/238: mkdir dc/de/d1e/d43/d4e 0 2024-06-03T01:21:00.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/373: sync 2024-06-03T01:21:00.055 INFO:tasks.workunit.client.1.smithi044.stdout:0/907: symlink d3/d9/d55/d5a/d29/d58/l14a 0 2024-06-03T01:21:00.055 INFO:tasks.workunit.client.0.smithi032.stdout:0/277: symlink d1/da/d10/d53/l59 0 2024-06-03T01:21:00.056 INFO:tasks.workunit.client.0.smithi032.stdout:0/278: fsync d1/f18 0 2024-06-03T01:21:00.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/283: symlink dc/d12/d22/l58 0 2024-06-03T01:21:00.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/284: chown dc/d12/d24/c3c 52 1 2024-06-03T01:21:00.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/285: dread - dc/d12/d22/d42/f50 zero size 2024-06-03T01:21:00.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/286: fsync dc/d12/d1b/f2c 0 2024-06-03T01:21:00.059 INFO:tasks.workunit.client.0.smithi032.stdout:8/239: dread dc/de/f37 [0,4194304] 0 2024-06-03T01:21:00.060 INFO:tasks.workunit.client.0.smithi032.stdout:4/267: getdents d4/db/da/d2d 0 2024-06-03T01:21:00.061 INFO:tasks.workunit.client.1.smithi044.stdout:4/818: rename d17/d1c/d75/d7a/db8/d28/lea to d17/d1c/d75/d7a/db8/d6a/l112 0 2024-06-03T01:21:00.063 INFO:tasks.workunit.client.1.smithi044.stdout:3/656: creat df/d13/d1b/d59/fd8 x:0 0 0 2024-06-03T01:21:00.064 INFO:tasks.workunit.client.1.smithi044.stdout:1/914: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/f72 [866866,14813] 0 2024-06-03T01:21:00.064 INFO:tasks.workunit.client.1.smithi044.stdout:7/747: getdents d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0 0 2024-06-03T01:21:00.065 INFO:tasks.workunit.client.1.smithi044.stdout:7/748: chown d0/d3/d8/d95/d63/d7f/d9c/c9d 5227878 1 2024-06-03T01:21:00.065 INFO:tasks.workunit.client.1.smithi044.stdout:7/749: write d0/d3/d8/d32/d33/fd6 [452408,109444] 0 2024-06-03T01:21:00.065 INFO:tasks.workunit.client.1.smithi044.stdout:7/750: dread - d0/d3/d8/d95/d63/fc1 zero size 2024-06-03T01:21:00.066 INFO:tasks.workunit.client.1.smithi044.stdout:7/751: truncate d0/d3/d8/d95/d63/d89/d8b/f97 1016063 0 2024-06-03T01:21:00.066 INFO:tasks.workunit.client.1.smithi044.stdout:7/752: getdents d0/d3/da8/de5 0 2024-06-03T01:21:00.069 INFO:tasks.workunit.client.0.smithi032.stdout:1/163: mkdir d18/d2d/d2f 0 2024-06-03T01:21:00.070 INFO:tasks.workunit.client.1.smithi044.stdout:8/808: getdents d5/d55/d74/d82 0 2024-06-03T01:21:00.070 INFO:tasks.workunit.client.1.smithi044.stdout:8/809: fdatasync d5/d55/d74/db4/dba/fc3 0 2024-06-03T01:21:00.070 INFO:tasks.workunit.client.1.smithi044.stdout:5/697: dwrite d2/fa [0,4194304] 0 2024-06-03T01:21:00.071 INFO:tasks.workunit.client.0.smithi032.stdout:0/279: dread d1/f2b [0,4194304] 0 2024-06-03T01:21:00.071 INFO:tasks.workunit.client.0.smithi032.stdout:0/280: chown d1/f6 6614 1 2024-06-03T01:21:00.071 INFO:tasks.workunit.client.0.smithi032.stdout:0/281: write d1/da/d28/f33 [1878811,73869] 0 2024-06-03T01:21:00.071 INFO:tasks.workunit.client.0.smithi032.stdout:0/282: chown d1/da/d10/f19 116101053 1 2024-06-03T01:21:00.072 INFO:tasks.workunit.client.1.smithi044.stdout:4/819: symlink d17/d1c/d75/d7a/l113 0 2024-06-03T01:21:00.074 INFO:tasks.workunit.client.1.smithi044.stdout:3/657: chown df/d13/f1d 1 1 2024-06-03T01:21:00.074 INFO:tasks.workunit.client.1.smithi044.stdout:3/658: fdatasync df/d13/d1b/d56/f7f 0 2024-06-03T01:21:00.074 INFO:tasks.workunit.client.1.smithi044.stdout:3/659: chown df/d13/d1b/d4b/f4f 362123727 1 2024-06-03T01:21:00.074 INFO:tasks.workunit.client.1.smithi044.stdout:7/753: mknod d0/d3/d8/d95/cf0 0 2024-06-03T01:21:00.077 INFO:tasks.workunit.client.0.smithi032.stdout:0/283: dread d1/d46/f48 [0,4194304] 0 2024-06-03T01:21:00.078 INFO:tasks.workunit.client.1.smithi044.stdout:0/908: dwrite d3/f14 [0,4194304] 0 2024-06-03T01:21:00.079 INFO:tasks.workunit.client.1.smithi044.stdout:8/810: rename d5/d16/d81/de3/d58/dd5/ff4 to d5/d20/db3/f10b 0 2024-06-03T01:21:00.079 INFO:tasks.workunit.client.1.smithi044.stdout:4/820: rename d17/d1c/d70/dfb to d17/d1c/d70/dfb/dff/d114 22 2024-06-03T01:21:00.079 INFO:tasks.workunit.client.1.smithi044.stdout:8/811: truncate d5/d16/f24 2047621 0 2024-06-03T01:21:00.080 INFO:tasks.workunit.client.1.smithi044.stdout:4/821: write d17/d1c/d75/d7a/db8/d59/f82 [3997160,7277] 0 2024-06-03T01:21:00.080 INFO:tasks.workunit.client.1.smithi044.stdout:0/909: dread d3/da6/fb1 [0,4194304] 0 2024-06-03T01:21:00.081 INFO:tasks.workunit.client.0.smithi032.stdout:6/230: rename d8/d11/d20/f44 to d8/d11/f49 0 2024-06-03T01:21:00.081 INFO:tasks.workunit.client.0.smithi032.stdout:6/231: chown d8/d11/d28/c32 3123344 1 2024-06-03T01:21:00.082 INFO:tasks.workunit.client.0.smithi032.stdout:3/279: symlink de/d29/d2e/d31/d24/l55 0 2024-06-03T01:21:00.082 INFO:tasks.workunit.client.0.smithi032.stdout:3/280: readlink de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:00.083 INFO:tasks.workunit.client.1.smithi044.stdout:1/915: dwrite d0/dd2/dfe/dbe/d9e/fa9 [0,4194304] 0 2024-06-03T01:21:00.084 INFO:tasks.workunit.client.0.smithi032.stdout:9/374: mknod d0/d23/d24/d26/d41/c81 0 2024-06-03T01:21:00.086 INFO:tasks.workunit.client.0.smithi032.stdout:7/287: symlink dc/d12/d2a/d2f/l59 0 2024-06-03T01:21:00.087 INFO:tasks.workunit.client.0.smithi032.stdout:7/288: dread dc/d12/d24/d33/f4c [0,4194304] 0 2024-06-03T01:21:00.088 INFO:tasks.workunit.client.0.smithi032.stdout:7/289: chown dc/d12/d24/d33/f4c 17897 1 2024-06-03T01:21:00.088 INFO:tasks.workunit.client.0.smithi032.stdout:7/290: dread - dc/d12/d2a/d2f/f44 zero size 2024-06-03T01:21:00.089 INFO:tasks.workunit.client.0.smithi032.stdout:1/164: truncate f15 2964154 0 2024-06-03T01:21:00.093 INFO:tasks.workunit.client.0.smithi032.stdout:3/281: mkdir de/d29/d2e/d31/d3c/d50/d56 0 2024-06-03T01:21:00.094 INFO:tasks.workunit.client.0.smithi032.stdout:2/315: sync 2024-06-03T01:21:00.094 INFO:tasks.workunit.client.0.smithi032.stdout:6/232: dread f3 [4194304,4194304] 0 2024-06-03T01:21:00.094 INFO:tasks.workunit.client.0.smithi032.stdout:6/233: read d8/d11/f17 [3075959,20376] 0 2024-06-03T01:21:00.095 INFO:tasks.workunit.client.0.smithi032.stdout:6/234: creat d8/d11/d20/d2b/f4a x:0 0 0 2024-06-03T01:21:00.095 INFO:tasks.workunit.client.0.smithi032.stdout:6/235: dread - d8/d11/d20/d2b/d39/d19/f40 zero size 2024-06-03T01:21:00.095 INFO:tasks.workunit.client.0.smithi032.stdout:4/268: dwrite d4/db/da/f54 [0,4194304] 0 2024-06-03T01:21:00.095 INFO:tasks.workunit.client.0.smithi032.stdout:4/269: write d4/db/da/f49 [177991,66731] 0 2024-06-03T01:21:00.095 INFO:tasks.workunit.client.0.smithi032.stdout:8/240: symlink dc/d14/l4f 0 2024-06-03T01:21:00.097 INFO:tasks.workunit.client.0.smithi032.stdout:3/282: creat de/f57 x:0 0 0 2024-06-03T01:21:00.098 INFO:tasks.workunit.client.0.smithi032.stdout:1/165: dread fa [0,4194304] 0 2024-06-03T01:21:00.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/375: dread d0/d11/d1d/d38/f3e [0,4194304] 0 2024-06-03T01:21:00.100 INFO:tasks.workunit.client.1.smithi044.stdout:1/916: read d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/f63 [835734,60375] 0 2024-06-03T01:21:00.100 INFO:tasks.workunit.client.1.smithi044.stdout:0/910: dwrite d3/d9/d36/f48 [4194304,4194304] 0 2024-06-03T01:21:00.101 INFO:tasks.workunit.client.0.smithi032.stdout:7/291: link dc/d12/d1b/l1c dc/dd/l5a 0 2024-06-03T01:21:00.101 INFO:tasks.workunit.client.0.smithi032.stdout:7/292: write dc/d12/d24/d33/f4c [805072,99826] 0 2024-06-03T01:21:00.104 INFO:tasks.workunit.client.0.smithi032.stdout:6/236: mknod d8/c4b 0 2024-06-03T01:21:00.105 INFO:tasks.workunit.client.1.smithi044.stdout:5/698: symlink d2/d11/d1e/d3b/le8 0 2024-06-03T01:21:00.106 INFO:tasks.workunit.client.0.smithi032.stdout:2/316: dread d5/da/d2e/f32 [0,4194304] 0 2024-06-03T01:21:00.106 INFO:tasks.workunit.client.0.smithi032.stdout:2/317: stat d5/d38/f3b 0 2024-06-03T01:21:00.107 INFO:tasks.workunit.client.1.smithi044.stdout:5/699: dread d2/d11/d12/d2b/f58 [0,4194304] 0 2024-06-03T01:21:00.107 INFO:tasks.workunit.client.1.smithi044.stdout:5/700: chown d2/d11/d12/d2b/d44/l7a 138772572 1 2024-06-03T01:21:00.107 INFO:tasks.workunit.client.1.smithi044.stdout:5/701: fdatasync d2/d11/d12/f1b 0 2024-06-03T01:21:00.108 INFO:tasks.workunit.client.1.smithi044.stdout:5/702: dread - d2/d11/d1e/d21/d24/faa zero size 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:8/241: creat dc/d25/d28/d32/f50 x:0 0 0 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:9/376: write d0/f15 [3828164,84651] 0 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:9/377: write d0/d23/d25/f62 [79806,29721] 0 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/283: mknod de/d29/c58 0 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/284: creat de/d29/d2e/d31/d23/f59 x:0 0 0 2024-06-03T01:21:00.112 INFO:tasks.workunit.client.0.smithi032.stdout:7/293: dread f6 [0,4194304] 0 2024-06-03T01:21:00.113 INFO:tasks.workunit.client.1.smithi044.stdout:7/754: rename d0/d3/d8/l41 to d0/d3/d8/d95/d84/dd5/lf1 0 2024-06-03T01:21:00.113 INFO:tasks.workunit.client.1.smithi044.stdout:7/755: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/f9b zero size 2024-06-03T01:21:00.114 INFO:tasks.workunit.client.0.smithi032.stdout:1/166: chown d18/l21 8137360 1 2024-06-03T01:21:00.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/812: mknod d5/d55/d56/d5c/d68/de5/dea/de2/c10c 0 2024-06-03T01:21:00.116 INFO:tasks.workunit.client.0.smithi032.stdout:4/270: getdents d4/db/da/d2d/d27 0 2024-06-03T01:21:00.116 INFO:tasks.workunit.client.0.smithi032.stdout:4/271: fsync d4/db/da/d2d/d27/f2b 0 2024-06-03T01:21:00.117 INFO:tasks.workunit.client.0.smithi032.stdout:2/318: dread d5/da/f2c [4194304,4194304] 0 2024-06-03T01:21:00.117 INFO:tasks.workunit.client.1.smithi044.stdout:4/822: link d17/d1c/d75/d7a/db8/c105 d17/d1c/d75/d7a/db3/de9/c115 0 2024-06-03T01:21:00.117 INFO:tasks.workunit.client.0.smithi032.stdout:2/319: stat d5/da/d1a/f41 0 2024-06-03T01:21:00.119 INFO:tasks.workunit.client.1.smithi044.stdout:3/660: dread df/d13/d1b/f34 [0,4194304] 0 2024-06-03T01:21:00.120 INFO:tasks.workunit.client.0.smithi032.stdout:8/242: rmdir dc/d14/d31 39 2024-06-03T01:21:00.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/243: write dc/d25/d28/d3c/f3f [547799,1907] 0 2024-06-03T01:21:00.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/244: read - dc/d25/f47 zero size 2024-06-03T01:21:00.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/245: chown dc/d25 9463 1 2024-06-03T01:21:00.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/246: write dc/d14/f24 [1526179,44595] 0 2024-06-03T01:21:00.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/247: fdatasync dc/f15 0 2024-06-03T01:21:00.125 INFO:tasks.workunit.client.1.smithi044.stdout:1/917: mknod d0/d3/d20/d7/d13/d2d/dcd/d108/c137 0 2024-06-03T01:21:00.128 INFO:tasks.workunit.client.1.smithi044.stdout:0/911: creat d3/d6b/de8/d147/f14b x:0 0 0 2024-06-03T01:21:00.128 INFO:tasks.workunit.client.0.smithi032.stdout:9/378: rmdir d0/d23/d24/d26/d41 39 2024-06-03T01:21:00.128 INFO:tasks.workunit.client.0.smithi032.stdout:7/294: dread dc/f14 [0,4194304] 0 2024-06-03T01:21:00.129 INFO:tasks.workunit.client.1.smithi044.stdout:5/703: creat d2/d11/d12/de0/fe9 x:0 0 0 2024-06-03T01:21:00.130 INFO:tasks.workunit.client.1.smithi044.stdout:5/704: fsync d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f61 0 2024-06-03T01:21:00.133 INFO:tasks.workunit.client.0.smithi032.stdout:8/248: dread dc/d25/d28/d3c/f3f [0,4194304] 0 2024-06-03T01:21:00.133 INFO:tasks.workunit.client.0.smithi032.stdout:8/249: truncate dc/de/f37 1012690 0 2024-06-03T01:21:00.133 INFO:tasks.workunit.client.0.smithi032.stdout:1/167: mknod d18/d2d/d2f/c30 0 2024-06-03T01:21:00.133 INFO:tasks.workunit.client.1.smithi044.stdout:4/823: symlink d17/d63/dae/l116 0 2024-06-03T01:21:00.134 INFO:tasks.workunit.client.0.smithi032.stdout:4/272: mkdir d4/d5c 0 2024-06-03T01:21:00.135 INFO:tasks.workunit.client.0.smithi032.stdout:2/320: mkdir d5/da/d1a/d69 0 2024-06-03T01:21:00.139 INFO:tasks.workunit.client.1.smithi044.stdout:3/661: creat df/d13/d1b/d56/fd9 x:0 0 0 2024-06-03T01:21:00.140 INFO:tasks.workunit.client.1.smithi044.stdout:9/707: sync 2024-06-03T01:21:00.141 INFO:tasks.workunit.client.1.smithi044.stdout:6/908: sync 2024-06-03T01:21:00.141 INFO:tasks.workunit.client.1.smithi044.stdout:2/634: sync 2024-06-03T01:21:00.141 INFO:tasks.workunit.client.1.smithi044.stdout:6/909: fsync d2/d23/d55/d5f/d53/fb9 0 2024-06-03T01:21:00.141 INFO:tasks.workunit.client.1.smithi044.stdout:9/708: fdatasync d6/dd/d51/d46/f48 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/295: creat dc/d12/d1b/f5b x:0 0 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/296: creat dc/d12/d24/d33/f5c x:0 0 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:9/379: dread d0/d11/f22 [0,4194304] 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:8/250: write dc/d25/f35 [3908972,83189] 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:1/168: creat d18/d27/f31 x:0 0 0 2024-06-03T01:21:00.143 INFO:tasks.workunit.client.0.smithi032.stdout:1/169: stat d18/d2d 0 2024-06-03T01:21:00.145 INFO:tasks.workunit.client.1.smithi044.stdout:1/918: getdents d0 0 2024-06-03T01:21:00.146 INFO:tasks.workunit.client.0.smithi032.stdout:8/251: mkdir dc/d25/d28/d51 0 2024-06-03T01:21:00.146 INFO:tasks.workunit.client.1.smithi044.stdout:8/813: dwrite d5/d16/d1d/f38 [0,4194304] 0 2024-06-03T01:21:00.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/170: fsync d18/f19 0 2024-06-03T01:21:00.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/171: read d18/f1e [2472797,35951] 0 2024-06-03T01:21:00.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/285: dwrite f5 [4194304,4194304] 0 2024-06-03T01:21:00.152 INFO:tasks.workunit.client.1.smithi044.stdout:0/912: rename d3/d6b/f6f to d3/d9/d55/f14c 0 2024-06-03T01:21:00.152 INFO:tasks.workunit.client.1.smithi044.stdout:9/709: mknod d6/dd/d51/d16/ce5 0 2024-06-03T01:21:00.152 INFO:tasks.workunit.client.1.smithi044.stdout:9/710: write d6/dd/d51/d46/f9f [1037824,42567] 0 2024-06-03T01:21:00.154 INFO:tasks.workunit.client.1.smithi044.stdout:7/756: dwrite d0/d3/d8/d95/d63/d7f/f82 [0,4194304] 0 2024-06-03T01:21:00.156 INFO:tasks.workunit.client.0.smithi032.stdout:9/380: read d0/d11/d76/f7c [2558821,53388] 0 2024-06-03T01:21:00.156 INFO:tasks.workunit.client.1.smithi044.stdout:8/814: creat d5/f10d x:0 0 0 2024-06-03T01:21:00.157 INFO:tasks.workunit.client.0.smithi032.stdout:4/273: rmdir d4/db/da 39 2024-06-03T01:21:00.158 INFO:tasks.workunit.client.1.smithi044.stdout:0/913: creat d3/d9/d55/d5a/dde/f14d x:0 0 0 2024-06-03T01:21:00.160 INFO:tasks.workunit.client.1.smithi044.stdout:9/711: mkdir d6/dd/d51/d16/d27/d5e/de6 0 2024-06-03T01:21:00.162 INFO:tasks.workunit.client.0.smithi032.stdout:3/286: truncate de/f10 3154010 0 2024-06-03T01:21:00.162 INFO:tasks.workunit.client.0.smithi032.stdout:3/287: fsync de/d29/d2e/d31/f1c 0 2024-06-03T01:21:00.165 INFO:tasks.workunit.client.1.smithi044.stdout:7/757: creat d0/d3/d8/d32/ff2 x:0 0 0 2024-06-03T01:21:00.170 INFO:tasks.workunit.client.1.smithi044.stdout:6/910: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/fc9 [0,4194304] 0 2024-06-03T01:21:00.170 INFO:tasks.workunit.client.1.smithi044.stdout:6/911: readlink d2/d23/d55/d5f/d99/l11c 0 2024-06-03T01:21:00.171 INFO:tasks.workunit.client.1.smithi044.stdout:6/912: readlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/lcf 0 2024-06-03T01:21:00.171 INFO:tasks.workunit.client.1.smithi044.stdout:0/914: getdents d3/d9/d36 0 2024-06-03T01:21:00.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/274: mkdir d4/db/da/d5d 0 2024-06-03T01:21:00.175 INFO:tasks.workunit.client.1.smithi044.stdout:7/758: rename d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/f80 to d0/d3/d8/d95/d63/d89/da3/ff3 0 2024-06-03T01:21:00.176 INFO:tasks.workunit.client.0.smithi032.stdout:8/252: symlink dc/d14/l52 0 2024-06-03T01:21:00.176 INFO:tasks.workunit.client.1.smithi044.stdout:3/662: read df/f58 [3078983,53941] 0 2024-06-03T01:21:00.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/913: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/df8/f13d x:0 0 0 2024-06-03T01:21:00.178 INFO:tasks.workunit.client.0.smithi032.stdout:1/172: mkdir d18/d32 0 2024-06-03T01:21:00.179 INFO:tasks.workunit.client.1.smithi044.stdout:0/915: dread d3/da6/d20/d25/f4b [0,4194304] 0 2024-06-03T01:21:00.180 INFO:tasks.workunit.client.1.smithi044.stdout:3/663: readlink df/d13/d1b/l41 0 2024-06-03T01:21:00.181 INFO:tasks.workunit.client.0.smithi032.stdout:3/288: unlink de/c14 0 2024-06-03T01:21:00.183 INFO:tasks.workunit.client.0.smithi032.stdout:4/275: creat d4/d5c/f5e x:0 0 0 2024-06-03T01:21:00.183 INFO:tasks.workunit.client.1.smithi044.stdout:8/815: rmdir d5/d16/d81/de3/d58/dd5 0 2024-06-03T01:21:00.185 INFO:tasks.workunit.client.1.smithi044.stdout:1/919: dwrite d0/d3/d20/d7/fb1 [0,4194304] 0 2024-06-03T01:21:00.185 INFO:tasks.workunit.client.1.smithi044.stdout:1/920: creat d0/d3/d20/d7/d13/d112/f138 x:0 0 0 2024-06-03T01:21:00.186 INFO:tasks.workunit.client.1.smithi044.stdout:7/759: getdents d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c 0 2024-06-03T01:21:00.186 INFO:tasks.workunit.client.1.smithi044.stdout:7/760: readlink d0/d3/d8/d95/d63/d89/da3/d9a/lc6 0 2024-06-03T01:21:00.187 INFO:tasks.workunit.client.0.smithi032.stdout:4/276: link d4/db/da/d2d/f55 d4/db/da/d5d/f5f 0 2024-06-03T01:21:00.188 INFO:tasks.workunit.client.1.smithi044.stdout:5/705: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/fd3 [0,4194304] 0 2024-06-03T01:21:00.188 INFO:tasks.workunit.client.1.smithi044.stdout:5/706: chown d2/d11/d12/d2b/d4a/f5d 446014597 1 2024-06-03T01:21:00.190 INFO:tasks.workunit.client.0.smithi032.stdout:1/173: write f17 [5758932,103228] 0 2024-06-03T01:21:00.190 INFO:tasks.workunit.client.1.smithi044.stdout:0/916: mkdir d3/da6/dc1/dc2/de4/d14e 0 2024-06-03T01:21:00.190 INFO:tasks.workunit.client.1.smithi044.stdout:3/664: rmdir df/d13/d39 39 2024-06-03T01:21:00.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/297: dwrite dc/d12/d1b/f5b [0,4194304] 0 2024-06-03T01:21:00.191 INFO:tasks.workunit.client.0.smithi032.stdout:4/277: creat d4/db/da/d2d/d27/f60 x:0 0 0 2024-06-03T01:21:00.191 INFO:tasks.workunit.client.0.smithi032.stdout:4/278: dread - d4/f41 zero size 2024-06-03T01:21:00.193 INFO:tasks.workunit.client.0.smithi032.stdout:9/381: dwrite d0/d11/d76/f7c [0,4194304] 0 2024-06-03T01:21:00.193 INFO:tasks.workunit.client.0.smithi032.stdout:9/382: chown d0/d23/d24/f72 11793331 1 2024-06-03T01:21:00.197 INFO:tasks.workunit.client.1.smithi044.stdout:1/921: getdents d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/dae 0 2024-06-03T01:21:00.197 INFO:tasks.workunit.client.1.smithi044.stdout:7/761: creat d0/d3/ddf/ff4 x:0 0 0 2024-06-03T01:21:00.197 INFO:tasks.workunit.client.1.smithi044.stdout:7/762: dread - d0/d3/d8/d95/d63/fc1 zero size 2024-06-03T01:21:00.197 INFO:tasks.workunit.client.1.smithi044.stdout:4/824: dwrite d17/d1c/d9c/fc1 [0,4194304] 0 2024-06-03T01:21:00.198 INFO:tasks.workunit.client.1.smithi044.stdout:5/707: dread d2/d11/d12/d2b/d4a/fa5 [0,4194304] 0 2024-06-03T01:21:00.198 INFO:tasks.workunit.client.1.smithi044.stdout:5/708: write d2/d11/d1e/d21/d24/f69 [276071,19029] 0 2024-06-03T01:21:00.198 INFO:tasks.workunit.client.1.smithi044.stdout:5/709: dread - d2/d11/d12/d2b/d4a/fca zero size 2024-06-03T01:21:00.198 INFO:tasks.workunit.client.1.smithi044.stdout:5/710: chown d2/d11/d1e/d3b/da6/fd6 420837411 1 2024-06-03T01:21:00.202 INFO:tasks.workunit.client.1.smithi044.stdout:1/922: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d81/f139 x:0 0 0 2024-06-03T01:21:00.203 INFO:tasks.workunit.client.1.smithi044.stdout:1/923: write d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/f72 [579824,68331] 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.0.smithi032.stdout:4/279: dread d4/db/da/d1d/f23 [0,4194304] 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.0.smithi032.stdout:2/321: dwrite d5/da/d2e/f32 [0,4194304] 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.1.smithi044.stdout:2/635: dwrite db/d1b/d2a/d67/d53/d9b/f9e [0,4194304] 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/917: dread d3/da6/d56/d62/fa1 [0,4194304] 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/918: readlink d3/d9/d55/d5a/l103 0 2024-06-03T01:21:00.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/919: fdatasync d3/da6/d20/d25/f40 0 2024-06-03T01:21:00.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/636: write db/d15/d49/d6c/fa3 [1998515,5941] 0 2024-06-03T01:21:00.213 INFO:tasks.workunit.client.1.smithi044.stdout:3/665: dwrite df/d13/d39/d3d/d6a/d9c/fbd [0,4194304] 0 2024-06-03T01:21:00.214 INFO:tasks.workunit.client.0.smithi032.stdout:3/289: dwrite de/d29/f35 [0,4194304] 0 2024-06-03T01:21:00.214 INFO:tasks.workunit.client.0.smithi032.stdout:3/290: creat de/d29/d2e/d31/d23/f5a x:0 0 0 2024-06-03T01:21:00.215 INFO:tasks.workunit.client.1.smithi044.stdout:6/914: dwrite d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/f137 [0,4194304] 0 2024-06-03T01:21:00.215 INFO:tasks.workunit.client.1.smithi044.stdout:4/825: mkdir d17/d1c/d75/d7a/db8/d35/d117 0 2024-06-03T01:21:00.217 INFO:tasks.workunit.client.1.smithi044.stdout:5/711: unlink d2/f13 0 2024-06-03T01:21:00.217 INFO:tasks.workunit.client.1.smithi044.stdout:5/712: readlink d2/d11/d12/d2b/lc6 0 2024-06-03T01:21:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:20:59 smithi044 ceph-mon[36956]: mgrmap e23: smithi044.wszjlw(active, since 3s) 2024-06-03T01:21:00.224 INFO:tasks.workunit.client.1.smithi044.stdout:1/924: dwrite d0/dd2/dfe/dbe/d9e/d10d/dcb/f123 [0,4194304] 0 2024-06-03T01:21:00.224 INFO:tasks.workunit.client.1.smithi044.stdout:1/925: chown d0/dd2/dfe/dbe/d9e/d10d/d45/c7c 3904 1 2024-06-03T01:21:00.227 INFO:tasks.workunit.client.0.smithi032.stdout:3/291: dread f9 [0,4194304] 0 2024-06-03T01:21:00.227 INFO:tasks.workunit.client.0.smithi032.stdout:3/292: readlink de/d29/d2e/d31/l22 0 2024-06-03T01:21:00.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/174: symlink d18/d27/d2b/l33 0 2024-06-03T01:21:00.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/175: write d18/d27/f31 [517307,81935] 0 2024-06-03T01:21:00.232 INFO:tasks.workunit.client.1.smithi044.stdout:0/920: symlink d3/d9/d102/l14f 0 2024-06-03T01:21:00.236 INFO:tasks.workunit.client.1.smithi044.stdout:2/637: mknod db/d15/d49/d82/ccb 0 2024-06-03T01:21:00.238 INFO:tasks.workunit.client.1.smithi044.stdout:3/666: mkdir df/d1e/d78/dda 0 2024-06-03T01:21:00.240 INFO:tasks.workunit.client.0.smithi032.stdout:7/298: unlink dc/d12/d2a/d2f/f44 0 2024-06-03T01:21:00.241 INFO:tasks.workunit.client.1.smithi044.stdout:4/826: creat d17/d1c/f118 x:0 0 0 2024-06-03T01:21:00.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/915: dwrite d2/f27 [4194304,4194304] 0 2024-06-03T01:21:00.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/916: fsync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/dc7/f5d 0 2024-06-03T01:21:00.242 INFO:tasks.workunit.client.0.smithi032.stdout:8/253: rmdir dc/d14 39 2024-06-03T01:21:00.243 INFO:tasks.workunit.client.1.smithi044.stdout:5/713: mknod d2/d11/d1e/d21/d24/cea 0 2024-06-03T01:21:00.243 INFO:tasks.workunit.client.1.smithi044.stdout:7/763: dwrite d0/d16/f4d [0,4194304] 0 2024-06-03T01:21:00.248 INFO:tasks.workunit.client.0.smithi032.stdout:9/383: creat d0/d11/d1d/d4a/d71/f82 x:0 0 0 2024-06-03T01:21:00.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/926: truncate d0/d3/d20/d7/d13/d2d/dcd/d108/f26 2406569 0 2024-06-03T01:21:00.252 INFO:tasks.workunit.client.1.smithi044.stdout:0/921: rmdir d3/da6/d56/d8e 39 2024-06-03T01:21:00.253 INFO:tasks.workunit.client.0.smithi032.stdout:3/293: dwrite de/d29/d2e/f4d [0,4194304] 0 2024-06-03T01:21:00.254 INFO:tasks.workunit.client.1.smithi044.stdout:3/667: write df/d13/d39/d3d/d6a/d9c/fbd [839483,5241] 0 2024-06-03T01:21:00.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/638: unlink db/d1b/d3f/cc3 0 2024-06-03T01:21:00.260 INFO:tasks.workunit.client.0.smithi032.stdout:4/280: link d4/db/da/d37/c5b d4/db/da/d2d/d27/d53/c61 0 2024-06-03T01:21:00.268 INFO:tasks.workunit.client.1.smithi044.stdout:5/714: mkdir d2/d11/d1e/d21/d24/d9e/db1/deb 0 2024-06-03T01:21:00.270 INFO:tasks.workunit.client.0.smithi032.stdout:2/322: symlink d5/da/d1a/d69/l6a 0 2024-06-03T01:21:00.271 INFO:tasks.workunit.client.1.smithi044.stdout:7/764: unlink d0/d3/d8/d95/d63/d89/da3/f9e 0 2024-06-03T01:21:00.273 INFO:tasks.workunit.client.1.smithi044.stdout:7/765: creat d0/d3/d8/d95/d63/d89/ff5 x:0 0 0 2024-06-03T01:21:00.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/827: dwrite d17/d63/fd0 [4194304,4194304] 0 2024-06-03T01:21:00.274 INFO:tasks.workunit.client.1.smithi044.stdout:1/927: creat d0/d3/d20/d7/d13/d2d/dd9/f13a x:0 0 0 2024-06-03T01:21:00.275 INFO:tasks.workunit.client.1.smithi044.stdout:6/917: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fe3 [0,4194304] 0 2024-06-03T01:21:00.275 INFO:tasks.workunit.client.1.smithi044.stdout:6/918: write d2/d23/d30/f1f [790480,119757] 0 2024-06-03T01:21:00.278 INFO:tasks.workunit.client.1.smithi044.stdout:3/668: dwrite df/d1e/d3a/f4a [0,4194304] 0 2024-06-03T01:21:00.278 INFO:tasks.workunit.client.1.smithi044.stdout:0/922: unlink d3/da6/d56/d62/de7/df3/f11d 0 2024-06-03T01:21:00.279 INFO:tasks.workunit.client.1.smithi044.stdout:2/639: mknod db/d1b/ccc 0 2024-06-03T01:21:00.283 INFO:tasks.workunit.client.1.smithi044.stdout:5/715: read d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [864295,104809] 0 2024-06-03T01:21:00.283 INFO:tasks.workunit.client.1.smithi044.stdout:5/716: readlink d2/d11/d12/d2b/l6f 0 2024-06-03T01:21:00.287 INFO:tasks.workunit.client.1.smithi044.stdout:6/919: dread d2/d23/d55/d5f/d53/f61 [0,4194304] 0 2024-06-03T01:21:00.287 INFO:tasks.workunit.client.1.smithi044.stdout:6/920: write d2/f134 [93327,20880] 0 2024-06-03T01:21:00.289 INFO:tasks.workunit.client.0.smithi032.stdout:2/323: dwrite d5/d38/d46/d53/f5e [0,4194304] 0 2024-06-03T01:21:00.290 INFO:tasks.workunit.client.1.smithi044.stdout:7/766: unlink d0/d3/d8/d95/d63/d7f/d9c/c9d 0 2024-06-03T01:21:00.294 INFO:tasks.workunit.client.1.smithi044.stdout:4/828: mknod d17/c119 0 2024-06-03T01:21:00.295 INFO:tasks.workunit.client.1.smithi044.stdout:4/829: write d17/d1c/d75/d7a/db8/d28/fd1 [1841278,107537] 0 2024-06-03T01:21:00.295 INFO:tasks.workunit.client.1.smithi044.stdout:4/830: write d17/d1c/d75/d7a/db8/d59/f82 [3761442,29917] 0 2024-06-03T01:21:00.295 INFO:tasks.workunit.client.1.smithi044.stdout:4/831: dread - d17/d1c/d75/d7a/db3/f10a zero size 2024-06-03T01:21:00.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/928: rename d0/d3/d20/d7/d13/d7a to d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/d13b 0 2024-06-03T01:21:00.299 INFO:tasks.workunit.client.1.smithi044.stdout:1/929: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/f13c x:0 0 0 2024-06-03T01:21:00.305 INFO:tasks.workunit.client.1.smithi044.stdout:3/669: unlink df/d13/d39/d3d/d43/d72/c7d 0 2024-06-03T01:21:00.307 INFO:tasks.workunit.client.1.smithi044.stdout:0/923: truncate d3/da6/f3e 749143 0 2024-06-03T01:21:00.307 INFO:tasks.workunit.client.1.smithi044.stdout:0/924: fsync d3/da6/dc1/dc2/fef 0 2024-06-03T01:21:00.307 INFO:tasks.workunit.client.1.smithi044.stdout:0/925: dread - d3/d6b/d95/f135 zero size 2024-06-03T01:21:00.309 INFO:tasks.workunit.client.1.smithi044.stdout:2/640: mkdir db/d15/d49/d6c/d72/dcd 0 2024-06-03T01:21:00.311 INFO:tasks.workunit.client.1.smithi044.stdout:7/767: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/df6 0 2024-06-03T01:21:00.312 INFO:tasks.workunit.client.1.smithi044.stdout:4/832: getdents d17/d1c/d75/d7a/db8/d6a/d84/df4 0 2024-06-03T01:21:00.316 INFO:tasks.workunit.client.0.smithi032.stdout:2/324: dwrite d5/da/f39 [0,4194304] 0 2024-06-03T01:21:00.316 INFO:tasks.workunit.client.0.smithi032.stdout:2/325: creat d5/f6b x:0 0 0 2024-06-03T01:21:00.316 INFO:tasks.workunit.client.1.smithi044.stdout:1/930: rename d0/d3/d20/d7/l73 to d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/dae/l13d 0 2024-06-03T01:21:00.316 INFO:tasks.workunit.client.1.smithi044.stdout:1/931: chown d0/d3/d20/f6d 15362 1 2024-06-03T01:21:00.316 INFO:tasks.workunit.client.1.smithi044.stdout:6/921: dwrite d2/f134 [0,4194304] 0 2024-06-03T01:21:00.317 INFO:tasks.workunit.client.1.smithi044.stdout:0/926: mkdir d3/d9/d55/d5a/dde/dea/d150 0 2024-06-03T01:21:00.318 INFO:tasks.workunit.client.1.smithi044.stdout:5/717: getdents d2/d11/d1e/d21/d24/d93/d9d/d28 0 2024-06-03T01:21:00.318 INFO:tasks.workunit.client.1.smithi044.stdout:5/718: chown d2/d11/d1e/d3b/cdc 166183750 1 2024-06-03T01:21:00.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/833: dread d17/d1c/d75/f5d [0,4194304] 0 2024-06-03T01:21:00.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/834: creat d17/d1c/d75/d7a/db8/d8d/ded/f11a x:0 0 0 2024-06-03T01:21:00.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/254: rename dc/cf to dc/d25/d28/d51/c53 0 2024-06-03T01:21:00.321 INFO:tasks.workunit.client.1.smithi044.stdout:2/641: dread db/d15/d49/d6c/d72/d7a/f81 [0,4194304] 0 2024-06-03T01:21:00.321 INFO:tasks.workunit.client.1.smithi044.stdout:2/642: dread - db/fa2 zero size 2024-06-03T01:21:00.321 INFO:tasks.workunit.client.1.smithi044.stdout:7/768: truncate d0/d3/d8/d32/d33/fd6 35177 0 2024-06-03T01:21:00.323 INFO:tasks.workunit.client.0.smithi032.stdout:9/384: link d0/d11/d1d/d38/l64 d0/d11/d79/l83 0 2024-06-03T01:21:00.323 INFO:tasks.workunit.client.0.smithi032.stdout:9/385: fsync d0/fe 0 2024-06-03T01:21:00.328 INFO:tasks.workunit.client.1.smithi044.stdout:6/922: creat d2/d23/d44/dda/f13e x:0 0 0 2024-06-03T01:21:00.336 INFO:tasks.workunit.client.0.smithi032.stdout:8/255: dwrite dc/d25/f47 [0,4194304] 0 2024-06-03T01:21:00.337 INFO:tasks.workunit.client.0.smithi032.stdout:2/326: rmdir d5/d5d 0 2024-06-03T01:21:00.339 INFO:tasks.workunit.client.1.smithi044.stdout:5/719: creat d2/d11/d3e/dce/fec x:0 0 0 2024-06-03T01:21:00.340 INFO:tasks.workunit.client.0.smithi032.stdout:9/386: link d0/d11/d1d/d4a/d71/f82 d0/d23/d24/f84 0 2024-06-03T01:21:00.343 INFO:tasks.workunit.client.0.smithi032.stdout:3/294: getdents de 0 2024-06-03T01:21:00.345 INFO:tasks.workunit.client.1.smithi044.stdout:2/643: mkdir db/d15/d49/d6c/d72/dce 0 2024-06-03T01:21:00.347 INFO:tasks.workunit.client.1.smithi044.stdout:7/769: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/lf7 0 2024-06-03T01:21:00.349 INFO:tasks.workunit.client.1.smithi044.stdout:1/932: rename d0/d3/d20/d7/d13/d2d/dcd to d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e 0 2024-06-03T01:21:00.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/256: symlink dc/d14/d31/l54 0 2024-06-03T01:21:00.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/257: creat dc/d25/f55 x:0 0 0 2024-06-03T01:21:00.352 INFO:tasks.workunit.client.0.smithi032.stdout:4/281: rename d4/db/l4b to d4/db/l62 0 2024-06-03T01:21:00.352 INFO:tasks.workunit.client.0.smithi032.stdout:4/282: fdatasync d4/db/da/f1b 0 2024-06-03T01:21:00.353 INFO:tasks.workunit.client.0.smithi032.stdout:2/327: mkdir d5/d38/d46/d5c/d6c 0 2024-06-03T01:21:00.354 INFO:tasks.workunit.client.0.smithi032.stdout:2/328: write d5/d38/f3b [4918453,30889] 0 2024-06-03T01:21:00.355 INFO:tasks.workunit.client.1.smithi044.stdout:0/927: rmdir d3/d9 39 2024-06-03T01:21:00.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/720: creat d2/d11/d12/d2b/d4a/d57/fed x:0 0 0 2024-06-03T01:21:00.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/721: chown d2/d11/d12/f1b 14104879 1 2024-06-03T01:21:00.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/722: chown d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/ce1 5895060 1 2024-06-03T01:21:00.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/723: write d2/d11/d12/d2b/d44/fb8 [856962,86648] 0 2024-06-03T01:21:00.360 INFO:tasks.workunit.client.0.smithi032.stdout:3/295: creat de/f5b x:0 0 0 2024-06-03T01:21:00.361 INFO:tasks.workunit.client.0.smithi032.stdout:9/387: creat d0/d23/f85 x:0 0 0 2024-06-03T01:21:00.361 INFO:tasks.workunit.client.1.smithi044.stdout:7/770: creat d0/d3/d8/d95/ff8 x:0 0 0 2024-06-03T01:21:00.361 INFO:tasks.workunit.client.1.smithi044.stdout:7/771: fdatasync d0/d16/f54 0 2024-06-03T01:21:00.361 INFO:tasks.workunit.client.1.smithi044.stdout:7/772: creat d0/d3/d8/d95/d63/d89/ff9 x:0 0 0 2024-06-03T01:21:00.362 INFO:tasks.workunit.client.1.smithi044.stdout:6/923: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/l13f 0 2024-06-03T01:21:00.362 INFO:tasks.workunit.client.1.smithi044.stdout:6/924: chown d2/f14 4 1 2024-06-03T01:21:00.362 INFO:tasks.workunit.client.1.smithi044.stdout:6/925: creat d2/d23/d55/f140 x:0 0 0 2024-06-03T01:21:00.364 INFO:tasks.workunit.client.0.smithi032.stdout:8/258: symlink dc/l56 0 2024-06-03T01:21:00.366 INFO:tasks.workunit.client.1.smithi044.stdout:7/773: mknod d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/cfa 0 2024-06-03T01:21:00.368 INFO:tasks.workunit.client.1.smithi044.stdout:0/928: dread d3/d9/d36/d82/f9c [0,4194304] 0 2024-06-03T01:21:00.368 INFO:tasks.workunit.client.0.smithi032.stdout:4/283: unlink d4/db/da/d5d/f5f 0 2024-06-03T01:21:00.369 INFO:tasks.workunit.client.1.smithi044.stdout:7/774: link d0/d3/d8/d95/d63/d89/da3/c5a d0/d3/d8/d95/d63/d89/da3/dde/cfb 0 2024-06-03T01:21:00.370 INFO:tasks.workunit.client.0.smithi032.stdout:4/284: dread d4/db/da/f49 [0,4194304] 0 2024-06-03T01:21:00.370 INFO:tasks.workunit.client.1.smithi044.stdout:7/775: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/lfc 0 2024-06-03T01:21:00.370 INFO:tasks.workunit.client.1.smithi044.stdout:7/776: chown d0/d3/d8/d95/d63/d89/d8b 57 1 2024-06-03T01:21:00.370 INFO:tasks.workunit.client.1.smithi044.stdout:2/644: dwrite db/d1b/d2a/d2f/d4c/f77 [0,4194304] 0 2024-06-03T01:21:00.371 INFO:tasks.workunit.client.0.smithi032.stdout:2/329: mkdir d5/d38/d46/d53/d57/d6d 0 2024-06-03T01:21:00.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/388: write d0/d11/d1d/d38/f3e [63203,97789] 0 2024-06-03T01:21:00.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/389: truncate d0/d23/d24/f2a 582665 0 2024-06-03T01:21:00.375 INFO:tasks.workunit.client.1.smithi044.stdout:7/777: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/ffd x:0 0 0 2024-06-03T01:21:00.378 INFO:tasks.workunit.client.1.smithi044.stdout:2/645: truncate db/f73 3072764 0 2024-06-03T01:21:00.378 INFO:tasks.workunit.client.1.smithi044.stdout:2/646: write db/f48 [4806091,1520] 0 2024-06-03T01:21:00.380 INFO:tasks.workunit.client.0.smithi032.stdout:2/330: dread d5/f25 [0,4194304] 0 2024-06-03T01:21:00.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/778: rename d0/d3/d8/d95/d63/d7f/d9c/c29 to d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/cfe 0 2024-06-03T01:21:00.383 INFO:tasks.workunit.client.0.smithi032.stdout:4/285: mknod d4/db/da/d2d/c63 0 2024-06-03T01:21:00.386 INFO:tasks.workunit.client.0.smithi032.stdout:3/296: creat de/d29/d2e/d31/d3c/d50/d56/f5c x:0 0 0 2024-06-03T01:21:00.389 INFO:tasks.workunit.client.0.smithi032.stdout:9/390: dread d0/f5 [0,4194304] 0 2024-06-03T01:21:00.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/933: dwrite d0/d3/d20/d7/d13/f24 [8388608,4194304] 0 2024-06-03T01:21:00.390 INFO:tasks.workunit.client.1.smithi044.stdout:5/724: dwrite d2/d11/d12/d2b/d4a/f5d [0,4194304] 0 2024-06-03T01:21:00.392 INFO:tasks.workunit.client.0.smithi032.stdout:8/259: unlink dc/d14/l27 0 2024-06-03T01:21:00.393 INFO:tasks.workunit.client.0.smithi032.stdout:8/260: readlink dc/d14/d31/l45 0 2024-06-03T01:21:00.393 INFO:tasks.workunit.client.1.smithi044.stdout:1/934: symlink d0/d3/d20/d7/l13f 0 2024-06-03T01:21:00.395 INFO:tasks.workunit.client.1.smithi044.stdout:0/929: dwrite d3/d9/d55/d5a/fe9 [4194304,4194304] 0 2024-06-03T01:21:00.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/297: read de/f4e [2578392,96222] 0 2024-06-03T01:21:00.396 INFO:tasks.workunit.client.0.smithi032.stdout:9/391: creat d0/d11/f86 x:0 0 0 2024-06-03T01:21:00.399 INFO:tasks.workunit.client.0.smithi032.stdout:3/298: truncate de/d29/d2e/f45 2746376 0 2024-06-03T01:21:00.399 INFO:tasks.workunit.client.0.smithi032.stdout:3/299: chown de/d29/d2e/d31/f21 22597611 1 2024-06-03T01:21:00.399 INFO:tasks.workunit.client.0.smithi032.stdout:3/300: fsync de/d29/d2e/d31/f38 0 2024-06-03T01:21:00.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/725: dread d2/d11/d1e/d21/d24/d93/d9d/d28/d66/f8e [0,4194304] 0 2024-06-03T01:21:00.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/726: write d2/d11/d1e/d21/d24/d93/d9d/f8c [944378,50404] 0 2024-06-03T01:21:00.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/727: creat d2/d11/d12/d2b/d4a/d57/fee x:0 0 0 2024-06-03T01:21:00.407 INFO:tasks.workunit.client.1.smithi044.stdout:1/935: unlink d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d30/c39 0 2024-06-03T01:21:00.408 INFO:tasks.workunit.client.0.smithi032.stdout:9/392: symlink d0/d11/d1d/d38/d4f/l87 0 2024-06-03T01:21:00.408 INFO:tasks.workunit.client.0.smithi032.stdout:9/393: write d0/f15 [144374,93988] 0 2024-06-03T01:21:00.411 INFO:tasks.workunit.client.0.smithi032.stdout:3/301: mknod de/d29/d2e/d31/d3c/d50/d56/c5d 0 2024-06-03T01:21:00.411 INFO:tasks.workunit.client.0.smithi032.stdout:3/302: fsync de/d29/d2e/d31/d23/f59 0 2024-06-03T01:21:00.411 INFO:tasks.workunit.client.0.smithi032.stdout:3/303: write de/d29/f52 [1389114,72191] 0 2024-06-03T01:21:00.415 INFO:tasks.workunit.client.1.smithi044.stdout:0/930: mknod d3/d9/d55/d7b/dfc/d10c/d13d/c151 0 2024-06-03T01:21:00.419 INFO:tasks.workunit.client.1.smithi044.stdout:7/779: dwrite d0/d3/d8/d95/d63/d7f/f99 [0,4194304] 0 2024-06-03T01:21:00.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/728: symlink d2/d11/d1e/d21/d24/d93/d9d/d28/d66/lef 0 2024-06-03T01:21:00.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/331: rename d5/l14 to d5/da/l6e 0 2024-06-03T01:21:00.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/332: write d5/da/d1a/d2a/d36/f66 [272271,81686] 0 2024-06-03T01:21:00.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/333: creat d5/da/d1a/f6f x:0 0 0 2024-06-03T01:21:00.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/729: dread d2/d11/d1e/f95 [0,4194304] 0 2024-06-03T01:21:00.423 INFO:tasks.workunit.client.0.smithi032.stdout:9/394: dread d0/d23/d25/f52 [0,4194304] 0 2024-06-03T01:21:00.424 INFO:tasks.workunit.client.0.smithi032.stdout:9/395: write d0/d23/d25/f4c [2728861,81200] 0 2024-06-03T01:21:00.424 INFO:tasks.workunit.client.0.smithi032.stdout:9/396: creat d0/d23/f88 x:0 0 0 2024-06-03T01:21:00.424 INFO:tasks.workunit.client.0.smithi032.stdout:8/261: dwrite dc/d25/f35 [0,4194304] 0 2024-06-03T01:21:00.425 INFO:tasks.workunit.client.0.smithi032.stdout:4/286: rename d4/db/da/d1d/c3c to d4/db/da/d1d/d58/c64 0 2024-06-03T01:21:00.425 INFO:tasks.workunit.client.1.smithi044.stdout:7/780: unlink d0/d3/d8/d32/ca7 0 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:9/712: sync 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:8/816: sync 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:7/781: truncate d0/f74 3711000 0 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:7/782: chown d0/d3/d8/d95/l44 110936 1 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:8/817: creat d5/d16/d1d/d59/f10e x:0 0 0 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:1/936: dwrite d0/dd2/dfe/dbe/d9e/d10d/f88 [0,4194304] 0 2024-06-03T01:21:00.428 INFO:tasks.workunit.client.1.smithi044.stdout:1/937: chown d0/d3/c43 27 1 2024-06-03T01:21:00.430 INFO:tasks.workunit.client.0.smithi032.stdout:2/334: mkdir d5/da/d1a/d70 0 2024-06-03T01:21:00.432 INFO:tasks.workunit.client.1.smithi044.stdout:9/713: mknod d6/dd/d51/d16/d5f/d8b/ce7 0 2024-06-03T01:21:00.432 INFO:tasks.workunit.client.1.smithi044.stdout:9/714: chown d6/dd/d51/d16/d5f/f80 4375081 1 2024-06-03T01:21:00.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/397: mknod d0/d23/d24/d26/c89 0 2024-06-03T01:21:00.434 INFO:tasks.workunit.client.1.smithi044.stdout:7/783: unlink d0/d3/d30/cb5 0 2024-06-03T01:21:00.436 INFO:tasks.workunit.client.1.smithi044.stdout:8/818: creat d5/d20/df5/f10f x:0 0 0 2024-06-03T01:21:00.436 INFO:tasks.workunit.client.1.smithi044.stdout:8/819: creat d5/d55/d56/d5c/d68/de5/f110 x:0 0 0 2024-06-03T01:21:00.436 INFO:tasks.workunit.client.1.smithi044.stdout:2/647: read db/f73 [1150990,130310] 0 2024-06-03T01:21:00.438 INFO:tasks.workunit.client.0.smithi032.stdout:3/304: dwrite de/f1a [0,4194304] 0 2024-06-03T01:21:00.438 INFO:tasks.workunit.client.0.smithi032.stdout:4/287: creat d4/db/da/d2d/d27/d53/f65 x:0 0 0 2024-06-03T01:21:00.439 INFO:tasks.workunit.client.0.smithi032.stdout:4/288: creat d4/db/da/d2d/d27/d53/f66 x:0 0 0 2024-06-03T01:21:00.439 INFO:tasks.workunit.client.0.smithi032.stdout:4/289: fsync d4/db/da/d1d/f33 0 2024-06-03T01:21:00.441 INFO:tasks.workunit.client.0.smithi032.stdout:8/262: rename dc/d25/d28/d32/c3e to dc/d25/d28/d33/c57 0 2024-06-03T01:21:00.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/715: unlink d6/l99 0 2024-06-03T01:21:00.442 INFO:tasks.workunit.client.0.smithi032.stdout:2/335: write d5/f23 [252464,89130] 0 2024-06-03T01:21:00.444 INFO:tasks.workunit.client.0.smithi032.stdout:9/398: getdents d0/d11/d1d 0 2024-06-03T01:21:00.446 INFO:tasks.workunit.client.1.smithi044.stdout:2/648: creat db/d15/d49/d6c/d72/dcd/fcf x:0 0 0 2024-06-03T01:21:00.446 INFO:tasks.workunit.client.0.smithi032.stdout:3/305: mkdir de/d29/d2e/d31/d3c/d50/d5e 0 2024-06-03T01:21:00.446 INFO:tasks.workunit.client.0.smithi032.stdout:3/306: chown de/d29/f52 15724 1 2024-06-03T01:21:00.447 INFO:tasks.workunit.client.0.smithi032.stdout:3/307: chown de/d29 2627498 1 2024-06-03T01:21:00.447 INFO:tasks.workunit.client.1.smithi044.stdout:9/716: creat d6/dd/d51/d16/d5f/ddd/d67/fe8 x:0 0 0 2024-06-03T01:21:00.448 INFO:tasks.workunit.client.1.smithi044.stdout:2/649: mkdir db/d1b/dd0 0 2024-06-03T01:21:00.448 INFO:tasks.workunit.client.1.smithi044.stdout:2/650: truncate db/d15/d49/d6c/d72/d7a/f68 4863498 0 2024-06-03T01:21:00.448 INFO:tasks.workunit.client.1.smithi044.stdout:2/651: creat db/d15/d49/d6c/d72/fd1 x:0 0 0 2024-06-03T01:21:00.449 INFO:tasks.workunit.client.0.smithi032.stdout:8/263: mkdir dc/d25/d28/d58 0 2024-06-03T01:21:00.449 INFO:tasks.workunit.client.1.smithi044.stdout:8/820: dread d5/d16/f6e [0,4194304] 0 2024-06-03T01:21:00.450 INFO:tasks.workunit.client.1.smithi044.stdout:2/652: dread db/d1b/d2a/d52/d54/f56 [0,4194304] 0 2024-06-03T01:21:00.451 INFO:tasks.workunit.client.0.smithi032.stdout:2/336: creat d5/da/d1a/d2a/d49/f71 x:0 0 0 2024-06-03T01:21:00.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/821: unlink d5/d55/fad 0 2024-06-03T01:21:00.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/822: fdatasync d5/d16/f7c 0 2024-06-03T01:21:00.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/823: chown d5/d20/f69 61 1 2024-06-03T01:21:00.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/824: readlink d5/d55/d56/d5c/d68/de5/dea/d79/d88/l90 0 2024-06-03T01:21:00.452 INFO:tasks.workunit.client.1.smithi044.stdout:1/938: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/f133 [0,4194304] 0 2024-06-03T01:21:00.454 INFO:tasks.workunit.client.1.smithi044.stdout:8/825: mknod d5/d55/d74/d82/db0/c111 0 2024-06-03T01:21:00.454 INFO:tasks.workunit.client.0.smithi032.stdout:3/308: mknod de/d29/c5f 0 2024-06-03T01:21:00.455 INFO:tasks.workunit.client.0.smithi032.stdout:8/264: stat dc/d25/l2f 0 2024-06-03T01:21:00.457 INFO:tasks.workunit.client.1.smithi044.stdout:9/717: write d6/dd/d51/f62 [2163258,7855] 0 2024-06-03T01:21:00.457 INFO:tasks.workunit.client.1.smithi044.stdout:0/931: dwrite d3/d9/f105 [0,4194304] 0 2024-06-03T01:21:00.457 INFO:tasks.workunit.client.1.smithi044.stdout:0/932: read - d3/d5c/fbd zero size 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:6/237: sync 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:6/238: chown d8/fb 1 1 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:6/239: fsync d8/d11/d20/d2b/d39/d19/f1d 0 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/284: sync 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:5/335: sync 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:5/336: creat da/dd/df/d10/d16/d25/f66 x:0 0 0 2024-06-03T01:21:00.459 INFO:tasks.workunit.client.0.smithi032.stdout:5/337: readlink da/dd/df/d10/d16/l55 0 2024-06-03T01:21:00.463 INFO:tasks.workunit.client.0.smithi032.stdout:3/309: dread de/d29/f2b [0,4194304] 0 2024-06-03T01:21:00.463 INFO:tasks.workunit.client.0.smithi032.stdout:3/310: creat de/d29/d2e/d31/d3c/d50/d56/f60 x:0 0 0 2024-06-03T01:21:00.465 INFO:tasks.workunit.client.1.smithi044.stdout:8/826: dread d5/d16/d1d/f38 [0,4194304] 0 2024-06-03T01:21:00.466 INFO:tasks.workunit.client.0.smithi032.stdout:8/265: mkdir dc/de/d1e/d43/d4e/d59 0 2024-06-03T01:21:00.467 INFO:tasks.workunit.client.0.smithi032.stdout:2/337: rmdir d5/d38/d46/d53 39 2024-06-03T01:21:00.469 INFO:tasks.workunit.client.1.smithi044.stdout:2/653: dwrite db/dd/f11 [8388608,4194304] 0 2024-06-03T01:21:00.470 INFO:tasks.workunit.client.1.smithi044.stdout:2/654: read db/d1b/d2a/d67/d53/f93 [540507,85921] 0 2024-06-03T01:21:00.470 INFO:tasks.workunit.client.0.smithi032.stdout:6/240: mkdir d8/d4c 0 2024-06-03T01:21:00.470 INFO:tasks.workunit.client.0.smithi032.stdout:6/241: write d8/d11/f35 [4258189,46201] 0 2024-06-03T01:21:00.472 INFO:tasks.workunit.client.0.smithi032.stdout:0/285: mknod d1/da/c5a 0 2024-06-03T01:21:00.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/266: dread dc/d14/f24 [0,4194304] 0 2024-06-03T01:21:00.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/267: creat dc/d25/d28/d3c/f5a x:0 0 0 2024-06-03T01:21:00.477 INFO:tasks.workunit.client.1.smithi044.stdout:8/827: truncate d5/d55/d56/f8a 1408370 0 2024-06-03T01:21:00.478 INFO:tasks.workunit.client.1.smithi044.stdout:9/718: rename d6/dd/d51/d16/d5f/ddd to d6/dd/d51/d16/d5f/d8b/d8d/dac/de9 0 2024-06-03T01:21:00.478 INFO:tasks.workunit.client.1.smithi044.stdout:9/719: chown d6/dd/d51/db/dd5/fda 2 1 2024-06-03T01:21:00.478 INFO:tasks.workunit.client.1.smithi044.stdout:9/720: stat d6/dd/d51/d16/d5f/d8b 0 2024-06-03T01:21:00.480 INFO:tasks.workunit.client.1.smithi044.stdout:2/655: symlink db/d1b/d2a/d67/ld2 0 2024-06-03T01:21:00.481 INFO:tasks.workunit.client.1.smithi044.stdout:9/721: mknod d6/dd/d51/d16/d27/d5e/db7/dd6/cea 0 2024-06-03T01:21:00.482 INFO:tasks.workunit.client.1.smithi044.stdout:9/722: fsync d6/dd/d51/d16/f5a 0 2024-06-03T01:21:00.482 INFO:tasks.workunit.client.1.smithi044.stdout:2/656: unlink db/d1b/d2a/d2f/f9f 0 2024-06-03T01:21:00.483 INFO:tasks.workunit.client.0.smithi032.stdout:9/399: getdents d0/d23 0 2024-06-03T01:21:00.483 INFO:tasks.workunit.client.0.smithi032.stdout:9/400: read - d0/d23/f88 zero size 2024-06-03T01:21:00.483 INFO:tasks.workunit.client.0.smithi032.stdout:9/401: write d0/d23/d25/f62 [377072,65974] 0 2024-06-03T01:21:00.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/338: write d5/da/f39 [364576,82877] 0 2024-06-03T01:21:00.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/339: chown d5/f35 468994 1 2024-06-03T01:21:00.485 INFO:tasks.workunit.client.1.smithi044.stdout:0/933: dwrite d3/f4 [0,4194304] 0 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/934: rmdir d3/d9/d36/d82 39 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/935: chown d3/d9/d36/fd7 612 1 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/936: chown d3/da6/d21/f76 32459 1 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/937: write d3/da6/d20/d25/f112 [949089,94318] 0 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:1/939: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/fca [0,4194304] 0 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:1/940: chown d0/dd2/dfe/dbe/d9e/d10d/dcb/f123 10010 1 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/670: sync 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/671: dread - df/d1e/d3a/f88 zero size 2024-06-03T01:21:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/672: write df/d13/d1b/d4b/d69/f75 [899423,1546] 0 2024-06-03T01:21:00.490 INFO:tasks.workunit.client.0.smithi032.stdout:3/311: rename de/d29/f3e to de/d29/d2e/d31/d3c/f61 0 2024-06-03T01:21:00.490 INFO:tasks.workunit.client.1.smithi044.stdout:8/828: write d5/d55/d56/f75 [956416,125382] 0 2024-06-03T01:21:00.490 INFO:tasks.workunit.client.1.smithi044.stdout:8/829: truncate d5/d16/d81/de3/d58/fa3 4221051 0 2024-06-03T01:21:00.490 INFO:tasks.workunit.client.1.smithi044.stdout:0/938: mkdir d3/d9/d55/d5a/d29/d58/d152 0 2024-06-03T01:21:00.495 INFO:tasks.workunit.client.0.smithi032.stdout:0/286: creat d1/da/d10/f5b x:0 0 0 2024-06-03T01:21:00.495 INFO:tasks.workunit.client.1.smithi044.stdout:3/673: fdatasync df/d13/f24 0 2024-06-03T01:21:00.495 INFO:tasks.workunit.client.1.smithi044.stdout:8/830: symlink d5/d55/d56/d5c/dee/l112 0 2024-06-03T01:21:00.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/242: dwrite d8/d11/d20/d2b/d39/d19/f1d [0,4194304] 0 2024-06-03T01:21:00.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/243: readlink d8/la 0 2024-06-03T01:21:00.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/244: readlink d8/l10 0 2024-06-03T01:21:00.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/245: chown d8/d11/d20/d21/f24 24653901 1 2024-06-03T01:21:00.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/246: creat d8/d11/d20/d2b/d39/d19/f4d x:0 0 0 2024-06-03T01:21:00.498 INFO:tasks.workunit.client.1.smithi044.stdout:1/941: mkdir d0/dd2/d140 0 2024-06-03T01:21:00.500 INFO:tasks.workunit.client.1.smithi044.stdout:3/674: mknod df/d13/d39/d3d/cdb 0 2024-06-03T01:21:00.502 INFO:tasks.workunit.client.1.smithi044.stdout:8/831: creat d5/dd4/df9/f113 x:0 0 0 2024-06-03T01:21:00.502 INFO:tasks.workunit.client.0.smithi032.stdout:9/402: symlink d0/d23/d24/d26/d41/l8a 0 2024-06-03T01:21:00.504 INFO:tasks.workunit.client.1.smithi044.stdout:8/832: creat d5/d55/d56/d5c/d68/de5/dea/de2/f114 x:0 0 0 2024-06-03T01:21:00.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/312: chown de/l19 118807 1 2024-06-03T01:21:00.509 INFO:tasks.workunit.client.1.smithi044.stdout:1/942: write d0/d3/d20/d7/d13/d2d/d101/f132 [4050872,122782] 0 2024-06-03T01:21:00.510 INFO:tasks.workunit.client.1.smithi044.stdout:1/943: creat d0/dd2/dfe/dbe/d9e/d10d/dcb/f141 x:0 0 0 2024-06-03T01:21:00.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/338: getdents da/dd/df/d10/d16/d25 0 2024-06-03T01:21:00.510 INFO:tasks.workunit.client.1.smithi044.stdout:9/723: dwrite d6/dd/d51/d46/f76 [0,4194304] 0 2024-06-03T01:21:00.512 INFO:tasks.workunit.client.1.smithi044.stdout:1/944: creat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/d116/f142 x:0 0 0 2024-06-03T01:21:00.513 INFO:tasks.workunit.client.1.smithi044.stdout:1/945: write d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/f111 [635750,105967] 0 2024-06-03T01:21:00.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/268: rename l3 to dc/de/d1e/l5b 0 2024-06-03T01:21:00.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/269: fsync dc/f15 0 2024-06-03T01:21:00.515 INFO:tasks.workunit.client.1.smithi044.stdout:9/724: rename d6/dd/d51/d16/d5f/d8b/ca7 to d6/dd/d51/d16/d5f/d8b/ceb 0 2024-06-03T01:21:00.515 INFO:tasks.workunit.client.1.smithi044.stdout:9/725: write d6/dd/d51/d46/f48 [619467,100218] 0 2024-06-03T01:21:00.516 INFO:tasks.workunit.client.1.smithi044.stdout:9/726: stat d6/dd/d51/l35 0 2024-06-03T01:21:00.516 INFO:tasks.workunit.client.0.smithi032.stdout:0/287: rmdir d1/da/d10/d53 39 2024-06-03T01:21:00.517 INFO:tasks.workunit.client.1.smithi044.stdout:8/833: dread d5/d16/d1d/f42 [0,4194304] 0 2024-06-03T01:21:00.518 INFO:tasks.workunit.client.1.smithi044.stdout:0/939: dwrite d3/d9/d55/d5a/d29/d58/f136 [0,4194304] 0 2024-06-03T01:21:00.519 INFO:tasks.workunit.client.1.smithi044.stdout:2/657: dwrite db/dd/f9a [0,4194304] 0 2024-06-03T01:21:00.519 INFO:tasks.workunit.client.1.smithi044.stdout:9/727: dread d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf [0,4194304] 0 2024-06-03T01:21:00.519 INFO:tasks.workunit.client.1.smithi044.stdout:9/728: truncate d6/dd/d51/d16/d27/d5e/db7/fc0 22200 0 2024-06-03T01:21:00.520 INFO:tasks.workunit.client.1.smithi044.stdout:8/834: mkdir d5/d20/d33/dfd/d115 0 2024-06-03T01:21:00.520 INFO:tasks.workunit.client.1.smithi044.stdout:3/675: dwrite df/d13/d6d/d9a/faa [0,4194304] 0 2024-06-03T01:21:00.520 INFO:tasks.workunit.client.0.smithi032.stdout:6/247: unlink d8/d11/d20/d2b/d39/c42 0 2024-06-03T01:21:00.520 INFO:tasks.workunit.client.0.smithi032.stdout:6/248: readlink d8/d11/d20/d2b/d39/l29 0 2024-06-03T01:21:00.520 INFO:tasks.workunit.client.0.smithi032.stdout:9/403: mknod d0/d23/d24/d26/c8b 0 2024-06-03T01:21:00.521 INFO:tasks.workunit.client.0.smithi032.stdout:9/404: dread - d0/d11/d1d/d4a/f7f zero size 2024-06-03T01:21:00.522 INFO:tasks.workunit.client.0.smithi032.stdout:3/313: mkdir de/d29/d2e/d31/d3c/d50/d62 0 2024-06-03T01:21:00.522 INFO:tasks.workunit.client.0.smithi032.stdout:3/314: stat de/d29/d2e/d31/f1c 0 2024-06-03T01:21:00.524 INFO:tasks.workunit.client.1.smithi044.stdout:2/658: unlink db/d15/d49/d6c/d72/d7a/ca8 0 2024-06-03T01:21:00.524 INFO:tasks.workunit.client.1.smithi044.stdout:2/659: write db/d1b/d2a/f2b [49866,32013] 0 2024-06-03T01:21:00.524 INFO:tasks.workunit.client.1.smithi044.stdout:2/660: fsync db/d1b/d2a/f43 0 2024-06-03T01:21:00.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/339: rename l9 to da/dd/df/d10/d16/d18/d1a/d30/d58/l67 0 2024-06-03T01:21:00.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/340: dread - da/dd/df/d10/d16/d25/f56 zero size 2024-06-03T01:21:00.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/341: readlink da/l1e 0 2024-06-03T01:21:00.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/342: write da/dd/f2d [726750,26611] 0 2024-06-03T01:21:00.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/343: fsync da/dd/df/d10/d16/d18/d1a/d30/f5a 0 2024-06-03T01:21:00.526 INFO:tasks.workunit.client.1.smithi044.stdout:9/729: truncate d6/dd/d51/d16/d27/d5e/d71/fba 932036 0 2024-06-03T01:21:00.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/405: rmdir d0/d11/d1d 39 2024-06-03T01:21:00.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/406: fsync d0/d11/f63 0 2024-06-03T01:21:00.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/407: write d0/d23/f3d [4523354,57249] 0 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:6/926: sync 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/835: sync 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/836: chown d17/d1c/d75/d7a/db3/f10a 28947107 1 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/837: write d17/d1c/d75/d7a/db8/d28/d2b/f24 [4706227,63675] 0 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/838: fdatasync d17/d63/dae/ff2 0 2024-06-03T01:21:00.528 INFO:tasks.workunit.client.1.smithi044.stdout:4/839: fdatasync d17/d1c/d75/d7a/db8/d35/fce 0 2024-06-03T01:21:00.533 INFO:tasks.workunit.client.0.smithi032.stdout:3/315: truncate de/f57 649942 0 2024-06-03T01:21:00.533 INFO:tasks.workunit.client.0.smithi032.stdout:0/288: dread d1/da/f1a [0,4194304] 0 2024-06-03T01:21:00.533 INFO:tasks.workunit.client.0.smithi032.stdout:0/289: write d1/da/d28/f51 [319762,107820] 0 2024-06-03T01:21:00.534 INFO:tasks.workunit.client.0.smithi032.stdout:5/344: dread da/f12 [0,4194304] 0 2024-06-03T01:21:00.534 INFO:tasks.workunit.client.0.smithi032.stdout:9/408: symlink d0/d11/l8c 0 2024-06-03T01:21:00.534 INFO:tasks.workunit.client.0.smithi032.stdout:9/409: write d0/fd [804340,82663] 0 2024-06-03T01:21:00.537 INFO:tasks.workunit.client.0.smithi032.stdout:8/270: rename dc/l39 to dc/d14/d31/d3b/l5c 0 2024-06-03T01:21:00.537 INFO:tasks.workunit.client.0.smithi032.stdout:5/345: creat da/dd/f68 x:0 0 0 2024-06-03T01:21:00.538 INFO:tasks.workunit.client.0.smithi032.stdout:9/410: truncate d0/d23/d25/f62 298644 0 2024-06-03T01:21:00.540 INFO:tasks.workunit.client.0.smithi032.stdout:3/316: mkdir de/d63 0 2024-06-03T01:21:00.540 INFO:tasks.workunit.client.0.smithi032.stdout:3/317: write de/d29/d2e/d31/d3c/d50/d56/f60 [619406,76652] 0 2024-06-03T01:21:00.540 INFO:tasks.workunit.client.1.smithi044.stdout:6/927: read d2/d23/d30/d1e/d38/f72 [2464094,89614] 0 2024-06-03T01:21:00.540 INFO:tasks.workunit.client.1.smithi044.stdout:6/928: dread - d2/d23/d55/d5f/d99/f127 zero size 2024-06-03T01:21:00.541 INFO:tasks.workunit.client.0.smithi032.stdout:9/411: creat d0/d23/d25/f8d x:0 0 0 2024-06-03T01:21:00.541 INFO:tasks.workunit.client.0.smithi032.stdout:9/412: truncate d0/d11/d1d/d38/d4f/f60 27469 0 2024-06-03T01:21:00.542 INFO:tasks.workunit.client.0.smithi032.stdout:8/271: dread dc/d14/f24 [0,4194304] 0 2024-06-03T01:21:00.543 INFO:tasks.workunit.client.0.smithi032.stdout:6/249: dwrite d8/ff [0,4194304] 0 2024-06-03T01:21:00.544 INFO:tasks.workunit.client.0.smithi032.stdout:9/413: unlink d0/d11/l13 0 2024-06-03T01:21:00.546 INFO:tasks.workunit.client.0.smithi032.stdout:8/272: mkdir dc/d25/d28/d51/d5d 0 2024-06-03T01:21:00.546 INFO:tasks.workunit.client.0.smithi032.stdout:8/273: fdatasync dc/d25/d28/d32/f50 0 2024-06-03T01:21:00.546 INFO:tasks.workunit.client.0.smithi032.stdout:8/274: fdatasync dc/d25/d28/d33/f48 0 2024-06-03T01:21:00.546 INFO:tasks.workunit.client.0.smithi032.stdout:8/275: readlink dc/d25/l49 0 2024-06-03T01:21:00.546 INFO:tasks.workunit.client.0.smithi032.stdout:6/250: stat d8/d11/d20/d2b/d39/d19/l1a 0 2024-06-03T01:21:00.551 INFO:tasks.workunit.client.1.smithi044.stdout:0/940: dwrite d3/f14 [0,4194304] 0 2024-06-03T01:21:00.551 INFO:tasks.workunit.client.1.smithi044.stdout:0/941: stat d3/d6b/d95/dae/lce 0 2024-06-03T01:21:00.551 INFO:tasks.workunit.client.1.smithi044.stdout:0/942: readlink d3/d5c/l66 0 2024-06-03T01:21:00.552 INFO:tasks.workunit.client.0.smithi032.stdout:9/414: link d0/d11/d1d/f1e d0/d23/d25/f8e 0 2024-06-03T01:21:00.552 INFO:tasks.workunit.client.0.smithi032.stdout:3/318: dread de/d29/f3b [0,4194304] 0 2024-06-03T01:21:00.554 INFO:tasks.workunit.client.0.smithi032.stdout:8/276: mkdir dc/d25/d28/d51/d5d/d5e 0 2024-06-03T01:21:00.555 INFO:tasks.workunit.client.1.smithi044.stdout:3/676: mknod df/d13/d6d/d9a/db7/cdc 0 2024-06-03T01:21:00.557 INFO:tasks.workunit.client.0.smithi032.stdout:6/251: mkdir d8/d11/d20/d3b/d4e 0 2024-06-03T01:21:00.558 INFO:tasks.workunit.client.1.smithi044.stdout:2/661: link l4 db/d1b/d5d/ld3 0 2024-06-03T01:21:00.558 INFO:tasks.workunit.client.1.smithi044.stdout:9/730: mknod d6/dd/d51/d16/d5f/cec 0 2024-06-03T01:21:00.559 INFO:tasks.workunit.client.1.smithi044.stdout:9/731: write d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf [1179683,44499] 0 2024-06-03T01:21:00.560 INFO:tasks.workunit.client.0.smithi032.stdout:0/290: dwrite d1/da/d28/d3b/d50/f58 [0,4194304] 0 2024-06-03T01:21:00.560 INFO:tasks.workunit.client.0.smithi032.stdout:0/291: read d1/f17 [2428494,2419] 0 2024-06-03T01:21:00.561 INFO:tasks.workunit.client.1.smithi044.stdout:0/943: link d3/da6/lba d3/d6b/l153 0 2024-06-03T01:21:00.561 INFO:tasks.workunit.client.1.smithi044.stdout:0/944: fdatasync d3/da6/dc1/dc2/de4/ff4 0 2024-06-03T01:21:00.561 INFO:tasks.workunit.client.1.smithi044.stdout:3/677: symlink df/d13/d1b/d56/ldd 0 2024-06-03T01:21:00.563 INFO:tasks.workunit.client.1.smithi044.stdout:9/732: symlink d6/dd/d51/d16/d27/d5e/d71/dc1/led 0 2024-06-03T01:21:00.563 INFO:tasks.workunit.client.0.smithi032.stdout:6/252: mkdir d8/d11/d20/d31/d3f/d4f 0 2024-06-03T01:21:00.564 INFO:tasks.workunit.client.1.smithi044.stdout:6/929: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/fdc [0,4194304] 0 2024-06-03T01:21:00.564 INFO:tasks.workunit.client.0.smithi032.stdout:5/346: dwrite da/dd/df/d10/d16/d18/d1a/d30/d51/f52 [0,4194304] 0 2024-06-03T01:21:00.566 INFO:tasks.workunit.client.1.smithi044.stdout:2/662: dread db/d1b/d3f/f4a [0,4194304] 0 2024-06-03T01:21:00.566 INFO:tasks.workunit.client.1.smithi044.stdout:0/945: unlink d3/da6/d21/d94/da3/dad/lf2 0 2024-06-03T01:21:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:0/946: dread - d3/da6/dc1/fe2 zero size 2024-06-03T01:21:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:0/947: truncate d3/d6b/d95/f135 824058 0 2024-06-03T01:21:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/663: fdatasync db/d15/d49/d6c/d72/d7a/f81 0 2024-06-03T01:21:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/664: write db/d1b/d27/f39 [1534857,26499] 0 2024-06-03T01:21:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/665: creat db/fd4 x:0 0 0 2024-06-03T01:21:00.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/733: mkdir d6/dd/d51/d16/dee 0 2024-06-03T01:21:00.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/292: unlink d1/da/d10/c1e 0 2024-06-03T01:21:00.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/293: chown d1/da/d10/d29/l4b 4129 1 2024-06-03T01:21:00.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/294: write d1/da/d28/f56 [656934,106820] 0 2024-06-03T01:21:00.571 INFO:tasks.workunit.client.1.smithi044.stdout:7/784: sync 2024-06-03T01:21:00.572 INFO:tasks.workunit.client.1.smithi044.stdout:5/730: sync 2024-06-03T01:21:00.572 INFO:tasks.workunit.client.1.smithi044.stdout:5/731: chown d2/d11/d12/cbe 3 1 2024-06-03T01:21:00.572 INFO:tasks.workunit.client.1.smithi044.stdout:7/785: rename d0/d3/d8/d95/d63/d7f to d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/dff 22 2024-06-03T01:21:00.572 INFO:tasks.workunit.client.0.smithi032.stdout:9/415: getdents d0 0 2024-06-03T01:21:00.578 INFO:tasks.workunit.client.1.smithi044.stdout:0/948: dread d3/d9/d36/f43 [0,4194304] 0 2024-06-03T01:21:00.578 INFO:tasks.workunit.client.1.smithi044.stdout:0/949: stat d3/da6/d20/d25/dbf/f120 0 2024-06-03T01:21:00.580 INFO:tasks.workunit.client.1.smithi044.stdout:2/666: unlink db/d1b/d3f/f61 0 2024-06-03T01:21:00.580 INFO:tasks.workunit.client.1.smithi044.stdout:2/667: creat db/d15/d49/d6c/d72/dcd/fd5 x:0 0 0 2024-06-03T01:21:00.585 INFO:tasks.workunit.client.0.smithi032.stdout:0/295: dread d1/d46/f48 [0,4194304] 0 2024-06-03T01:21:00.588 INFO:tasks.workunit.client.1.smithi044.stdout:0/950: mkdir d3/d6b/d154 0 2024-06-03T01:21:00.588 INFO:tasks.workunit.client.1.smithi044.stdout:3/678: dwrite df/f65 [0,4194304] 0 2024-06-03T01:21:00.589 INFO:tasks.workunit.client.0.smithi032.stdout:6/253: getdents d8/d11/d20/d31/d3f 0 2024-06-03T01:21:00.590 INFO:tasks.workunit.client.0.smithi032.stdout:9/416: creat d0/d11/d1d/f8f x:0 0 0 2024-06-03T01:21:00.593 INFO:tasks.workunit.client.1.smithi044.stdout:7/786: rmdir d0/d3/ddf 39 2024-06-03T01:21:00.594 INFO:tasks.workunit.client.0.smithi032.stdout:0/296: rename d1/da/d28/d3b/f3c to d1/da/d28/d3b/d50/f5c 0 2024-06-03T01:21:00.597 INFO:tasks.workunit.client.1.smithi044.stdout:0/951: truncate d3/f19 3725132 0 2024-06-03T01:21:00.598 INFO:tasks.workunit.client.1.smithi044.stdout:0/952: truncate d3/d5c/fcd 343127 0 2024-06-03T01:21:00.599 INFO:tasks.workunit.client.1.smithi044.stdout:3/679: creat df/d1e/d78/dda/fde x:0 0 0 2024-06-03T01:21:00.599 INFO:tasks.workunit.client.1.smithi044.stdout:3/680: dread - df/d13/d1b/d4b/d69/f95 zero size 2024-06-03T01:21:00.599 INFO:tasks.workunit.client.1.smithi044.stdout:3/681: stat df/d13/d39/d3d/d43/d71/lb8 0 2024-06-03T01:21:00.600 INFO:tasks.workunit.client.1.smithi044.stdout:3/682: chown df/f19 9 1 2024-06-03T01:21:00.600 INFO:tasks.workunit.client.1.smithi044.stdout:7/787: link d0/d3/d8/d95/d63/fce d0/d3/d8/d32/d33/dd1/f100 0 2024-06-03T01:21:00.601 INFO:tasks.workunit.client.1.smithi044.stdout:3/683: unlink df/d1e/d2a/d9b/fc3 0 2024-06-03T01:21:00.603 INFO:tasks.workunit.client.1.smithi044.stdout:3/684: mknod df/d1e/d2a/cdf 0 2024-06-03T01:21:00.603 INFO:tasks.workunit.client.1.smithi044.stdout:3/685: mknod df/d13/d6d/d9a/db7/ce0 0 2024-06-03T01:21:00.604 INFO:tasks.workunit.client.1.smithi044.stdout:3/686: fsync df/d1e/d3a/f4a 0 2024-06-03T01:21:00.625 INFO:tasks.workunit.client.1.smithi044.stdout:6/930: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/fdc [0,4194304] 0 2024-06-03T01:21:00.628 INFO:tasks.workunit.client.1.smithi044.stdout:5/732: dwrite d2/d11/d1e/d21/d24/d93/d9d/fb6 [0,4194304] 0 2024-06-03T01:21:00.629 INFO:tasks.workunit.client.1.smithi044.stdout:7/788: dread d0/d3/d8/d32/d33/fd6 [0,4194304] 0 2024-06-03T01:21:00.629 INFO:tasks.workunit.client.0.smithi032.stdout:5/347: dwrite da/dd/df/d10/f42 [0,4194304] 0 2024-06-03T01:21:00.629 INFO:tasks.workunit.client.0.smithi032.stdout:5/348: creat da/dd/df/d10/d16/d18/f69 x:0 0 0 2024-06-03T01:21:00.629 INFO:tasks.workunit.client.0.smithi032.stdout:5/349: write da/dd/df/d10/d16/d25/f66 [125511,40253] 0 2024-06-03T01:21:00.629 INFO:tasks.workunit.client.0.smithi032.stdout:9/417: rename d0/d23/d24/d26/c89 to d0/d11/d76/c90 0 2024-06-03T01:21:00.630 INFO:tasks.workunit.client.1.smithi044.stdout:9/734: dwrite d6/dd/fa3 [0,4194304] 0 2024-06-03T01:21:00.634 INFO:tasks.workunit.client.1.smithi044.stdout:6/931: mkdir d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/d102/d141 0 2024-06-03T01:21:00.634 INFO:tasks.workunit.client.1.smithi044.stdout:6/932: fdatasync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/d70/f131 0 2024-06-03T01:21:00.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/789: read d0/d3/f4f [1082747,84273] 0 2024-06-03T01:21:00.640 INFO:tasks.workunit.client.0.smithi032.stdout:5/350: dread da/dd/f48 [0,4194304] 0 2024-06-03T01:21:00.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/733: unlink d2/d11/d3e/ccc 0 2024-06-03T01:21:00.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/734: fdatasync d2/d11/d1e/d21/d24/d93/d9d/d28/f74 0 2024-06-03T01:21:00.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/735: getdents d2/d11/d3e/dce/dd1 0 2024-06-03T01:21:00.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/351: symlink da/dd/df/d10/d16/d18/d1a/d30/d58/l6a 0 2024-06-03T01:21:00.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/418: dread d0/d11/d1d/d38/d4f/f7a [0,4194304] 0 2024-06-03T01:21:00.649 INFO:tasks.workunit.client.0.smithi032.stdout:0/297: dwrite d1/f18 [0,4194304] 0 2024-06-03T01:21:00.649 INFO:tasks.workunit.client.0.smithi032.stdout:9/419: creat d0/d11/d1d/d4a/f91 x:0 0 0 2024-06-03T01:21:00.649 INFO:tasks.workunit.client.0.smithi032.stdout:6/254: dwrite d8/d11/d20/d2b/d39/d19/f1d [0,4194304] 0 2024-06-03T01:21:00.649 INFO:tasks.workunit.client.0.smithi032.stdout:0/298: write d1/f21 [640720,31906] 0 2024-06-03T01:21:00.650 INFO:tasks.workunit.client.1.smithi044.stdout:6/933: write d2/d23/d55/d5f/db2/d3f/fa1 [3652816,1319] 0 2024-06-03T01:21:00.650 INFO:tasks.workunit.client.1.smithi044.stdout:2/668: dwrite db/d15/d49/d6c/fa9 [0,4194304] 0 2024-06-03T01:21:00.650 INFO:tasks.workunit.client.1.smithi044.stdout:7/790: truncate d0/f7c 196196 0 2024-06-03T01:21:00.650 INFO:tasks.workunit.client.1.smithi044.stdout:9/735: rmdir d6/dd/d51/d16/d5f/d8b/d8d 39 2024-06-03T01:21:00.650 INFO:tasks.workunit.client.1.smithi044.stdout:0/953: dwrite d3/da6/dc1/dc2/de4/f122 [0,4194304] 0 2024-06-03T01:21:00.656 INFO:tasks.workunit.client.1.smithi044.stdout:2/669: dread db/d1b/d5d/f62 [4194304,4194304] 0 2024-06-03T01:21:00.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/352: write da/f15 [2139670,18841] 0 2024-06-03T01:21:00.658 INFO:tasks.workunit.client.0.smithi032.stdout:6/255: mkdir d8/d50 0 2024-06-03T01:21:00.658 INFO:tasks.workunit.client.0.smithi032.stdout:6/256: getdents d8/d11/d20/d21 0 2024-06-03T01:21:00.661 INFO:tasks.workunit.client.1.smithi044.stdout:6/934: write d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fac [2662308,85859] 0 2024-06-03T01:21:00.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/687: dwrite df/d13/f24 [4194304,4194304] 0 2024-06-03T01:21:00.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/688: fdatasync df/f10 0 2024-06-03T01:21:00.664 INFO:tasks.workunit.client.0.smithi032.stdout:6/257: rmdir d8/d11/d20/d31/d3f/d4f 0 2024-06-03T01:21:00.664 INFO:tasks.workunit.client.1.smithi044.stdout:7/791: unlink d0/d16/f4d 0 2024-06-03T01:21:00.666 INFO:tasks.workunit.client.1.smithi044.stdout:0/954: write d3/da6/d20/d25/f64 [1227969,121073] 0 2024-06-03T01:21:00.672 INFO:tasks.workunit.client.1.smithi044.stdout:3/689: write df/d1e/d2a/fc8 [41266,88005] 0 2024-06-03T01:21:00.677 INFO:tasks.workunit.client.0.smithi032.stdout:1/176: sync 2024-06-03T01:21:00.679 INFO:tasks.workunit.client.0.smithi032.stdout:1/177: creat d18/f34 x:0 0 0 2024-06-03T01:21:00.680 INFO:tasks.workunit.client.1.smithi044.stdout:2/670: unlink db/d1b/d5d/f95 0 2024-06-03T01:21:00.680 INFO:tasks.workunit.client.1.smithi044.stdout:2/671: write db/f19 [1635381,86395] 0 2024-06-03T01:21:00.680 INFO:tasks.workunit.client.1.smithi044.stdout:2/672: truncate db/d1b/d5d/f62 6774610 0 2024-06-03T01:21:00.681 INFO:tasks.workunit.client.0.smithi032.stdout:0/299: dwrite d1/f9 [4194304,4194304] 0 2024-06-03T01:21:00.682 INFO:tasks.workunit.client.0.smithi032.stdout:0/300: getdents d1/d46 0 2024-06-03T01:21:00.682 INFO:tasks.workunit.client.1.smithi044.stdout:6/935: rmdir d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/da8 39 2024-06-03T01:21:00.682 INFO:tasks.workunit.client.1.smithi044.stdout:6/936: write d2/d23/d55/d5f/db2/d3f/d106/d116/fb0 [646620,114245] 0 2024-06-03T01:21:00.682 INFO:tasks.workunit.client.1.smithi044.stdout:6/937: chown d2/d23/d55/d5f/db2/d3f/fa1 238 1 2024-06-03T01:21:00.683 INFO:tasks.workunit.client.0.smithi032.stdout:0/301: mkdir d1/da/d28/d3b/d5d 0 2024-06-03T01:21:00.683 INFO:tasks.workunit.client.0.smithi032.stdout:0/302: chown d1/da/d10/d29/l4b 62037524 1 2024-06-03T01:21:00.685 INFO:tasks.workunit.client.0.smithi032.stdout:9/420: dwrite d0/d11/d1d/d38/d4f/f7a [0,4194304] 0 2024-06-03T01:21:00.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/792: mkdir d0/d3/da8/de5/d101 0 2024-06-03T01:21:00.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/793: write d0/d3/d8/d32/f90 [1045566,29516] 0 2024-06-03T01:21:00.686 INFO:tasks.workunit.client.1.smithi044.stdout:9/736: dwrite d6/f84 [0,4194304] 0 2024-06-03T01:21:00.687 INFO:tasks.workunit.client.0.smithi032.stdout:9/421: mkdir d0/d23/d24/d92 0 2024-06-03T01:21:00.690 INFO:tasks.workunit.client.1.smithi044.stdout:7/794: creat d0/d3/d8/d95/d63/d89/da3/dde/f102 x:0 0 0 2024-06-03T01:21:00.693 INFO:tasks.workunit.client.0.smithi032.stdout:1/178: dread d18/f1c [0,4194304] 0 2024-06-03T01:21:00.694 INFO:tasks.workunit.client.1.smithi044.stdout:3/690: getdents df 0 2024-06-03T01:21:00.698 INFO:tasks.workunit.client.0.smithi032.stdout:5/353: dwrite da/dd/df/d10/d16/d18/d1a/d30/f5b [0,4194304] 0 2024-06-03T01:21:00.700 INFO:tasks.workunit.client.1.smithi044.stdout:2/673: getdents db/d1b/d2a/d67 0 2024-06-03T01:21:00.700 INFO:tasks.workunit.client.1.smithi044.stdout:2/674: write db/d1b/d2a/d52/d54/f56 [1160422,45702] 0 2024-06-03T01:21:00.701 INFO:tasks.workunit.client.1.smithi044.stdout:7/795: rmdir d0/d3/d8/d32/d33/dd1 39 2024-06-03T01:21:00.701 INFO:tasks.workunit.client.1.smithi044.stdout:9/737: dread d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68/f70 [0,4194304] 0 2024-06-03T01:21:00.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/675: symlink db/d15/d49/ld6 0 2024-06-03T01:21:00.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/676: chown db/d15/d49/d6c/fa9 0 1 2024-06-03T01:21:00.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/677: truncate db/d15/d49/d6c/d72/fd1 455650 0 2024-06-03T01:21:00.704 INFO:tasks.workunit.client.0.smithi032.stdout:6/258: dwrite d8/d11/d20/d31/f37 [0,4194304] 0 2024-06-03T01:21:00.704 INFO:tasks.workunit.client.0.smithi032.stdout:6/259: fsync d8/d11/f17 0 2024-06-03T01:21:00.704 INFO:tasks.workunit.client.0.smithi032.stdout:6/260: getdents d8/d11/d20/d21 0 2024-06-03T01:21:00.704 INFO:tasks.workunit.client.0.smithi032.stdout:6/261: stat d8/d11/d20/d31/f37 0 2024-06-03T01:21:00.705 INFO:tasks.workunit.client.1.smithi044.stdout:6/938: dwrite d2/d23/d30/d1e/d38/d6c/f92 [0,4194304] 0 2024-06-03T01:21:00.707 INFO:tasks.workunit.client.1.smithi044.stdout:2/678: mknod db/d1b/d2a/d52/d54/cd7 0 2024-06-03T01:21:00.707 INFO:tasks.workunit.client.1.smithi044.stdout:2/679: truncate db/d1b/f36 1064803 0 2024-06-03T01:21:00.707 INFO:tasks.workunit.client.0.smithi032.stdout:9/422: dwrite d0/f6 [0,4194304] 0 2024-06-03T01:21:00.710 INFO:tasks.workunit.client.1.smithi044.stdout:7/796: getdents d0/d3/d8/da0 0 2024-06-03T01:21:00.710 INFO:tasks.workunit.client.1.smithi044.stdout:7/797: chown d0/d16/fc4 746 1 2024-06-03T01:21:00.717 INFO:tasks.workunit.client.0.smithi032.stdout:0/303: getdents d1/da/d28/d3b 0 2024-06-03T01:21:00.719 INFO:tasks.workunit.client.0.smithi032.stdout:0/304: symlink d1/da/d28/d3b/d50/l5e 0 2024-06-03T01:21:00.719 INFO:tasks.workunit.client.0.smithi032.stdout:0/305: fsync d1/f9 0 2024-06-03T01:21:00.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/306: creat d1/d46/f5f x:0 0 0 2024-06-03T01:21:00.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/307: read d1/f17 [3803673,54028] 0 2024-06-03T01:21:00.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/308: creat d1/da/d10/d53/f60 x:0 0 0 2024-06-03T01:21:00.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/309: dread - d1/d46/f5f zero size 2024-06-03T01:21:00.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/310: chown d1/fc 2 1 2024-06-03T01:21:00.724 INFO:tasks.workunit.client.1.smithi044.stdout:1/946: sync 2024-06-03T01:21:00.724 INFO:tasks.workunit.client.1.smithi044.stdout:4/840: sync 2024-06-03T01:21:00.724 INFO:tasks.workunit.client.1.smithi044.stdout:8/835: sync 2024-06-03T01:21:00.725 INFO:tasks.workunit.client.0.smithi032.stdout:0/311: link d1/da/d28/d3b/d50/l5e d1/da/d28/d3b/d50/l61 0 2024-06-03T01:21:00.727 INFO:tasks.workunit.client.1.smithi044.stdout:2/680: mknod db/d15/cd8 0 2024-06-03T01:21:00.729 INFO:tasks.workunit.client.1.smithi044.stdout:7/798: symlink d0/d3/d8/d95/d63/d89/d8b/da9/l103 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:1/947: creat d0/d3/d20/d7/f143 x:0 0 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:1/948: getdents d0/d3/d20/def 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:1/949: creat d0/d3/d20/f144 x:0 0 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:1/950: fdatasync d0/d3/d20/d7/d2f/db7/f114 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:1/951: stat d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/ff2 0 2024-06-03T01:21:00.734 INFO:tasks.workunit.client.1.smithi044.stdout:4/841: mknod d17/d1c/d70/dfb/c11b 0 2024-06-03T01:21:00.735 INFO:tasks.workunit.client.1.smithi044.stdout:8/836: mkdir d5/d55/d56/d5c/d68/de5/dea/d116 0 2024-06-03T01:21:00.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/799: chown d0/d3/d8/d95/d63/d7f/cdd 0 1 2024-06-03T01:21:00.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/800: write d0/d3/d8/d32/ff2 [983845,63268] 0 2024-06-03T01:21:00.737 INFO:tasks.workunit.client.1.smithi044.stdout:6/939: dwrite d2/d23/d55/d5f/fa9 [0,4194304] 0 2024-06-03T01:21:00.739 INFO:tasks.workunit.client.1.smithi044.stdout:1/952: getdents d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c 0 2024-06-03T01:21:00.740 INFO:tasks.workunit.client.0.smithi032.stdout:6/262: dwrite d8/d11/d20/d2b/d39/d19/f46 [0,4194304] 0 2024-06-03T01:21:00.741 INFO:tasks.workunit.client.1.smithi044.stdout:4/842: mknod d17/d1c/d75/d7a/db8/d6a/d84/c11c 0 2024-06-03T01:21:00.742 INFO:tasks.workunit.client.1.smithi044.stdout:8/837: mkdir d5/dd4/df9/d117 0 2024-06-03T01:21:00.744 INFO:tasks.workunit.client.1.smithi044.stdout:2/681: getdents db/d1b/d2a/d2f/d4c 0 2024-06-03T01:21:00.744 INFO:tasks.workunit.client.1.smithi044.stdout:2/682: chown db/d1b/l23 414105 1 2024-06-03T01:21:00.744 INFO:tasks.workunit.client.1.smithi044.stdout:2/683: chown db/d15/d49/ld6 0 1 2024-06-03T01:21:00.745 INFO:tasks.workunit.client.1.smithi044.stdout:7/801: dread - d0/d3/d8/d95/d63/d89/ff9 zero size 2024-06-03T01:21:00.745 INFO:tasks.workunit.client.1.smithi044.stdout:7/802: truncate d0/d3/d8/d32/f38 4521368 0 2024-06-03T01:21:00.745 INFO:tasks.workunit.client.1.smithi044.stdout:7/803: chown d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 420573411 1 2024-06-03T01:21:00.747 INFO:tasks.workunit.client.1.smithi044.stdout:6/940: dread d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/fe9 [0,4194304] 0 2024-06-03T01:21:00.750 INFO:tasks.workunit.client.0.smithi032.stdout:0/312: rmdir d1/da/d28 39 2024-06-03T01:21:00.752 INFO:tasks.workunit.client.0.smithi032.stdout:0/313: mknod d1/d46/c62 0 2024-06-03T01:21:00.753 INFO:tasks.workunit.client.1.smithi044.stdout:1/953: truncate d0/d3/d20/d7/d2f/db7/fc7 553037 0 2024-06-03T01:21:00.754 INFO:tasks.workunit.client.0.smithi032.stdout:6/263: dread d8/d11/d20/d31/f45 [0,4194304] 0 2024-06-03T01:21:00.755 INFO:tasks.workunit.client.0.smithi032.stdout:0/314: symlink d1/l63 0 2024-06-03T01:21:00.755 INFO:tasks.workunit.client.1.smithi044.stdout:4/843: symlink d17/d1c/d9c/l11d 0 2024-06-03T01:21:00.755 INFO:tasks.workunit.client.1.smithi044.stdout:4/844: write d17/d1c/d75/d7a/db8/db9/ff1 [993979,59594] 0 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/838: mknod d5/d16/d1d/d59/c118 0 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/839: creat d5/d16/d1d/d32/f119 x:0 0 0 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/840: stat d5/d55/d56/d5c/d68/de5/dea/d116 0 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/841: chown d5/d55/d56/d5c/d68/de5/dea/de2/c10c 6680 1 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/842: dread - d5/d55/d56/d5c/d68/fcf zero size 2024-06-03T01:21:00.757 INFO:tasks.workunit.client.1.smithi044.stdout:7/804: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d104 0 2024-06-03T01:21:00.758 INFO:tasks.workunit.client.0.smithi032.stdout:6/264: rename d8/d11/d20/d2b/d39/l29 to d8/d11/d20/d31/l51 0 2024-06-03T01:21:00.758 INFO:tasks.workunit.client.0.smithi032.stdout:6/265: chown d8/d11/d20/d2b/d39/c22 217021 1 2024-06-03T01:21:00.758 INFO:tasks.workunit.client.0.smithi032.stdout:6/266: readlink d8/d11/d20/d2b/d39/d19/l1a 0 2024-06-03T01:21:00.759 INFO:tasks.workunit.client.1.smithi044.stdout:6/941: unlink d2/f27 0 2024-06-03T01:21:00.759 INFO:tasks.workunit.client.0.smithi032.stdout:6/267: mkdir d8/d4c/d52 0 2024-06-03T01:21:00.759 INFO:tasks.workunit.client.0.smithi032.stdout:6/268: symlink d8/d11/d20/d2b/d39/d19/l53 0 2024-06-03T01:21:00.759 INFO:tasks.workunit.client.0.smithi032.stdout:6/269: readlink l7 0 2024-06-03T01:21:00.760 INFO:tasks.workunit.client.0.smithi032.stdout:6/270: stat d8/d11/d28/c3a 0 2024-06-03T01:21:00.760 INFO:tasks.workunit.client.1.smithi044.stdout:4/845: rename d17/d1c/d75/d7a/db8/d28/d2b/lc0 to d17/d1c/d75/d7a/db8/d28/d2b/l11e 0 2024-06-03T01:21:00.760 INFO:tasks.workunit.client.1.smithi044.stdout:2/684: dwrite db/d15/d49/f8e [0,4194304] 0 2024-06-03T01:21:00.760 INFO:tasks.workunit.client.1.smithi044.stdout:2/685: stat db/d1b/d5d 0 2024-06-03T01:21:00.763 INFO:tasks.workunit.client.1.smithi044.stdout:6/942: mknod d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/c142 0 2024-06-03T01:21:00.766 INFO:tasks.workunit.client.1.smithi044.stdout:1/954: write d0/d3/d20/d7/f34 [485340,22324] 0 2024-06-03T01:21:00.766 INFO:tasks.workunit.client.0.smithi032.stdout:6/271: getdents d8/d4c 0 2024-06-03T01:21:00.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/272: rmdir d8/d11/d20/d36 39 2024-06-03T01:21:00.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/273: write d8/d11/f49 [216930,129819] 0 2024-06-03T01:21:00.769 INFO:tasks.workunit.client.1.smithi044.stdout:7/805: link d0/d3/d8/d32/d33/f42 d0/d3/d8/d95/d63/de6/f105 0 2024-06-03T01:21:00.770 INFO:tasks.workunit.client.1.smithi044.stdout:4/846: write d17/d63/fd0 [5738784,80624] 0 2024-06-03T01:21:00.770 INFO:tasks.workunit.client.1.smithi044.stdout:4/847: chown d17/d1c/d75/d7a/db8/d8d 97234 1 2024-06-03T01:21:00.771 INFO:tasks.workunit.client.0.smithi032.stdout:6/274: mknod d8/d11/c54 0 2024-06-03T01:21:00.771 INFO:tasks.workunit.client.0.smithi032.stdout:6/275: stat d8/d11/d20/d21/l26 0 2024-06-03T01:21:00.773 INFO:tasks.workunit.client.0.smithi032.stdout:6/276: rename d8/d11/d28 to d8/d55 0 2024-06-03T01:21:00.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/277: fsync d8/d11/d20/d3b/f41 0 2024-06-03T01:21:00.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/278: creat d8/d11/d20/d2b/d39/d19/f56 x:0 0 0 2024-06-03T01:21:00.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/279: chown d8/d11/d20/d21/f27 1583653 1 2024-06-03T01:21:00.781 INFO:tasks.workunit.client.1.smithi044.stdout:7/806: dread d0/d3/d8/d95/d63/d7f/f91 [4194304,4194304] 0 2024-06-03T01:21:00.786 INFO:tasks.workunit.client.1.smithi044.stdout:4/848: rmdir d17/d1c 39 2024-06-03T01:21:00.788 INFO:tasks.workunit.client.0.smithi032.stdout:1/179: dwrite f15 [0,4194304] 0 2024-06-03T01:21:00.788 INFO:tasks.workunit.client.0.smithi032.stdout:0/315: write d1/d46/f55 [633471,80106] 0 2024-06-03T01:21:00.788 INFO:tasks.workunit.client.1.smithi044.stdout:9/738: dwrite d6/dd/d51/d46/fb0 [0,4194304] 0 2024-06-03T01:21:00.788 INFO:tasks.workunit.client.1.smithi044.stdout:9/739: write d6/dd/d51/d16/d61/f6c [1521317,30407] 0 2024-06-03T01:21:00.790 INFO:tasks.workunit.client.1.smithi044.stdout:4/849: symlink d17/d63/d10c/l11f 0 2024-06-03T01:21:00.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/943: dwrite d2/d23/d30/d1e/d38/d77/fbd [0,4194304] 0 2024-06-03T01:21:00.795 INFO:tasks.workunit.client.0.smithi032.stdout:0/316: rename d1/da/d28/f51 to d1/f64 0 2024-06-03T01:21:00.797 INFO:tasks.workunit.client.1.smithi044.stdout:9/740: rmdir d6/dd/d51/d16/d27/d5e/db7 39 2024-06-03T01:21:00.800 INFO:tasks.workunit.client.0.smithi032.stdout:6/280: read f3 [839793,721] 0 2024-06-03T01:21:00.802 INFO:tasks.workunit.client.0.smithi032.stdout:6/281: mknod d8/d11/d20/c57 0 2024-06-03T01:21:00.802 INFO:tasks.workunit.client.0.smithi032.stdout:6/282: chown d8/d11/d20/d2b/f4a 8032238 1 2024-06-03T01:21:00.802 INFO:tasks.workunit.client.0.smithi032.stdout:6/283: readlink d8/d55/l30 0 2024-06-03T01:21:00.803 INFO:tasks.workunit.client.1.smithi044.stdout:2/686: read db/dd/f11 [1628389,89682] 0 2024-06-03T01:21:00.803 INFO:tasks.workunit.client.1.smithi044.stdout:3/691: dwrite df/f58 [0,4194304] 0 2024-06-03T01:21:00.803 INFO:tasks.workunit.client.0.smithi032.stdout:5/354: dwrite da/dd/df/f46 [4194304,4194304] 0 2024-06-03T01:21:00.804 INFO:tasks.workunit.client.0.smithi032.stdout:6/284: rename d8/d11/d20/d31/f37 to d8/d4c/f58 0 2024-06-03T01:21:00.807 INFO:tasks.workunit.client.1.smithi044.stdout:8/843: dwrite d5/dd4/df9/f113 [0,4194304] 0 2024-06-03T01:21:00.808 INFO:tasks.workunit.client.0.smithi032.stdout:5/355: truncate da/dd/df/f33 3798133 0 2024-06-03T01:21:00.809 INFO:tasks.workunit.client.1.smithi044.stdout:7/807: getdents d0/d3/d30 0 2024-06-03T01:21:00.809 INFO:tasks.workunit.client.0.smithi032.stdout:5/356: getdents da/dd/df/d10 0 2024-06-03T01:21:00.810 INFO:tasks.workunit.client.1.smithi044.stdout:1/955: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/d94/d116/f142 [0,4194304] 0 2024-06-03T01:21:00.810 INFO:tasks.workunit.client.1.smithi044.stdout:1/956: stat d0/dd2/dfe/dbe/le2 0 2024-06-03T01:21:00.811 INFO:tasks.workunit.client.1.smithi044.stdout:4/850: mknod d17/d1c/d70/dfb/c120 0 2024-06-03T01:21:00.812 INFO:tasks.workunit.client.0.smithi032.stdout:0/317: dread d1/d46/f55 [0,4194304] 0 2024-06-03T01:21:00.812 INFO:tasks.workunit.client.1.smithi044.stdout:2/687: dread db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:21:00.812 INFO:tasks.workunit.client.1.smithi044.stdout:2/688: creat db/d1b/d2a/d52/fd9 x:0 0 0 2024-06-03T01:21:00.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/285: dread d8/d11/f17 [0,4194304] 0 2024-06-03T01:21:00.813 INFO:tasks.workunit.client.0.smithi032.stdout:1/180: dwrite d18/f25 [4194304,4194304] 0 2024-06-03T01:21:00.813 INFO:tasks.workunit.client.0.smithi032.stdout:1/181: creat d18/d27/f35 x:0 0 0 2024-06-03T01:21:00.814 INFO:tasks.workunit.client.1.smithi044.stdout:3/692: rename df/l27 to df/d13/d1b/d56/le1 0 2024-06-03T01:21:00.816 INFO:tasks.workunit.client.0.smithi032.stdout:6/286: truncate d8/d11/d20/d21/f24 2406673 0 2024-06-03T01:21:00.816 INFO:tasks.workunit.client.0.smithi032.stdout:6/287: dread - d8/d11/d20/d3b/f41 zero size 2024-06-03T01:21:00.817 INFO:tasks.workunit.client.0.smithi032.stdout:6/288: dread - d8/d11/d20/d2b/f4a zero size 2024-06-03T01:21:00.820 INFO:tasks.workunit.client.1.smithi044.stdout:7/808: mkdir d0/d3/da8/de5/d106 0 2024-06-03T01:21:00.821 INFO:tasks.workunit.client.0.smithi032.stdout:1/182: dread d18/f1c [0,4194304] 0 2024-06-03T01:21:00.822 INFO:tasks.workunit.client.1.smithi044.stdout:1/957: symlink d0/d3/d20/def/l145 0 2024-06-03T01:21:00.823 INFO:tasks.workunit.client.1.smithi044.stdout:4/851: read d17/d1c/d75/d7a/db8/d28/f9d [362334,115609] 0 2024-06-03T01:21:00.823 INFO:tasks.workunit.client.1.smithi044.stdout:4/852: readlink d17/d1c/d75/d7a/db8/d6a/l112 0 2024-06-03T01:21:00.826 INFO:tasks.workunit.client.1.smithi044.stdout:2/689: creat db/d15/d49/d6c/d72/d8f/fda x:0 0 0 2024-06-03T01:21:00.826 INFO:tasks.workunit.client.1.smithi044.stdout:2/690: write db/dd/f60 [1803688,7443] 0 2024-06-03T01:21:00.826 INFO:tasks.workunit.client.1.smithi044.stdout:3/693: mknod df/d13/d6d/d9a/db7/ce2 0 2024-06-03T01:21:00.827 INFO:tasks.workunit.client.1.smithi044.stdout:3/694: write df/d13/d39/d3d/d43/d71/f94 [402107,121443] 0 2024-06-03T01:21:00.828 INFO:tasks.workunit.client.1.smithi044.stdout:1/958: mknod d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5c/c146 0 2024-06-03T01:21:00.829 INFO:tasks.workunit.client.1.smithi044.stdout:1/959: write d0/d3/d20/d7/d13/fab [795476,86658] 0 2024-06-03T01:21:00.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/853: mknod d17/d1c/d75/d7a/db8/d28/d2b/d8e/c121 0 2024-06-03T01:21:00.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/854: rename d17/d1c/d75/d7a/db8/d28 to d17/d1c/d75/d7a/db8/d28/d2b/d122 22 2024-06-03T01:21:00.832 INFO:tasks.workunit.client.0.smithi032.stdout:6/289: dread d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:21:00.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/691: symlink db/d1b/ldb 0 2024-06-03T01:21:00.836 INFO:tasks.workunit.client.1.smithi044.stdout:3/695: mkdir df/d13/d1b/d56/d5c/de3 0 2024-06-03T01:21:00.840 INFO:tasks.workunit.client.1.smithi044.stdout:8/844: dwrite d5/d55/d56/d5c/dee/f103 [0,4194304] 0 2024-06-03T01:21:00.843 INFO:tasks.workunit.client.0.smithi032.stdout:6/290: read d8/d4c/f58 [3876095,111746] 0 2024-06-03T01:21:00.843 INFO:tasks.workunit.client.1.smithi044.stdout:8/845: truncate d5/d20/f69 1133520 0 2024-06-03T01:21:00.845 INFO:tasks.workunit.client.1.smithi044.stdout:8/846: mknod d5/d16/d1d/d59/c11a 0 2024-06-03T01:21:00.845 INFO:tasks.workunit.client.0.smithi032.stdout:6/291: rmdir d8/d11/d20/d3b/d4e 0 2024-06-03T01:21:00.845 INFO:tasks.workunit.client.0.smithi032.stdout:6/292: fsync d8/fb 0 2024-06-03T01:21:00.845 INFO:tasks.workunit.client.0.smithi032.stdout:6/293: fdatasync d8/ff 0 2024-06-03T01:21:00.847 INFO:tasks.workunit.client.0.smithi032.stdout:6/294: symlink d8/d11/d20/d2b/l59 0 2024-06-03T01:21:00.848 INFO:tasks.workunit.client.0.smithi032.stdout:6/295: stat d8/d11/d20/d31/l51 0 2024-06-03T01:21:00.848 INFO:tasks.workunit.client.0.smithi032.stdout:6/296: truncate d8/d11/d20/d2b/d39/d19/f4d 840545 0 2024-06-03T01:21:00.850 INFO:tasks.workunit.client.0.smithi032.stdout:6/297: symlink d8/d11/d20/d31/d3f/l5a 0 2024-06-03T01:21:00.852 INFO:tasks.workunit.client.1.smithi044.stdout:2/692: write db/d1b/d2a/d52/d54/f56 [291733,7253] 0 2024-06-03T01:21:00.852 INFO:tasks.workunit.client.1.smithi044.stdout:2/693: creat db/d1b/d2a/d52/fdc x:0 0 0 2024-06-03T01:21:00.853 INFO:tasks.workunit.client.1.smithi044.stdout:5/736: sync 2024-06-03T01:21:00.853 INFO:tasks.workunit.client.0.smithi032.stdout:5/357: dwrite da/f15 [4194304,4194304] 0 2024-06-03T01:21:00.854 INFO:tasks.workunit.client.1.smithi044.stdout:1/960: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/ff2 [0,4194304] 0 2024-06-03T01:21:00.859 INFO:tasks.workunit.client.1.smithi044.stdout:5/737: symlink d2/d11/d12/dad/lf0 0 2024-06-03T01:21:00.860 INFO:tasks.workunit.client.1.smithi044.stdout:3/696: write df/d1e/d78/da4/fbf [2831367,15888] 0 2024-06-03T01:21:00.863 INFO:tasks.workunit.client.1.smithi044.stdout:1/961: mknod d0/d3/d20/d7/d2f/c147 0 2024-06-03T01:21:00.863 INFO:tasks.workunit.client.0.smithi032.stdout:6/298: dread d8/d11/f35 [0,4194304] 0 2024-06-03T01:21:00.863 INFO:tasks.workunit.client.0.smithi032.stdout:6/299: chown d8/lc 16042276 1 2024-06-03T01:21:00.865 INFO:tasks.workunit.client.0.smithi032.stdout:6/300: mknod d8/d11/d20/c5b 0 2024-06-03T01:21:00.866 INFO:tasks.workunit.client.0.smithi032.stdout:6/301: write d8/d11/f3e [86341,113097] 0 2024-06-03T01:21:00.866 INFO:tasks.workunit.client.0.smithi032.stdout:6/302: fsync d8/d11/f35 0 2024-06-03T01:21:00.866 INFO:tasks.workunit.client.1.smithi044.stdout:3/697: mknod df/d13/d39/ce4 0 2024-06-03T01:21:00.866 INFO:tasks.workunit.client.0.smithi032.stdout:1/183: dwrite d18/f1a [0,4194304] 0 2024-06-03T01:21:00.868 INFO:tasks.workunit.client.0.smithi032.stdout:6/303: symlink d8/d55/l5c 0 2024-06-03T01:21:00.868 INFO:tasks.workunit.client.1.smithi044.stdout:1/962: symlink d0/d3/d20/d7/d13/d2d/l148 0 2024-06-03T01:21:00.868 INFO:tasks.workunit.client.1.smithi044.stdout:1/963: chown d0/dd2/dfe/dbe/f75 179 1 2024-06-03T01:21:00.868 INFO:tasks.workunit.client.1.smithi044.stdout:7/809: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/fd7 [0,4194304] 0 2024-06-03T01:21:00.868 INFO:tasks.workunit.client.1.smithi044.stdout:7/810: fdatasync d0/d16/fea 0 2024-06-03T01:21:00.869 INFO:tasks.workunit.client.0.smithi032.stdout:6/304: getdents d8/d4c 0 2024-06-03T01:21:00.869 INFO:tasks.workunit.client.0.smithi032.stdout:6/305: read f4 [3277091,39700] 0 2024-06-03T01:21:00.870 INFO:tasks.workunit.client.0.smithi032.stdout:7/299: sync 2024-06-03T01:21:00.871 INFO:tasks.workunit.client.0.smithi032.stdout:6/306: getdents d8/d11/d20/d2b/d39 0 2024-06-03T01:21:00.872 INFO:tasks.workunit.client.1.smithi044.stdout:1/964: mkdir d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/d149 0 2024-06-03T01:21:00.873 INFO:tasks.workunit.client.1.smithi044.stdout:5/738: dread d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [0,4194304] 0 2024-06-03T01:21:00.873 INFO:tasks.workunit.client.1.smithi044.stdout:5/739: stat d2/d11/d12/d2b/d4a/d57 0 2024-06-03T01:21:00.873 INFO:tasks.workunit.client.0.smithi032.stdout:6/307: link d8/d11/d20/c5b d8/d11/d20/c5d 0 2024-06-03T01:21:00.874 INFO:tasks.workunit.client.0.smithi032.stdout:6/308: symlink d8/d55/l5e 0 2024-06-03T01:21:00.875 INFO:tasks.workunit.client.1.smithi044.stdout:5/740: dread d2/d11/d1e/d21/d24/d93/d9d/d28/f2c [0,4194304] 0 2024-06-03T01:21:00.875 INFO:tasks.workunit.client.1.smithi044.stdout:5/741: stat d2/d11/d1e/d21/d24/f69 0 2024-06-03T01:21:00.875 INFO:tasks.workunit.client.0.smithi032.stdout:6/309: mkdir d8/d5f 0 2024-06-03T01:21:00.875 INFO:tasks.workunit.client.0.smithi032.stdout:6/310: write d8/d11/d20/d2b/d39/d19/f40 [677681,12449] 0 2024-06-03T01:21:00.876 INFO:tasks.workunit.client.1.smithi044.stdout:8/847: dwrite d5/d55/d56/fde [0,4194304] 0 2024-06-03T01:21:00.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/300: write dc/d12/d2a/f43 [2257393,70942] 0 2024-06-03T01:21:00.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/301: creat dc/d12/d2a/d2f/f5d x:0 0 0 2024-06-03T01:21:00.884 INFO:tasks.workunit.client.1.smithi044.stdout:1/965: rename d0/d3/d20/def to d0/d3/d20/d7/d14a 0 2024-06-03T01:21:00.884 INFO:tasks.workunit.client.1.smithi044.stdout:1/966: fsync d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d81/ffa 0 2024-06-03T01:21:00.884 INFO:tasks.workunit.client.1.smithi044.stdout:1/967: write d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d30/f117 [1537339,24360] 0 2024-06-03T01:21:00.885 INFO:tasks.workunit.client.1.smithi044.stdout:5/742: mkdir d2/d11/df1 0 2024-06-03T01:21:00.888 INFO:tasks.workunit.client.0.smithi032.stdout:7/302: rename dc/f16 to dc/d12/d22/d42/d4e/f5e 0 2024-06-03T01:21:00.888 INFO:tasks.workunit.client.0.smithi032.stdout:7/303: readlink dc/d12/d2a/d2f/l59 0 2024-06-03T01:21:00.890 INFO:tasks.workunit.client.1.smithi044.stdout:2/694: dwrite db/d1b/d2a/d52/f8b [0,4194304] 0 2024-06-03T01:21:00.890 INFO:tasks.workunit.client.1.smithi044.stdout:7/811: write d0/d3/d8/d95/d63/feb [3944187,24656] 0 2024-06-03T01:21:00.890 INFO:tasks.workunit.client.0.smithi032.stdout:6/311: dread d8/fd [0,4194304] 0 2024-06-03T01:21:00.891 INFO:tasks.workunit.client.0.smithi032.stdout:6/312: chown d8/d55/l30 15228832 1 2024-06-03T01:21:00.892 INFO:tasks.workunit.client.1.smithi044.stdout:5/743: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/ff2 x:0 0 0 2024-06-03T01:21:00.892 INFO:tasks.workunit.client.1.smithi044.stdout:5/744: dread - d2/d11/d1e/d21/fe6 zero size 2024-06-03T01:21:00.893 INFO:tasks.workunit.client.0.smithi032.stdout:5/358: dwrite da/dd/df/d10/d16/d18/f50 [0,4194304] 0 2024-06-03T01:21:00.893 INFO:tasks.workunit.client.0.smithi032.stdout:5/359: chown da/dd/df/d10/d16/d18/d1a/d30/d58/l62 4 1 2024-06-03T01:21:00.895 INFO:tasks.workunit.client.0.smithi032.stdout:1/184: dwrite d18/d27/f31 [0,4194304] 0 2024-06-03T01:21:00.895 INFO:tasks.workunit.client.0.smithi032.stdout:5/360: symlink da/dd/df/d10/d16/d18/d1a/d30/d58/l6b 0 2024-06-03T01:21:00.896 INFO:tasks.workunit.client.1.smithi044.stdout:3/698: rmdir df/d13/d1b 39 2024-06-03T01:21:00.899 INFO:tasks.workunit.client.1.smithi044.stdout:1/968: dread d0/d3/d20/d7/d13/fd8 [0,4194304] 0 2024-06-03T01:21:00.899 INFO:tasks.workunit.client.1.smithi044.stdout:1/969: fdatasync d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/fe5 0 2024-06-03T01:21:00.899 INFO:tasks.workunit.client.1.smithi044.stdout:1/970: dread - d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/d64/ff6 zero size 2024-06-03T01:21:00.901 INFO:tasks.workunit.client.0.smithi032.stdout:1/185: mknod d18/c36 0 2024-06-03T01:21:00.902 INFO:tasks.workunit.client.0.smithi032.stdout:1/186: symlink d18/d32/l37 0 2024-06-03T01:21:00.902 INFO:tasks.workunit.client.1.smithi044.stdout:4/855: dwrite d17/d1c/d9c/fbd [0,4194304] 0 2024-06-03T01:21:00.904 INFO:tasks.workunit.client.1.smithi044.stdout:5/745: symlink d2/d11/d1e/d21/d24/d9e/db1/deb/lf3 0 2024-06-03T01:21:00.905 INFO:tasks.workunit.client.0.smithi032.stdout:1/187: link d18/l2c d18/d2d/d2f/l38 0 2024-06-03T01:21:00.906 INFO:tasks.workunit.client.1.smithi044.stdout:2/695: rmdir db/d1b/d3f 39 2024-06-03T01:21:00.906 INFO:tasks.workunit.client.1.smithi044.stdout:2/696: fsync db/f26 0 2024-06-03T01:21:00.909 INFO:tasks.workunit.client.0.smithi032.stdout:6/313: read d8/d11/d20/d36/f38 [2815369,110787] 0 2024-06-03T01:21:00.911 INFO:tasks.workunit.client.0.smithi032.stdout:6/314: unlink c2 0 2024-06-03T01:21:00.915 INFO:tasks.workunit.client.1.smithi044.stdout:5/746: symlink d2/d11/d12/d2b/d44/lf4 0 2024-06-03T01:21:00.916 INFO:tasks.workunit.client.1.smithi044.stdout:5/747: dread - d2/d11/fb2 zero size 2024-06-03T01:21:00.918 INFO:tasks.workunit.client.1.smithi044.stdout:3/699: mkdir df/d13/de5 0 2024-06-03T01:21:00.919 INFO:tasks.workunit.client.1.smithi044.stdout:2/697: mknod db/d15/d49/d6c/d72/dcd/cdd 0 2024-06-03T01:21:00.922 INFO:tasks.workunit.client.1.smithi044.stdout:5/748: mkdir d2/d11/d1e/d2f/df5 0 2024-06-03T01:21:00.923 INFO:tasks.workunit.client.1.smithi044.stdout:4/856: write d17/d1c/d9c/fbd [891858,35477] 0 2024-06-03T01:21:00.923 INFO:tasks.workunit.client.1.smithi044.stdout:8/848: dwrite d5/d55/d56/d5c/d68/de5/dea/d79/d88/fc2 [0,4194304] 0 2024-06-03T01:21:00.923 INFO:tasks.workunit.client.1.smithi044.stdout:4/857: truncate d17/d1c/d75/d7a/db8/d28/d2b/d8e/f107 863650 0 2024-06-03T01:21:00.924 INFO:tasks.workunit.client.1.smithi044.stdout:3/700: mkdir df/d13/d1b/d56/d5c/de3/de6 0 2024-06-03T01:21:00.924 INFO:tasks.workunit.client.1.smithi044.stdout:3/701: truncate df/d1e/d2a/d70/fa3 911873 0 2024-06-03T01:21:00.926 INFO:tasks.workunit.client.0.smithi032.stdout:7/304: dwrite dc/d12/d2a/d2f/f5d [0,4194304] 0 2024-06-03T01:21:00.926 INFO:tasks.workunit.client.0.smithi032.stdout:7/305: chown dc/d12/d24 29520297 1 2024-06-03T01:21:00.927 INFO:tasks.workunit.client.1.smithi044.stdout:2/698: rename db/d1b/d2a/d52/d54/cc5 to db/d1b/d27/da5/cde 0 2024-06-03T01:21:00.929 INFO:tasks.workunit.client.1.smithi044.stdout:5/749: symlink d2/d11/d1e/d21/d24/d93/d9d/lf6 0 2024-06-03T01:21:00.929 INFO:tasks.workunit.client.1.smithi044.stdout:5/750: chown d2/d11/d1e/d21/f75 455 1 2024-06-03T01:21:00.929 INFO:tasks.workunit.client.1.smithi044.stdout:5/751: dread - d2/d11/d3e/f73 zero size 2024-06-03T01:21:00.930 INFO:tasks.workunit.client.0.smithi032.stdout:5/361: dwrite da/dd/f29 [0,4194304] 0 2024-06-03T01:21:00.930 INFO:tasks.workunit.client.0.smithi032.stdout:7/306: creat dc/d12/d22/d42/d4e/f5f x:0 0 0 2024-06-03T01:21:00.931 INFO:tasks.workunit.client.1.smithi044.stdout:7/812: dwrite d0/f9f [0,4194304] 0 2024-06-03T01:21:00.931 INFO:tasks.workunit.client.1.smithi044.stdout:1/971: dwrite d0/dd2/dfe/dbe/d9e/d11b/f122 [0,4194304] 0 2024-06-03T01:21:00.932 INFO:tasks.workunit.client.0.smithi032.stdout:7/307: write dc/f3d [186820,42046] 0 2024-06-03T01:21:00.932 INFO:tasks.workunit.client.0.smithi032.stdout:1/188: dwrite fa [0,4194304] 0 2024-06-03T01:21:00.932 INFO:tasks.workunit.client.0.smithi032.stdout:6/315: fsync d8/d11/d20/d2b/d39/d19/f40 0 2024-06-03T01:21:00.932 INFO:tasks.workunit.client.0.smithi032.stdout:6/316: write d8/d11/d20/d36/f38 [952563,22505] 0 2024-06-03T01:21:00.933 INFO:tasks.workunit.client.0.smithi032.stdout:1/189: mkdir d18/d32/d39 0 2024-06-03T01:21:00.935 INFO:tasks.workunit.client.0.smithi032.stdout:5/362: getdents da/dd/df/d10/d16/d18/d35 0 2024-06-03T01:21:00.937 INFO:tasks.workunit.client.1.smithi044.stdout:3/702: mknod df/d1e/d2a/d70/dbc/ce7 0 2024-06-03T01:21:00.937 INFO:tasks.workunit.client.1.smithi044.stdout:3/703: truncate df/d13/d1b/f73 762632 0 2024-06-03T01:21:00.937 INFO:tasks.workunit.client.1.smithi044.stdout:3/704: fsync df/d13/d1b/fc6 0 2024-06-03T01:21:00.939 INFO:tasks.workunit.client.0.smithi032.stdout:1/190: symlink d18/d27/l3a 0 2024-06-03T01:21:00.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/858: rename d17/d1c/d75/d7a/db8/d28/d2b/d8e/ldf to d17/d1c/d75/d7a/db8/d35/l123 0 2024-06-03T01:21:00.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/859: write d17/d1c/d75/d7a/db3/de9/f108 [346580,112852] 0 2024-06-03T01:21:00.943 INFO:tasks.workunit.client.0.smithi032.stdout:5/363: symlink da/dd/df/d10/d16/d18/l6c 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:5/364: write da/dd/f5e [1013542,62149] 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/317: dread d8/d11/d20/d2b/d39/d19/f46 [0,4194304] 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/318: chown d8/d11/d20/c5d 10957 1 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/319: getdents d8/d11/d20/d21 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/320: readlink d8/d11/d20/d2b/d39/d19/l1a 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/321: truncate d8/d11/f34 570228 0 2024-06-03T01:21:00.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/322: chown d8/d55/c32 8 1 2024-06-03T01:21:00.946 INFO:tasks.workunit.client.1.smithi044.stdout:2/699: link db/d15/d49/cb0 db/d1b/d2a/d67/cdf 0 2024-06-03T01:21:00.948 INFO:tasks.workunit.client.0.smithi032.stdout:1/191: link d18/f19 d18/d2d/d2f/f3b 0 2024-06-03T01:21:00.951 INFO:tasks.workunit.client.0.smithi032.stdout:5/365: rename da/dd/f2d to da/dd/df/d10/d16/d18/d1a/f6d 0 2024-06-03T01:21:00.952 INFO:tasks.workunit.client.0.smithi032.stdout:5/366: chown da/lc 11 1 2024-06-03T01:21:00.952 INFO:tasks.workunit.client.0.smithi032.stdout:5/367: write da/dd/df/d10/f39 [3626915,45300] 0 2024-06-03T01:21:00.955 INFO:tasks.workunit.client.0.smithi032.stdout:1/192: truncate d18/f19 1902684 0 2024-06-03T01:21:00.956 INFO:tasks.workunit.client.1.smithi044.stdout:1/972: dwrite d0/dd2/dfe/dbe/d9e/fa9 [4194304,4194304] 0 2024-06-03T01:21:00.958 INFO:tasks.workunit.client.0.smithi032.stdout:5/368: rmdir da/dd/df/d10/d16/d18/d1a/d30/d51/d64 0 2024-06-03T01:21:00.959 INFO:tasks.workunit.client.0.smithi032.stdout:5/369: read da/dd/df/d10/f39 [370551,70572] 0 2024-06-03T01:21:00.959 INFO:tasks.workunit.client.0.smithi032.stdout:5/370: chown da/dd/df/d10/d16/d18/f69 2760 1 2024-06-03T01:21:00.960 INFO:tasks.workunit.client.0.smithi032.stdout:6/323: rmdir d8/d11 39 2024-06-03T01:21:00.964 INFO:tasks.workunit.client.1.smithi044.stdout:5/752: dwrite d2/d11/d1e/d21/d24/f77 [0,4194304] 0 2024-06-03T01:21:00.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/193: mkdir d18/d32/d3c 0 2024-06-03T01:21:00.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/194: write d18/d27/f31 [4289911,10769] 0 2024-06-03T01:21:00.965 INFO:tasks.workunit.client.0.smithi032.stdout:7/308: dwrite dc/f54 [0,4194304] 0 2024-06-03T01:21:00.968 INFO:tasks.workunit.client.1.smithi044.stdout:8/849: rename d5/d16/d1d/d59/la8 to d5/d55/d56/l11b 0 2024-06-03T01:21:00.968 INFO:tasks.workunit.client.1.smithi044.stdout:3/705: dread df/d13/d1b/f38 [0,4194304] 0 2024-06-03T01:21:00.968 INFO:tasks.workunit.client.0.smithi032.stdout:6/324: unlink d8/d11/d20/d2b/d39/c2f 0 2024-06-03T01:21:00.970 INFO:tasks.workunit.client.1.smithi044.stdout:7/813: rmdir d0/d3/da8 39 2024-06-03T01:21:00.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/850: mknod d5/d16/d81/de3/d58/d89/dbf/c11c 0 2024-06-03T01:21:00.972 INFO:tasks.workunit.client.1.smithi044.stdout:1/973: write d0/d3/d20/f6d [2556891,107732] 0 2024-06-03T01:21:00.974 INFO:tasks.workunit.client.1.smithi044.stdout:2/700: rename db/d15/d49/d6c/d72/c98 to db/d15/d49/d6c/d72/dcd/ce0 0 2024-06-03T01:21:00.977 INFO:tasks.workunit.client.0.smithi032.stdout:7/309: readlink l2 0 2024-06-03T01:21:00.978 INFO:tasks.workunit.client.1.smithi044.stdout:7/814: dread d0/d3/f4f [0,4194304] 0 2024-06-03T01:21:00.980 INFO:tasks.workunit.client.1.smithi044.stdout:3/706: rename df/d13/d1b/d4b/c61 to df/d1e/d8a/ce8 0 2024-06-03T01:21:00.980 INFO:tasks.workunit.client.1.smithi044.stdout:1/974: mkdir d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d30/d14b 0 2024-06-03T01:21:00.980 INFO:tasks.workunit.client.1.smithi044.stdout:4/860: dwrite d17/d1c/d75/d7a/db3/f10a [0,4194304] 0 2024-06-03T01:21:00.983 INFO:tasks.workunit.client.1.smithi044.stdout:7/815: mknod d0/d3/d8/d95/d63/de6/c107 0 2024-06-03T01:21:00.983 INFO:tasks.workunit.client.1.smithi044.stdout:3/707: unlink df/d13/d1b/d56/d5c/f8d 0 2024-06-03T01:21:00.984 INFO:tasks.workunit.client.1.smithi044.stdout:3/708: stat df/d13/d1b/fae 0 2024-06-03T01:21:00.989 INFO:tasks.workunit.client.1.smithi044.stdout:4/861: dread d17/d1c/d75/f76 [0,4194304] 0 2024-06-03T01:21:00.991 INFO:tasks.workunit.client.1.smithi044.stdout:7/816: mknod d0/d3/d8/d95/d63/d89/da3/d9a/c108 0 2024-06-03T01:21:00.993 INFO:tasks.workunit.client.1.smithi044.stdout:3/709: mkdir df/d13/d39/d3d/de9 0 2024-06-03T01:21:00.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/862: unlink d17/d63/dae/lcd 0 2024-06-03T01:21:00.998 INFO:tasks.workunit.client.1.smithi044.stdout:3/710: truncate f9 111171 0 2024-06-03T01:21:00.998 INFO:tasks.workunit.client.1.smithi044.stdout:3/711: fsync df/d1e/d2a/d70/fbe 0 2024-06-03T01:21:00.998 INFO:tasks.workunit.client.1.smithi044.stdout:3/712: readlink df/d13/d39/d3d/d50/lcf 0 2024-06-03T01:21:01.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/713: creat df/d13/d1b/d4b/fea x:0 0 0 2024-06-03T01:21:01.002 INFO:tasks.workunit.client.1.smithi044.stdout:3/714: symlink df/d1e/d3a/leb 0 2024-06-03T01:21:01.002 INFO:tasks.workunit.client.1.smithi044.stdout:3/715: rename df/d13/d1b/d56 to df/d13/d1b/d56/d5c/dec 22 2024-06-03T01:21:01.003 INFO:tasks.workunit.client.1.smithi044.stdout:3/716: truncate df/d13/d6d/d9a/dad/fd6 133796 0 2024-06-03T01:21:01.003 INFO:tasks.workunit.client.1.smithi044.stdout:0/955: sync 2024-06-03T01:21:01.004 INFO:tasks.workunit.client.1.smithi044.stdout:0/956: symlink d3/d5c/l155 0 2024-06-03T01:21:01.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/957: rename d3/d9/d102/l14f to d3/d9/d55/d7b/dfc/d10c/l156 0 2024-06-03T01:21:01.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/958: write d3/f3a [4988393,89430] 0 2024-06-03T01:21:01.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/959: creat d3/d5c/de3/f157 x:0 0 0 2024-06-03T01:21:01.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/960: creat d3/da6/d21/d94/da3/dad/f158 x:0 0 0 2024-06-03T01:21:01.010 INFO:tasks.workunit.client.1.smithi044.stdout:9/741: sync 2024-06-03T01:21:01.010 INFO:tasks.workunit.client.1.smithi044.stdout:6/944: sync 2024-06-03T01:21:01.010 INFO:tasks.workunit.client.1.smithi044.stdout:6/945: creat d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/f143 x:0 0 0 2024-06-03T01:21:01.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/742: getdents d6/dd/d51/d16/d5f/d8b 0 2024-06-03T01:21:01.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/743: write d6/dd/d51/fe [3455687,40550] 0 2024-06-03T01:21:01.013 INFO:tasks.workunit.client.1.smithi044.stdout:6/946: link d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/da8/le4 d2/d23/d55/d5f/db2/dca/l144 0 2024-06-03T01:21:01.014 INFO:tasks.workunit.client.1.smithi044.stdout:9/744: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/lef 0 2024-06-03T01:21:01.016 INFO:tasks.workunit.client.1.smithi044.stdout:6/947: mkdir d2/d23/d30/d1e/d38/d145 0 2024-06-03T01:21:01.018 INFO:tasks.workunit.client.1.smithi044.stdout:6/948: rename d2/d23/d30/c26 to d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/c146 0 2024-06-03T01:21:01.018 INFO:tasks.workunit.client.1.smithi044.stdout:6/949: stat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d9b/db4/d102 0 2024-06-03T01:21:01.025 INFO:tasks.workunit.client.0.smithi032.stdout:5/371: dwrite da/dd/f29 [0,4194304] 0 2024-06-03T01:21:01.034 INFO:tasks.workunit.client.1.smithi044.stdout:1/975: dwrite d0/fba [0,4194304] 0 2024-06-03T01:21:01.037 INFO:tasks.workunit.client.1.smithi044.stdout:1/976: truncate d0/d3/d20/fd7 756802 0 2024-06-03T01:21:01.037 INFO:tasks.workunit.client.1.smithi044.stdout:1/977: fdatasync d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/fbd 0 2024-06-03T01:21:01.045 INFO:tasks.workunit.client.0.smithi032.stdout:6/325: dwrite d8/fb [0,4194304] 0 2024-06-03T01:21:01.047 INFO:tasks.workunit.client.0.smithi032.stdout:6/326: mknod d8/d11/d20/d2b/d39/d19/c60 0 2024-06-03T01:21:01.048 INFO:tasks.workunit.client.1.smithi044.stdout:5/753: dwrite d2/d11/d12/d2b/d4a/d57/f76 [0,4194304] 0 2024-06-03T01:21:01.048 INFO:tasks.workunit.client.0.smithi032.stdout:1/195: dwrite d18/d29/f2e [0,4194304] 0 2024-06-03T01:21:01.050 INFO:tasks.workunit.client.0.smithi032.stdout:6/327: symlink d8/d11/d20/d3b/l61 0 2024-06-03T01:21:01.050 INFO:tasks.workunit.client.0.smithi032.stdout:6/328: chown d8/d11/d20/d2b/d39/d19/f40 1 1 2024-06-03T01:21:01.051 INFO:tasks.workunit.client.0.smithi032.stdout:1/196: mkdir d18/d29/d3d 0 2024-06-03T01:21:01.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/197: fdatasync d18/d29/f2e 0 2024-06-03T01:21:01.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/198: dread - d18/d27/f35 zero size 2024-06-03T01:21:01.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/199: fdatasync f17 0 2024-06-03T01:21:01.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/200: getdents d18/d32/d39 0 2024-06-03T01:21:01.053 INFO:tasks.workunit.client.1.smithi044.stdout:5/754: creat d2/d11/d1e/d21/d24/d9e/dbb/ff7 x:0 0 0 2024-06-03T01:21:01.055 INFO:tasks.workunit.client.1.smithi044.stdout:7/817: dwrite d0/d3/d30/fc2 [0,4194304] 0 2024-06-03T01:21:01.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/310: dwrite dc/f14 [0,4194304] 0 2024-06-03T01:21:01.058 INFO:tasks.workunit.client.0.smithi032.stdout:5/372: dwrite da/dd/df/d10/d16/d18/d1a/f6d [0,4194304] 0 2024-06-03T01:21:01.058 INFO:tasks.workunit.client.0.smithi032.stdout:7/311: fsync dc/f15 0 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:0/961: dwrite d3/f13c [0,4194304] 0 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/745: dwrite d6/f44 [0,4194304] 0 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/746: chown d6/f92 29 1 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/747: chown d6/dd/d51/d16/d5f/fa8 66891 1 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/748: truncate d6/f65 632497 0 2024-06-03T01:21:01.059 INFO:tasks.workunit.client.1.smithi044.stdout:1/978: dwrite d0/fba [4194304,4194304] 0 2024-06-03T01:21:01.060 INFO:tasks.workunit.client.1.smithi044.stdout:1/979: truncate d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/f26 2624635 0 2024-06-03T01:21:01.060 INFO:tasks.workunit.client.1.smithi044.stdout:1/980: write d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/f67 [123992,119249] 0 2024-06-03T01:21:01.060 INFO:tasks.workunit.client.1.smithi044.stdout:1/981: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/f5f 0 1 2024-06-03T01:21:01.060 INFO:tasks.workunit.client.1.smithi044.stdout:1/982: read - d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d81/f69 zero size 2024-06-03T01:21:01.064 INFO:tasks.workunit.client.1.smithi044.stdout:5/755: symlink d2/d11/d12/d2b/d4a/d57/lf8 0 2024-06-03T01:21:01.065 INFO:tasks.workunit.client.0.smithi032.stdout:5/373: rename da/dd/df/d10/d16/d18/d1a/d30/d58/l60 to da/dd/df/d10/d16/d18/d1a/d30/l6e 0 2024-06-03T01:21:01.070 INFO:tasks.workunit.client.0.smithi032.stdout:7/312: dread dc/d12/d1b/f21 [0,4194304] 0 2024-06-03T01:21:01.072 INFO:tasks.workunit.client.1.smithi044.stdout:0/962: truncate d3/f85 338846 0 2024-06-03T01:21:01.072 INFO:tasks.workunit.client.0.smithi032.stdout:7/313: creat dc/d12/d1b/f60 x:0 0 0 2024-06-03T01:21:01.072 INFO:tasks.workunit.client.0.smithi032.stdout:7/314: write dc/dd/f20 [1062034,118286] 0 2024-06-03T01:21:01.074 INFO:tasks.workunit.client.1.smithi044.stdout:7/818: dwrite d0/d3/d8/d95/f65 [0,4194304] 0 2024-06-03T01:21:01.074 INFO:tasks.workunit.client.1.smithi044.stdout:7/819: write d0/fd [1449808,38823] 0 2024-06-03T01:21:01.074 INFO:tasks.workunit.client.0.smithi032.stdout:5/374: dread da/dd/df/d10/d16/d18/d1a/d30/f5b [0,4194304] 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/375: stat da/dd/df/d10/d16/l2e 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/376: stat c6 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/315: mkdir dc/d12/d22/d42/d61 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/377: creat da/dd/df/d10/d16/d18/d35/d44/f6f x:0 0 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/378: readlink da/dd/df/d10/l2f 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/379: dread - da/dd/df/d10/f4c zero size 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/380: dread - da/dd/df/d10/d16/d18/d35/d44/f4f zero size 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.1.smithi044.stdout:9/749: mknod d6/dd/d45/cf0 0 2024-06-03T01:21:01.075 INFO:tasks.workunit.client.1.smithi044.stdout:9/750: readlink d6/dd/d51/l1d 0 2024-06-03T01:21:01.077 INFO:tasks.workunit.client.0.smithi032.stdout:6/329: dwrite d8/d11/d20/d2b/f4a [0,4194304] 0 2024-06-03T01:21:01.078 INFO:tasks.workunit.client.0.smithi032.stdout:5/381: mkdir da/dd/df/d10/d16/d18/d35/d70 0 2024-06-03T01:21:01.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/382: mkdir da/dd/df/d10/d71 0 2024-06-03T01:21:01.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/383: truncate da/dd/df/d10/d16/d18/d35/d44/f6f 632684 0 2024-06-03T01:21:01.083 INFO:tasks.workunit.client.1.smithi044.stdout:5/756: mkdir d2/d11/d1e/d21/d24/d9e/df9 0 2024-06-03T01:21:01.093 INFO:tasks.workunit.client.1.smithi044.stdout:7/820: rmdir d0/d3/d8/da0/dc7 0 2024-06-03T01:21:01.093 INFO:tasks.workunit.client.1.smithi044.stdout:9/751: dread d6/dd/d51/d16/d27/d5e/db7/de1/fab [0,4194304] 0 2024-06-03T01:21:01.094 INFO:tasks.workunit.client.1.smithi044.stdout:9/752: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/fe8 [78833,48397] 0 2024-06-03T01:21:01.094 INFO:tasks.workunit.client.1.smithi044.stdout:7/821: write d0/d16/fb8 [3811805,63261] 0 2024-06-03T01:21:01.098 INFO:tasks.workunit.client.1.smithi044.stdout:5/757: dread d2/d11/d3e/f59 [4194304,4194304] 0 2024-06-03T01:21:01.100 INFO:tasks.workunit.client.1.smithi044.stdout:9/753: creat d6/dd/d51/d16/d27/ff1 x:0 0 0 2024-06-03T01:21:01.100 INFO:tasks.workunit.client.1.smithi044.stdout:9/754: stat d6/dd/d51/d16/d27/ff1 0 2024-06-03T01:21:01.101 INFO:tasks.workunit.client.1.smithi044.stdout:7/822: mknod d0/d3/ddf/c109 0 2024-06-03T01:21:01.104 INFO:tasks.workunit.client.1.smithi044.stdout:9/755: creat d6/dd/d51/d16/d27/d5e/db7/de1/ff2 x:0 0 0 2024-06-03T01:21:01.104 INFO:tasks.workunit.client.1.smithi044.stdout:9/756: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68/fd4 [77491,114088] 0 2024-06-03T01:21:01.105 INFO:tasks.workunit.client.1.smithi044.stdout:5/758: dread d2/d11/d1e/d21/d24/d93/d9d/f8c [0,4194304] 0 2024-06-03T01:21:01.105 INFO:tasks.workunit.client.0.smithi032.stdout:5/384: write da/dd/df/d10/d16/d18/f50 [1816073,79612] 0 2024-06-03T01:21:01.105 INFO:tasks.workunit.client.0.smithi032.stdout:5/385: creat da/dd/df/d10/d16/d18/d35/d44/f72 x:0 0 0 2024-06-03T01:21:01.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/823: mkdir d0/d3/d8/d95/d63/d89/da3/dde/d10a 0 2024-06-03T01:21:01.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/824: read d0/d3/d8/d95/d63/d7f/d9c/fe3 [24799,130930] 0 2024-06-03T01:21:01.107 INFO:tasks.workunit.client.1.smithi044.stdout:9/757: mknod d6/dd/d51/d16/d27/d5e/db7/dd6/cf3 0 2024-06-03T01:21:01.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/386: creat da/f73 x:0 0 0 2024-06-03T01:21:01.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/387: write da/dd/df/d10/d16/d18/f1d [2731536,108595] 0 2024-06-03T01:21:01.110 INFO:tasks.workunit.client.1.smithi044.stdout:5/759: rename d2/d11/d3e/fd7 to d2/d11/d1e/d21/d24/ffa 0 2024-06-03T01:21:01.110 INFO:tasks.workunit.client.0.smithi032.stdout:5/388: mkdir da/dd/df/d10/d16/d25/d74 0 2024-06-03T01:21:01.111 INFO:tasks.workunit.client.0.smithi032.stdout:5/389: mknod da/dd/df/d10/d16/d18/d1a/d30/d58/c75 0 2024-06-03T01:21:01.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/758: getdents d6/dd/d51/d16/d27/d5e/db7 0 2024-06-03T01:21:01.112 INFO:tasks.workunit.client.1.smithi044.stdout:3/717: dwrite df/d1e/d2a/f36 [0,4194304] 0 2024-06-03T01:21:01.113 INFO:tasks.workunit.client.1.smithi044.stdout:5/760: getdents d2/d11/d1e/d21/d24/d93 0 2024-06-03T01:21:01.113 INFO:tasks.workunit.client.1.smithi044.stdout:5/761: fdatasync d2/d11/d1e/d21/d24/fa7 0 2024-06-03T01:21:01.113 INFO:tasks.workunit.client.1.smithi044.stdout:5/762: creat d2/d11/d3e/ffb x:0 0 0 2024-06-03T01:21:01.113 INFO:tasks.workunit.client.1.smithi044.stdout:5/763: write d2/d11/d1e/d21/d24/fa7 [400315,48176] 0 2024-06-03T01:21:01.115 INFO:tasks.workunit.client.0.smithi032.stdout:7/316: dwrite f6 [0,4194304] 0 2024-06-03T01:21:01.116 INFO:tasks.workunit.client.0.smithi032.stdout:4/290: sync 2024-06-03T01:21:01.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/340: sync 2024-06-03T01:21:01.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/341: creat d5/f72 x:0 0 0 2024-06-03T01:21:01.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/342: creat d5/d38/d52/f73 x:0 0 0 2024-06-03T01:21:01.117 INFO:tasks.workunit.client.1.smithi044.stdout:7/825: dread d0/d3/d8/d95/d63/d7f/d9c/f68 [4194304,4194304] 0 2024-06-03T01:21:01.117 INFO:tasks.workunit.client.1.smithi044.stdout:0/963: dwrite d3/f69 [4194304,4194304] 0 2024-06-03T01:21:01.117 INFO:tasks.workunit.client.1.smithi044.stdout:7/826: write d0/d3/ddf/ff4 [397783,66494] 0 2024-06-03T01:21:01.117 INFO:tasks.workunit.client.0.smithi032.stdout:6/330: dwrite d8/d11/d20/d2b/d39/d19/f40 [0,4194304] 0 2024-06-03T01:21:01.117 INFO:tasks.workunit.client.0.smithi032.stdout:5/390: dread da/f17 [0,4194304] 0 2024-06-03T01:21:01.118 INFO:tasks.workunit.client.1.smithi044.stdout:5/764: rename d2/d11/d12/d2b/f84 to d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/ffc 0 2024-06-03T01:21:01.118 INFO:tasks.workunit.client.1.smithi044.stdout:5/765: write d2/d11/d12/dad/fbf [4974834,90605] 0 2024-06-03T01:21:01.119 INFO:tasks.workunit.client.1.smithi044.stdout:0/964: write d3/fc6 [218359,2848] 0 2024-06-03T01:21:01.123 INFO:tasks.workunit.client.0.smithi032.stdout:2/343: unlink d5/da/d1a/d2a/d36/d3f/f56 0 2024-06-03T01:21:01.125 INFO:tasks.workunit.client.1.smithi044.stdout:3/718: dread df/d13/d39/d3d/f44 [0,4194304] 0 2024-06-03T01:21:01.126 INFO:tasks.workunit.client.0.smithi032.stdout:6/331: write d8/ff [140795,31984] 0 2024-06-03T01:21:01.126 INFO:tasks.workunit.client.0.smithi032.stdout:7/317: read dc/f3d [162948,118806] 0 2024-06-03T01:21:01.126 INFO:tasks.workunit.client.1.smithi044.stdout:5/766: link d2/d11/d3e/c8f d2/d11/d1e/d2f/df5/cfd 0 2024-06-03T01:21:01.127 INFO:tasks.workunit.client.1.smithi044.stdout:5/767: dread - d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/fc1 zero size 2024-06-03T01:21:01.131 INFO:tasks.workunit.client.1.smithi044.stdout:0/965: rename d3/d9/d36 to d3/d9/d102/d159 0 2024-06-03T01:21:01.132 INFO:tasks.workunit.client.0.smithi032.stdout:5/391: symlink da/dd/df/d10/d16/d18/l76 0 2024-06-03T01:21:01.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/768: mkdir d2/d11/d1e/d21/d24/d93/d9d/d28/dfe 0 2024-06-03T01:21:01.136 INFO:tasks.workunit.client.1.smithi044.stdout:0/966: mknod d3/d9/d55/d7b/dfc/d10c/c15a 0 2024-06-03T01:21:01.136 INFO:tasks.workunit.client.0.smithi032.stdout:4/291: symlink d4/db/da/d1d/l67 0 2024-06-03T01:21:01.138 INFO:tasks.workunit.client.0.smithi032.stdout:6/332: creat d8/d50/f62 x:0 0 0 2024-06-03T01:21:01.139 INFO:tasks.workunit.client.0.smithi032.stdout:6/333: stat d8/d11/d20/d2b/d39/c2d 0 2024-06-03T01:21:01.139 INFO:tasks.workunit.client.1.smithi044.stdout:7/827: dwrite d0/f7c [0,4194304] 0 2024-06-03T01:21:01.139 INFO:tasks.workunit.client.1.smithi044.stdout:0/967: rename d3/da6/d56/d8e to d3/d6b/d154/d15b 0 2024-06-03T01:21:01.139 INFO:tasks.workunit.client.1.smithi044.stdout:9/759: dwrite d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [0,4194304] 0 2024-06-03T01:21:01.139 INFO:tasks.workunit.client.1.smithi044.stdout:3/719: write df/d13/f24 [7712754,115993] 0 2024-06-03T01:21:01.140 INFO:tasks.workunit.client.0.smithi032.stdout:5/392: creat da/dd/df/d10/d16/d18/d1a/f77 x:0 0 0 2024-06-03T01:21:01.143 INFO:tasks.workunit.client.0.smithi032.stdout:6/334: symlink d8/d11/d20/d2b/d39/l63 0 2024-06-03T01:21:01.148 INFO:tasks.workunit.client.0.smithi032.stdout:5/393: symlink da/dd/df/d10/d16/d18/d1a/d30/d58/l78 0 2024-06-03T01:21:01.150 INFO:tasks.workunit.client.0.smithi032.stdout:5/394: rmdir da/dd/df/d10/d71 0 2024-06-03T01:21:01.150 INFO:tasks.workunit.client.0.smithi032.stdout:5/395: stat da/dd/f48 0 2024-06-03T01:21:01.150 INFO:tasks.workunit.client.0.smithi032.stdout:5/396: write da/dd/df/d10/d16/d18/d1a/d30/f5f [316325,45725] 0 2024-06-03T01:21:01.151 INFO:tasks.workunit.client.1.smithi044.stdout:7/828: mknod d0/d3/d8/d95/d63/de6/c10b 0 2024-06-03T01:21:01.151 INFO:tasks.workunit.client.1.smithi044.stdout:7/829: write d0/d3/d8/d32/d33/dd1/f100 [510212,66812] 0 2024-06-03T01:21:01.152 INFO:tasks.workunit.client.0.smithi032.stdout:4/292: write d4/db/da/d2d/d27/f2b [1852203,76116] 0 2024-06-03T01:21:01.154 INFO:tasks.workunit.client.0.smithi032.stdout:5/397: link da/dd/df/d10/d16/d18/l6c da/dd/df/d10/d16/d18/l79 0 2024-06-03T01:21:01.154 INFO:tasks.workunit.client.0.smithi032.stdout:5/398: truncate da/dd/df/d10/d16/d25/f56 67153 0 2024-06-03T01:21:01.156 INFO:tasks.workunit.client.0.smithi032.stdout:5/399: dread da/dd/df/d10/f4d [0,4194304] 0 2024-06-03T01:21:01.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/830: write d0/d3/d8/d95/d63/d7f/d9c/f68 [4199483,51926] 0 2024-06-03T01:21:01.158 INFO:tasks.workunit.client.0.smithi032.stdout:5/400: mkdir da/dd/df/d10/d16/d18/d1a/d30/d51/d7a 0 2024-06-03T01:21:01.158 INFO:tasks.workunit.client.0.smithi032.stdout:5/401: write da/dd/df/d10/d16/d18/d35/d44/f6f [197062,75885] 0 2024-06-03T01:21:01.159 INFO:tasks.workunit.client.0.smithi032.stdout:5/402: creat da/dd/df/d10/d16/d18/d1a/d30/d51/f7b x:0 0 0 2024-06-03T01:21:01.159 INFO:tasks.workunit.client.0.smithi032.stdout:5/403: chown da/dd/df/d10/d16/d18/d1a/d30/d51/f7b 0 1 2024-06-03T01:21:01.159 INFO:tasks.workunit.client.1.smithi044.stdout:7/831: write d0/d3/d8/d95/d63/d7f/d9c/d6e/fb4 [1293502,18133] 0 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/404: mknod da/dd/df/c7c 0 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/405: truncate da/dd/f68 737629 0 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/406: chown da/dd/df/d10/d16/d18/f1d 12 1 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/407: readlink da/dd/df/d10/d16/d18/d1a/d30/d58/l62 0 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/408: write da/dd/df/d10/f4c [186045,38857] 0 2024-06-03T01:21:01.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/409: chown da/dd/df/d10/f4c 204817416 1 2024-06-03T01:21:01.163 INFO:tasks.workunit.client.0.smithi032.stdout:5/410: mkdir da/dd/df/d10/d16/d18/d1a/d30/d51/d7a/d7d 0 2024-06-03T01:21:01.164 INFO:tasks.workunit.client.0.smithi032.stdout:5/411: symlink da/dd/l7e 0 2024-06-03T01:21:01.165 INFO:tasks.workunit.client.0.smithi032.stdout:4/293: dread d4/f3a [0,4194304] 0 2024-06-03T01:21:01.165 INFO:tasks.workunit.client.0.smithi032.stdout:4/294: stat d4/db/da/d2d/d27/f4f 0 2024-06-03T01:21:01.166 INFO:tasks.workunit.client.1.smithi044.stdout:0/968: symlink d3/da6/d20/d25/l15c 0 2024-06-03T01:21:01.166 INFO:tasks.workunit.client.1.smithi044.stdout:0/969: fsync d3/f91 0 2024-06-03T01:21:01.166 INFO:tasks.workunit.client.1.smithi044.stdout:0/970: write d3/da6/dc1/fe2 [358978,77153] 0 2024-06-03T01:21:01.167 INFO:tasks.workunit.client.0.smithi032.stdout:4/295: unlink d4/db/da/d37/c5b 0 2024-06-03T01:21:01.167 INFO:tasks.workunit.client.0.smithi032.stdout:4/296: fdatasync d4/db/da/d1d/f21 0 2024-06-03T01:21:01.169 INFO:tasks.workunit.client.0.smithi032.stdout:4/297: creat d4/db/f68 x:0 0 0 2024-06-03T01:21:01.169 INFO:tasks.workunit.client.0.smithi032.stdout:4/298: write d4/db/ff [1058281,57798] 0 2024-06-03T01:21:01.170 INFO:tasks.workunit.client.0.smithi032.stdout:4/299: symlink d4/l69 0 2024-06-03T01:21:01.171 INFO:tasks.workunit.client.0.smithi032.stdout:7/318: dwrite dc/d12/d24/d33/f4c [0,4194304] 0 2024-06-03T01:21:01.171 INFO:tasks.workunit.client.0.smithi032.stdout:7/319: fdatasync fa 0 2024-06-03T01:21:01.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/720: creat df/d13/d6d/fed x:0 0 0 2024-06-03T01:21:01.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/721: readlink df/d13/d39/d3d/d43/d71/lb8 0 2024-06-03T01:21:01.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/722: getdents df/d13/d1b/d4b/d69/dd1 0 2024-06-03T01:21:01.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/723: fdatasync df/f65 0 2024-06-03T01:21:01.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/724: truncate df/d1e/d2a/d70/fb2 751357 0 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/300: mknod d4/c6a 0 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/301: dread - d4/db/da/d1d/f1f zero size 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/302: write d4/d5c/f5e [880345,81292] 0 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/303: fsync d4/f7 0 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:4/304: write d4/db/f68 [732554,82787] 0 2024-06-03T01:21:01.174 INFO:tasks.workunit.client.0.smithi032.stdout:5/412: dread da/dd/df/d10/f42 [0,4194304] 0 2024-06-03T01:21:01.175 INFO:tasks.workunit.client.0.smithi032.stdout:2/344: dwrite d5/d38/d46/d53/f5e [0,4194304] 0 2024-06-03T01:21:01.175 INFO:tasks.workunit.client.0.smithi032.stdout:2/345: write d5/da/d1a/f68 [107163,4325] 0 2024-06-03T01:21:01.175 INFO:tasks.workunit.client.1.smithi044.stdout:9/760: dwrite f4 [4194304,4194304] 0 2024-06-03T01:21:01.180 INFO:tasks.workunit.client.0.smithi032.stdout:7/320: getdents dc/d12/d22/d42/d4e 0 2024-06-03T01:21:01.183 INFO:tasks.workunit.client.0.smithi032.stdout:4/305: rename d4/db/da/d1d/d58 to d4/db/da/d1d/d6b 0 2024-06-03T01:21:01.183 INFO:tasks.workunit.client.0.smithi032.stdout:4/306: chown d4/db/da/f3f 13694 1 2024-06-03T01:21:01.183 INFO:tasks.workunit.client.0.smithi032.stdout:4/307: write d4/db/da/f31 [1111286,121538] 0 2024-06-03T01:21:01.183 INFO:tasks.workunit.client.1.smithi044.stdout:3/725: read df/d13/f3f [2259445,11825] 0 2024-06-03T01:21:01.183 INFO:tasks.workunit.client.1.smithi044.stdout:3/726: write df/d13/d39/d3d/d6a/d9c/fbd [574873,85877] 0 2024-06-03T01:21:01.184 INFO:tasks.workunit.client.1.smithi044.stdout:3/727: rename df/d13/d39/d3d/d43/d72/cd3 to df/d1e/d78/cee 0 2024-06-03T01:21:01.185 INFO:tasks.workunit.client.0.smithi032.stdout:7/321: rename l2 to dc/d12/l62 0 2024-06-03T01:21:01.185 INFO:tasks.workunit.client.0.smithi032.stdout:4/308: mkdir d4/d57/d6c 0 2024-06-03T01:21:01.186 INFO:tasks.workunit.client.0.smithi032.stdout:7/322: mkdir dc/d12/d2a/d2f/d63 0 2024-06-03T01:21:01.187 INFO:tasks.workunit.client.0.smithi032.stdout:4/309: symlink d4/l6d 0 2024-06-03T01:21:01.187 INFO:tasks.workunit.client.0.smithi032.stdout:4/310: write d4/db/da/f54 [5236725,124657] 0 2024-06-03T01:21:01.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/323: rmdir dc 39 2024-06-03T01:21:01.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/324: chown dc/d12/d2a/d2f/l59 188827237 1 2024-06-03T01:21:01.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/325: chown dc/d12/d22/c3e 456 1 2024-06-03T01:21:01.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/326: dread - dc/d12/d1b/f60 zero size 2024-06-03T01:21:01.189 INFO:tasks.workunit.client.1.smithi044.stdout:9/761: dread d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [0,4194304] 0 2024-06-03T01:21:01.190 INFO:tasks.workunit.client.1.smithi044.stdout:9/762: chown d6/dd/d51/d16/cc6 167 1 2024-06-03T01:21:01.190 INFO:tasks.workunit.client.1.smithi044.stdout:9/763: fsync d6/dd/d51/d16/d61/f7e 0 2024-06-03T01:21:01.190 INFO:tasks.workunit.client.1.smithi044.stdout:9/764: chown d6/dd/d51/d16/d27 2135337 1 2024-06-03T01:21:01.190 INFO:tasks.workunit.client.0.smithi032.stdout:6/335: dwrite d8/d11/f3e [0,4194304] 0 2024-06-03T01:21:01.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/832: write d0/d3/d8/d95/d63/f7a [849944,114351] 0 2024-06-03T01:21:01.192 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:00 smithi032 ceph-mon[29360]: pgmap v5: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:21:01.192 INFO:tasks.workunit.client.1.smithi044.stdout:9/765: creat d6/d38/ff4 x:0 0 0 2024-06-03T01:21:01.197 INFO:tasks.workunit.client.1.smithi044.stdout:5/769: dwrite d2/d11/fd8 [0,4194304] 0 2024-06-03T01:21:01.198 INFO:tasks.workunit.client.1.smithi044.stdout:7/833: dread d0/f4 [0,4194304] 0 2024-06-03T01:21:01.198 INFO:tasks.workunit.client.1.smithi044.stdout:5/770: creat d2/d11/d1e/d3b/da6/fff x:0 0 0 2024-06-03T01:21:01.198 INFO:tasks.workunit.client.0.smithi032.stdout:7/327: getdents dc/d12/d2a 0 2024-06-03T01:21:01.200 INFO:tasks.workunit.client.1.smithi044.stdout:0/971: dwrite d3/da6/dc1/dc2/df8/f110 [0,4194304] 0 2024-06-03T01:21:01.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/336: creat d8/d11/d20/d36/f64 x:0 0 0 2024-06-03T01:21:01.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/337: rename d8 to d8/d11/d20/d36/d65 22 2024-06-03T01:21:01.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/338: chown d8/d11/d20/d2b/c2e 3 1 2024-06-03T01:21:01.202 INFO:tasks.workunit.client.0.smithi032.stdout:7/328: rmdir dc/d12/d22 39 2024-06-03T01:21:01.203 INFO:tasks.workunit.client.1.smithi044.stdout:9/766: symlink d6/dd/d51/lf5 0 2024-06-03T01:21:01.208 INFO:tasks.workunit.client.0.smithi032.stdout:2/346: dwrite d5/da/f1e [4194304,4194304] 0 2024-06-03T01:21:01.208 INFO:tasks.workunit.client.1.smithi044.stdout:7/834: mknod d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/c10c 0 2024-06-03T01:21:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:5/771: rename d2/d11/d12/de0/fe9 to d2/d11/f100 0 2024-06-03T01:21:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:5/772: write d2/d11/d1e/d21/d24/ffa [607730,125603] 0 2024-06-03T01:21:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:4/863: sync 2024-06-03T01:21:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:2/701: sync 2024-06-03T01:21:01.216 INFO:tasks.workunit.client.1.smithi044.stdout:9/767: unlink d6/dd/d51/d16/ce5 0 2024-06-03T01:21:01.219 INFO:tasks.workunit.client.1.smithi044.stdout:7/835: link d0/d3/fcf d0/d3/d8/d95/d63/d89/da3/dde/d10a/f10d 0 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.0.smithi032.stdout:2/347: truncate d5/da/f39 2373844 0 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.1.smithi044.stdout:6/950: sync 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.1.smithi044.stdout:6/951: stat d2/d23/d30/f45 0 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.1.smithi044.stdout:5/773: write d2/f9f [114591,47939] 0 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.1.smithi044.stdout:5/774: chown d2/d11/d1e/d3b/da6 1076 1 2024-06-03T01:21:01.220 INFO:tasks.workunit.client.1.smithi044.stdout:5/775: write d2/d11/d3e/dce/fec [722204,97192] 0 2024-06-03T01:21:01.221 INFO:tasks.workunit.client.1.smithi044.stdout:4/864: truncate d17/d1c/f3a 9026331 0 2024-06-03T01:21:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:00 smithi044 ceph-mon[36956]: pgmap v5: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:21:01.225 INFO:tasks.workunit.client.1.smithi044.stdout:2/702: rename db/d1b/ccc to db/d15/ce1 0 2024-06-03T01:21:01.225 INFO:tasks.workunit.client.1.smithi044.stdout:2/703: read - db/d15/d49/d6c/d72/d8f/fda zero size 2024-06-03T01:21:01.226 INFO:tasks.workunit.client.0.smithi032.stdout:5/413: rmdir da/dd/df/d10/d16/d18/d1a/d30 39 2024-06-03T01:21:01.226 INFO:tasks.workunit.client.0.smithi032.stdout:5/414: readlink da/dd/df/d10/d16/d18/d1a/d30/l6e 0 2024-06-03T01:21:01.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/851: sync 2024-06-03T01:21:01.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/852: truncate d5/d55/d56/d5c/d68/de5/dea/de2/f108 307015 0 2024-06-03T01:21:01.230 INFO:tasks.workunit.client.1.smithi044.stdout:7/836: truncate d0/d3/d8/d95/fa6 3822491 0 2024-06-03T01:21:01.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/776: mknod d2/d11/d1e/d21/d24/d9e/c101 0 2024-06-03T01:21:01.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/777: stat d2/d11/d1e/d3b 0 2024-06-03T01:21:01.231 INFO:tasks.workunit.client.1.smithi044.stdout:9/768: rename d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/l54 to d6/dd/d51/d16/d61/dd9/lf6 0 2024-06-03T01:21:01.233 INFO:tasks.workunit.client.1.smithi044.stdout:8/853: mknod d5/d16/d81/de3/d58/d89/dbf/dd2/c11d 0 2024-06-03T01:21:01.233 INFO:tasks.workunit.client.1.smithi044.stdout:5/778: mknod d2/d11/d3e/c102 0 2024-06-03T01:21:01.234 INFO:tasks.workunit.client.1.smithi044.stdout:8/854: mknod d5/d55/d74/d82/db0/c11e 0 2024-06-03T01:21:01.235 INFO:tasks.workunit.client.1.smithi044.stdout:8/855: fsync d5/d16/d81/de3/d58/d89/dbf/f80 0 2024-06-03T01:21:01.235 INFO:tasks.workunit.client.1.smithi044.stdout:8/856: fdatasync d5/d20/f9e 0 2024-06-03T01:21:01.235 INFO:tasks.workunit.client.1.smithi044.stdout:1/983: sync 2024-06-03T01:21:01.237 INFO:tasks.workunit.client.1.smithi044.stdout:5/779: symlink d2/d11/d1e/d2f/l103 0 2024-06-03T01:21:01.237 INFO:tasks.workunit.client.1.smithi044.stdout:9/769: dread d6/dd/d51/d16/d27/d5e/db7/fc0 [0,4194304] 0 2024-06-03T01:21:01.241 INFO:tasks.workunit.client.0.smithi032.stdout:2/348: creat d5/da/f74 x:0 0 0 2024-06-03T01:21:01.241 INFO:tasks.workunit.client.0.smithi032.stdout:2/349: dread - d5/d38/d52/f73 zero size 2024-06-03T01:21:01.242 INFO:tasks.workunit.client.0.smithi032.stdout:2/350: chown d5/da/d1a/d70 21306 1 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/857: rename d5/d16/l41 to d5/d55/d56/d5c/d68/de5/l11f 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/858: getdents d5/d16/d1d/d32/d109 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:5/780: creat d2/d11/d12/f104 x:0 0 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/859: creat d5/d20/df5/f120 x:0 0 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:9/770: creat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dcd/ff7 x:0 0 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/860: chown d5/d20/d33 310981 1 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/861: write d5/d16/d81/de3/d58/f107 [795576,98924] 0 2024-06-03T01:21:01.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/862: read - d5/d16/d1d/d32/f119 zero size 2024-06-03T01:21:01.245 INFO:tasks.workunit.client.0.smithi032.stdout:5/415: creat da/dd/df/d10/d16/d18/d1a/d30/d58/f7f x:0 0 0 2024-06-03T01:21:01.252 INFO:tasks.workunit.client.1.smithi044.stdout:5/781: unlink d2/d11/d12/d2b/d44/c41 0 2024-06-03T01:21:01.253 INFO:tasks.workunit.client.1.smithi044.stdout:1/984: dread d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/fd3 [0,4194304] 0 2024-06-03T01:21:01.253 INFO:tasks.workunit.client.1.smithi044.stdout:0/972: dwrite d3/d5c/f8f [0,4194304] 0 2024-06-03T01:21:01.253 INFO:tasks.workunit.client.1.smithi044.stdout:9/771: rmdir d6/dd/d51/d16/d27/d5e/db7/dd6 39 2024-06-03T01:21:01.253 INFO:tasks.workunit.client.1.smithi044.stdout:9/772: chown d6/d38/l85 177 1 2024-06-03T01:21:01.253 INFO:tasks.workunit.client.0.smithi032.stdout:5/416: mknod da/dd/df/d10/d16/d25/d74/c80 0 2024-06-03T01:21:01.254 INFO:tasks.workunit.client.1.smithi044.stdout:5/782: unlink d2/d11/d12/d2b/d4a/fa5 0 2024-06-03T01:21:01.255 INFO:tasks.workunit.client.1.smithi044.stdout:5/783: dread - d2/d11/d12/f104 zero size 2024-06-03T01:21:01.255 INFO:tasks.workunit.client.0.smithi032.stdout:2/351: getdents d5/da/d1a 0 2024-06-03T01:21:01.255 INFO:tasks.workunit.client.0.smithi032.stdout:7/329: dwrite f6 [0,4194304] 0 2024-06-03T01:21:01.256 INFO:tasks.workunit.client.0.smithi032.stdout:7/330: write dc/d12/d22/f2e [685881,38271] 0 2024-06-03T01:21:01.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/311: dwrite d4/db/da/d2d/d27/f4f [0,4194304] 0 2024-06-03T01:21:01.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/312: read - d4/db/da/d37/f3e zero size 2024-06-03T01:21:01.258 INFO:tasks.workunit.client.1.smithi044.stdout:1/985: dread d0/d3/d20/d7/d13/d2d/d101/f132 [0,4194304] 0 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.0.smithi032.stdout:6/339: dwrite d8/d11/d20/d21/f24 [0,4194304] 0 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.0.smithi032.stdout:6/340: chown d8/d11/c1e 17731 1 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.0.smithi032.stdout:6/341: dread - d8/d11/d20/d36/f64 zero size 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.1.smithi044.stdout:0/973: getdents d3/d6b/d95 0 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.1.smithi044.stdout:9/773: symlink d6/dd/d51/d16/d5f/lf8 0 2024-06-03T01:21:01.261 INFO:tasks.workunit.client.1.smithi044.stdout:5/784: symlink d2/d11/d3e/l105 0 2024-06-03T01:21:01.263 INFO:tasks.workunit.client.0.smithi032.stdout:5/417: rename da/dd/df/d10/d16/d25/f49 to da/dd/df/d10/d16/d18/d35/f81 0 2024-06-03T01:21:01.263 INFO:tasks.workunit.client.0.smithi032.stdout:5/418: write da/dd/df/f5c [1016776,81985] 0 2024-06-03T01:21:01.263 INFO:tasks.workunit.client.1.smithi044.stdout:7/837: dwrite d0/d16/f8e [4194304,4194304] 0 2024-06-03T01:21:01.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/352: rmdir d5/da/d1a/d70 0 2024-06-03T01:21:01.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/353: getdents d5/d38/d46/d5c/d6c 0 2024-06-03T01:21:01.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/354: creat d5/d38/d52/f75 x:0 0 0 2024-06-03T01:21:01.267 INFO:tasks.workunit.client.1.smithi044.stdout:6/952: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/f9d [0,4194304] 0 2024-06-03T01:21:01.267 INFO:tasks.workunit.client.1.smithi044.stdout:3/728: dwrite df/d13/d39/d3d/d50/f6f [0,4194304] 0 2024-06-03T01:21:01.272 INFO:tasks.workunit.client.0.smithi032.stdout:5/419: dread da/dd/df/f32 [0,4194304] 0 2024-06-03T01:21:01.273 INFO:tasks.workunit.client.1.smithi044.stdout:0/974: mkdir d3/da6/d21/d94/da3/d15d 0 2024-06-03T01:21:01.277 INFO:tasks.workunit.client.1.smithi044.stdout:8/863: dwrite d5/d20/d33/f4b [0,4194304] 0 2024-06-03T01:21:01.279 INFO:tasks.workunit.client.0.smithi032.stdout:6/342: mkdir d8/d11/d20/d3b/d66 0 2024-06-03T01:21:01.279 INFO:tasks.workunit.client.0.smithi032.stdout:6/343: chown d8/d11/d20/d2b/d39/d19/f40 4867059 1 2024-06-03T01:21:01.281 INFO:tasks.workunit.client.1.smithi044.stdout:5/785: mkdir d2/d11/d12/d2b/d4a/d57/d106 0 2024-06-03T01:21:01.281 INFO:tasks.workunit.client.1.smithi044.stdout:7/838: write d0/d3/d8/d95/d63/d7f/faf [4659362,19515] 0 2024-06-03T01:21:01.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/313: dwrite d4/f7 [0,4194304] 0 2024-06-03T01:21:01.282 INFO:tasks.workunit.client.1.smithi044.stdout:2/704: dwrite db/d15/d49/d7b/f88 [0,4194304] 0 2024-06-03T01:21:01.283 INFO:tasks.workunit.client.0.smithi032.stdout:7/331: rename dc/d12/d24/d33/l4a to dc/d12/l64 0 2024-06-03T01:21:01.283 INFO:tasks.workunit.client.0.smithi032.stdout:7/332: write dc/d12/d22/f29 [866344,33119] 0 2024-06-03T01:21:01.283 INFO:tasks.workunit.client.1.smithi044.stdout:9/774: mknod d6/dd/d51/d16/dee/cf9 0 2024-06-03T01:21:01.287 INFO:tasks.workunit.client.0.smithi032.stdout:2/355: symlink d5/d38/d46/d53/d57/l76 0 2024-06-03T01:21:01.287 INFO:tasks.workunit.client.0.smithi032.stdout:2/356: write d5/d38/d52/f75 [247609,93983] 0 2024-06-03T01:21:01.289 INFO:tasks.workunit.client.0.smithi032.stdout:5/420: symlink da/dd/df/d10/d16/d18/d35/l82 0 2024-06-03T01:21:01.293 INFO:tasks.workunit.client.0.smithi032.stdout:6/344: creat d8/d11/d20/d2b/d39/f67 x:0 0 0 2024-06-03T01:21:01.293 INFO:tasks.workunit.client.0.smithi032.stdout:6/345: fdatasync d8/d11/d20/d21/f27 0 2024-06-03T01:21:01.295 INFO:tasks.workunit.client.0.smithi032.stdout:5/421: dread da/dd/df/d10/f39 [0,4194304] 0 2024-06-03T01:21:01.295 INFO:tasks.workunit.client.0.smithi032.stdout:5/422: fsync f4 0 2024-06-03T01:21:01.297 INFO:tasks.workunit.client.1.smithi044.stdout:2/705: dread db/d15/d49/d6c/d72/d7a/f8d [0,4194304] 0 2024-06-03T01:21:01.298 INFO:tasks.workunit.client.1.smithi044.stdout:3/729: link df/d13/d1b/d56/d5c/d98/la1 df/d1e/d3a/d84/lef 0 2024-06-03T01:21:01.299 INFO:tasks.workunit.client.1.smithi044.stdout:6/953: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/l147 0 2024-06-03T01:21:01.299 INFO:tasks.workunit.client.1.smithi044.stdout:6/954: creat d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/f148 x:0 0 0 2024-06-03T01:21:01.302 INFO:tasks.workunit.client.0.smithi032.stdout:4/314: creat d4/d57/d6c/f6e x:0 0 0 2024-06-03T01:21:01.304 INFO:tasks.workunit.client.0.smithi032.stdout:3/319: sync 2024-06-03T01:21:01.304 INFO:tasks.workunit.client.0.smithi032.stdout:8/277: sync 2024-06-03T01:21:01.304 INFO:tasks.workunit.client.0.smithi032.stdout:8/278: truncate dc/d25/d28/d3c/f3f 550235 0 2024-06-03T01:21:01.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/333: rmdir dc/d12/d55 0 2024-06-03T01:21:01.309 INFO:tasks.workunit.client.0.smithi032.stdout:2/357: symlink d5/d38/d46/l77 0 2024-06-03T01:21:01.311 INFO:tasks.workunit.client.0.smithi032.stdout:4/315: creat d4/db/da/f6f x:0 0 0 2024-06-03T01:21:01.312 INFO:tasks.workunit.client.0.smithi032.stdout:3/320: symlink de/d29/d2e/d31/d3c/d50/d62/l64 0 2024-06-03T01:21:01.313 INFO:tasks.workunit.client.1.smithi044.stdout:1/986: rmdir d0/d3/d20/d7/d13/d112 39 2024-06-03T01:21:01.314 INFO:tasks.workunit.client.1.smithi044.stdout:0/975: mkdir d3/da6/d21/d94/da3/d15d/d15e 0 2024-06-03T01:21:01.315 INFO:tasks.workunit.client.0.smithi032.stdout:7/334: mknod dc/d12/c65 0 2024-06-03T01:21:01.315 INFO:tasks.workunit.client.0.smithi032.stdout:7/335: creat dc/d12/d24/d33/f66 x:0 0 0 2024-06-03T01:21:01.316 INFO:tasks.workunit.client.0.smithi032.stdout:8/279: dread dc/de/f11 [0,4194304] 0 2024-06-03T01:21:01.316 INFO:tasks.workunit.client.0.smithi032.stdout:0/318: sync 2024-06-03T01:21:01.317 INFO:tasks.workunit.client.0.smithi032.stdout:1/201: sync 2024-06-03T01:21:01.318 INFO:tasks.workunit.client.0.smithi032.stdout:9/423: sync 2024-06-03T01:21:01.319 INFO:tasks.workunit.client.1.smithi044.stdout:8/864: rename d5/d20/d33/dfd/d115 to d5/d16/d81/d121 0 2024-06-03T01:21:01.319 INFO:tasks.workunit.client.1.smithi044.stdout:8/865: write d5/d55/d56/d5c/d68/de5/dea/d79/fd3 [870587,111787] 0 2024-06-03T01:21:01.319 INFO:tasks.workunit.client.1.smithi044.stdout:8/866: write d5/d55/f86 [2131708,120900] 0 2024-06-03T01:21:01.319 INFO:tasks.workunit.client.1.smithi044.stdout:8/867: readlink d5/d55/db5/lc5 0 2024-06-03T01:21:01.320 INFO:tasks.workunit.client.1.smithi044.stdout:8/868: dread d5/d16/d81/de3/d58/d89/fab [0,4194304] 0 2024-06-03T01:21:01.321 INFO:tasks.workunit.client.0.smithi032.stdout:3/321: truncate de/f57 303901 0 2024-06-03T01:21:01.321 INFO:tasks.workunit.client.0.smithi032.stdout:3/322: creat de/d29/f65 x:0 0 0 2024-06-03T01:21:01.321 INFO:tasks.workunit.client.0.smithi032.stdout:7/336: mknod dc/c67 0 2024-06-03T01:21:01.321 INFO:tasks.workunit.client.0.smithi032.stdout:7/337: write dc/d12/d22/d42/f51 [417324,67733] 0 2024-06-03T01:21:01.321 INFO:tasks.workunit.client.0.smithi032.stdout:6/346: dwrite d8/d11/d20/d3b/f41 [0,4194304] 0 2024-06-03T01:21:01.322 INFO:tasks.workunit.client.1.smithi044.stdout:7/839: unlink d0/d3/d8/d95/d63/d7f/fd3 0 2024-06-03T01:21:01.323 INFO:tasks.workunit.client.1.smithi044.stdout:9/775: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/lfa 0 2024-06-03T01:21:01.326 INFO:tasks.workunit.client.0.smithi032.stdout:4/316: rmdir d4/d5c 39 2024-06-03T01:21:01.326 INFO:tasks.workunit.client.0.smithi032.stdout:4/317: fdatasync d4/db/da/f49 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/706: creat db/d1b/d2a/d2f/dbe/fe2 x:0 0 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/707: stat db/d1b/d3f/lb8 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/708: stat db/d15/d49/d6c/f8c 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/709: getdents db/d1b/d5d/da7/dc9 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/710: truncate db/d15/d49/d6c/d72/dcd/fd5 1011036 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.0.smithi032.stdout:2/358: read d5/da/f1e [5304763,119574] 0 2024-06-03T01:21:01.327 INFO:tasks.workunit.client.0.smithi032.stdout:2/359: fdatasync d5/da/d2e/f3d 0 2024-06-03T01:21:01.328 INFO:tasks.workunit.client.0.smithi032.stdout:5/423: dwrite f4 [0,4194304] 0 2024-06-03T01:21:01.329 INFO:tasks.workunit.client.0.smithi032.stdout:3/323: mkdir de/d29/d2e/d31/d3c/d50/d66 0 2024-06-03T01:21:01.329 INFO:tasks.workunit.client.0.smithi032.stdout:3/324: dread - de/d29/d2e/d31/d23/f59 zero size 2024-06-03T01:21:01.331 INFO:tasks.workunit.client.0.smithi032.stdout:7/338: creat dc/d12/d1b/f68 x:0 0 0 2024-06-03T01:21:01.333 INFO:tasks.workunit.client.0.smithi032.stdout:8/280: mknod dc/d14/c5f 0 2024-06-03T01:21:01.333 INFO:tasks.workunit.client.0.smithi032.stdout:8/281: fdatasync dc/f15 0 2024-06-03T01:21:01.337 INFO:tasks.workunit.client.1.smithi044.stdout:1/987: symlink d0/d3/d20/d7/d2f/l14c 0 2024-06-03T01:21:01.337 INFO:tasks.workunit.client.0.smithi032.stdout:9/424: rmdir d0/d11 39 2024-06-03T01:21:01.337 INFO:tasks.workunit.client.0.smithi032.stdout:9/425: write d0/d11/d1d/f8f [701940,51663] 0 2024-06-03T01:21:01.337 INFO:tasks.workunit.client.0.smithi032.stdout:9/426: chown d0/d23/d25/f8e 0 1 2024-06-03T01:21:01.341 INFO:tasks.workunit.client.1.smithi044.stdout:0/976: rmdir d3/d6b/d154 39 2024-06-03T01:21:01.343 INFO:tasks.workunit.client.0.smithi032.stdout:2/360: creat d5/da/d1a/d2a/d36/d3f/f78 x:0 0 0 2024-06-03T01:21:01.345 INFO:tasks.workunit.client.0.smithi032.stdout:7/339: read dc/d12/d1b/f1d [2142512,25719] 0 2024-06-03T01:21:01.345 INFO:tasks.workunit.client.0.smithi032.stdout:7/340: write dc/d12/d1b/f5b [4495258,32433] 0 2024-06-03T01:21:01.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/347: mknod d8/d11/d20/c68 0 2024-06-03T01:21:01.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/348: write d8/d11/f17 [1551131,6414] 0 2024-06-03T01:21:01.349 INFO:tasks.workunit.client.0.smithi032.stdout:9/427: symlink d0/d11/d76/l93 0 2024-06-03T01:21:01.349 INFO:tasks.workunit.client.0.smithi032.stdout:9/428: write d0/d23/f85 [158106,102823] 0 2024-06-03T01:21:01.351 INFO:tasks.workunit.client.1.smithi044.stdout:8/869: mknod d5/d55/d56/d5c/c122 0 2024-06-03T01:21:01.351 INFO:tasks.workunit.client.1.smithi044.stdout:8/870: chown d5/d55/d74/d82/l61 3 1 2024-06-03T01:21:01.353 INFO:tasks.workunit.client.0.smithi032.stdout:4/318: read d4/db/f1c [3609786,89901] 0 2024-06-03T01:21:01.354 INFO:tasks.workunit.client.1.smithi044.stdout:7/840: symlink d0/d3/da8/de5/d106/l10e 0 2024-06-03T01:21:01.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/776: mknod d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/dae/cfb 0 2024-06-03T01:21:01.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/777: dread - d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fb6 zero size 2024-06-03T01:21:01.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/778: truncate d6/dd/d51/d46/f5c 1185420 0 2024-06-03T01:21:01.358 INFO:tasks.workunit.client.0.smithi032.stdout:7/341: rmdir dc/d12/d22/d42/d4e 39 2024-06-03T01:21:01.360 INFO:tasks.workunit.client.1.smithi044.stdout:2/711: truncate db/d15/f70 1404734 0 2024-06-03T01:21:01.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/424: write da/dd/df/d10/d16/d18/d1a/d30/f5b [522983,68327] 0 2024-06-03T01:21:01.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/425: truncate da/dd/df/d10/d16/d18/d1a/d30/d51/f57 821288 0 2024-06-03T01:21:01.363 INFO:tasks.workunit.client.0.smithi032.stdout:1/202: dwrite d18/f22 [0,4194304] 0 2024-06-03T01:21:01.364 INFO:tasks.workunit.client.0.smithi032.stdout:9/429: mknod d0/d23/d24/d26/c94 0 2024-06-03T01:21:01.364 INFO:tasks.workunit.client.0.smithi032.stdout:6/349: unlink d8/d11/c16 0 2024-06-03T01:21:01.364 INFO:tasks.workunit.client.0.smithi032.stdout:9/430: fdatasync d0/d11/d1d/f7e 0 2024-06-03T01:21:01.365 INFO:tasks.workunit.client.0.smithi032.stdout:0/319: dwrite d1/f1c [0,4194304] 0 2024-06-03T01:21:01.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/988: creat d0/dd2/d140/f14d x:0 0 0 2024-06-03T01:21:01.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/989: dread - d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d81/f139 zero size 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:2/361: dwrite d5/d38/d52/f54 [0,4194304] 0 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:3/325: dwrite de/f10 [0,4194304] 0 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:3/326: getdents de/d29/d2e/d31 0 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:3/327: readlink de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:3/328: chown de/d29/d2e/d31/f1b 253 1 2024-06-03T01:21:01.367 INFO:tasks.workunit.client.0.smithi032.stdout:3/329: stat de/d29/c34 0 2024-06-03T01:21:01.368 INFO:tasks.workunit.client.1.smithi044.stdout:0/977: link d3/d6b/de8/d147/f14b d3/d9/d55/d5a/f15f 0 2024-06-03T01:21:01.368 INFO:tasks.workunit.client.1.smithi044.stdout:0/978: write d3/d5c/de3/f13e [681103,118341] 0 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:1/203: rename d18/d2d/d2f/f3b to d18/d27/d2b/f3e 0 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:1/204: fsync d18/d27/f31 0 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:2/362: rename d5/d38/d46 to d5/d38/d46/d53/d57/d6d/d79 22 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:1/205: fsync d18/f1a 0 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:1/206: chown d18/d32/l37 379729394 1 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:6/350: truncate d8/d11/d20/d21/f24 1663346 0 2024-06-03T01:21:01.369 INFO:tasks.workunit.client.0.smithi032.stdout:6/351: chown d8/d11/d20/d2b 5327129 1 2024-06-03T01:21:01.371 INFO:tasks.workunit.client.0.smithi032.stdout:9/431: symlink d0/d23/d24/d26/l95 0 2024-06-03T01:21:01.374 INFO:tasks.workunit.client.0.smithi032.stdout:0/320: creat d1/da/d28/d3b/d5d/f65 x:0 0 0 2024-06-03T01:21:01.380 INFO:tasks.workunit.client.1.smithi044.stdout:8/871: rmdir d5/d55/d74/d82/db0 39 2024-06-03T01:21:01.381 INFO:tasks.workunit.client.0.smithi032.stdout:3/330: rename de/d29/d2e/d31/f1b to de/d29/d2e/d31/d3c/d50/d56/f67 0 2024-06-03T01:21:01.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/363: symlink d5/d38/d46/d53/l7a 0 2024-06-03T01:21:01.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/364: dread - d5/da/f59 zero size 2024-06-03T01:21:01.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/365: write d5/f58 [929812,19564] 0 2024-06-03T01:21:01.383 INFO:tasks.workunit.client.0.smithi032.stdout:6/352: truncate d8/d11/d20/d2b/f4a 822572 0 2024-06-03T01:21:01.384 INFO:tasks.workunit.client.1.smithi044.stdout:0/979: dwrite d3/d9/d102/d159/f48 [4194304,4194304] 0 2024-06-03T01:21:01.384 INFO:tasks.workunit.client.1.smithi044.stdout:7/841: symlink d0/l10f 0 2024-06-03T01:21:01.384 INFO:tasks.workunit.client.0.smithi032.stdout:9/432: rmdir d0/d11/d1d/d4a 39 2024-06-03T01:21:01.384 INFO:tasks.workunit.client.1.smithi044.stdout:9/779: creat d6/dd/d51/d16/d5f/d8b/ffc x:0 0 0 2024-06-03T01:21:01.387 INFO:tasks.workunit.client.1.smithi044.stdout:2/712: getdents db/d1b/d3f 0 2024-06-03T01:21:01.387 INFO:tasks.workunit.client.0.smithi032.stdout:1/207: mkdir d18/d3f 0 2024-06-03T01:21:01.391 INFO:tasks.workunit.client.1.smithi044.stdout:1/990: mkdir d0/dd2/dfe/dbe/d9e/d10d/d45/d4b/d5a/d14e 0 2024-06-03T01:21:01.392 INFO:tasks.workunit.client.1.smithi044.stdout:5/786: rename d2/d11/d12/d2b/d44 to d2/d11/d1e/d21/d24/d9e/d107 0 2024-06-03T01:21:01.396 INFO:tasks.workunit.client.1.smithi044.stdout:7/842: mkdir d0/d3/d8/d95/d110 0 2024-06-03T01:21:01.396 INFO:tasks.workunit.client.0.smithi032.stdout:2/366: rmdir d5/d38/d46 39 2024-06-03T01:21:01.397 INFO:tasks.workunit.client.0.smithi032.stdout:2/367: write d5/da/f1e [6809455,127174] 0 2024-06-03T01:21:01.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/980: mkdir d3/da6/d21/d94/da3/d15d/d160 0 2024-06-03T01:21:01.399 INFO:tasks.workunit.client.1.smithi044.stdout:0/981: write d3/d9/d102/f11f [642392,126624] 0 2024-06-03T01:21:01.402 INFO:tasks.workunit.client.1.smithi044.stdout:9/780: mkdir d6/dd/d51/d16/d27/d5e/db7/dfd 0 2024-06-03T01:21:01.404 INFO:tasks.workunit.client.0.smithi032.stdout:0/321: dwrite d1/f35 [4194304,4194304] 0 2024-06-03T01:21:01.404 INFO:tasks.workunit.client.0.smithi032.stdout:0/322: getdents d1/d46/d3a 0 2024-06-03T01:21:01.404 INFO:tasks.workunit.client.0.smithi032.stdout:9/433: dwrite d0/d11/d1d/d4a/d71/f82 [0,4194304] 0 2024-06-03T01:21:01.404 INFO:tasks.workunit.client.0.smithi032.stdout:3/331: getdents de/d29/d2e/d31/d3c 0 2024-06-03T01:21:01.409 INFO:tasks.workunit.client.0.smithi032.stdout:1/208: mknod d18/d32/c40 0 2024-06-03T01:21:01.409 INFO:tasks.workunit.client.0.smithi032.stdout:1/209: dread - d18/f34 zero size 2024-06-03T01:21:01.411 INFO:tasks.workunit.client.0.smithi032.stdout:6/353: creat d8/d11/d20/d31/d3f/f69 x:0 0 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:1/991: mknod d0/d3/c14f 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:6/955: rename d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/cbe to d2/d23/df4/c149 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:2/713: creat db/d1b/d5d/da7/dc9/fe3 x:0 0 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:8/872: getdents d5/d55/d56/d5c 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:6/956: dread - d2/d23/d55/d5f/d53/fb9 zero size 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:6/957: write d2/d23/d30/f1c [6751605,27355] 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:0/982: mknod d3/d9/d55/d7b/c161 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:9/781: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dfe 0 2024-06-03T01:21:01.414 INFO:tasks.workunit.client.1.smithi044.stdout:0/983: write d3/da6/dc1/dc2/df8/f110 [5215269,13250] 0 2024-06-03T01:21:01.415 INFO:tasks.workunit.client.0.smithi032.stdout:0/323: mknod d1/da/d10/c66 0 2024-06-03T01:21:01.415 INFO:tasks.workunit.client.0.smithi032.stdout:0/324: chown d1/d46/f48 340580 1 2024-06-03T01:21:01.415 INFO:tasks.workunit.client.0.smithi032.stdout:0/325: creat d1/da/d28/d3b/d5d/f67 x:0 0 0 2024-06-03T01:21:01.415 INFO:tasks.workunit.client.1.smithi044.stdout:1/992: mknod d0/d3/d20/d7/d13/d112/c150 0 2024-06-03T01:21:01.415 INFO:tasks.workunit.client.1.smithi044.stdout:1/993: chown d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d81/fcf 1600267 1 2024-06-03T01:21:01.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/873: mknod d5/d16/d81/de3/d58/d89/dbf/dce/c123 0 2024-06-03T01:21:01.418 INFO:tasks.workunit.client.0.smithi032.stdout:9/434: rename d0/d23/d24/f3b to d0/d11/d1d/d4a/d71/f96 0 2024-06-03T01:21:01.421 INFO:tasks.workunit.client.1.smithi044.stdout:5/787: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/fc1 [0,4194304] 0 2024-06-03T01:21:01.421 INFO:tasks.workunit.client.1.smithi044.stdout:5/788: chown d2/d11/d1e/d21/d24/d93 25020 1 2024-06-03T01:21:01.422 INFO:tasks.workunit.client.0.smithi032.stdout:9/435: creat d0/d11/d1d/d38/f97 x:0 0 0 2024-06-03T01:21:01.424 INFO:tasks.workunit.client.1.smithi044.stdout:6/958: link d2/d23/f35 d2/d23/d30/d1e/d38/d6c/d5c/d79/f14a 0 2024-06-03T01:21:01.426 INFO:tasks.workunit.client.0.smithi032.stdout:2/368: link d5/da/d1a/f26 d5/da/d1a/f7b 0 2024-06-03T01:21:01.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/354: dread d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:21:01.428 INFO:tasks.workunit.client.0.smithi032.stdout:0/326: write d1/f1c [2007053,6041] 0 2024-06-03T01:21:01.431 INFO:tasks.workunit.client.0.smithi032.stdout:9/436: getdents d0/d11/d79 0 2024-06-03T01:21:01.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/782: mknod d6/dd/d51/d16/d27/d5e/db7/de3/cff 0 2024-06-03T01:21:01.434 INFO:tasks.workunit.client.0.smithi032.stdout:2/369: link d5/da/d2e/f32 d5/da/d1a/d2a/d49/f7c 0 2024-06-03T01:21:01.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/730: sync 2024-06-03T01:21:01.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/731: readlink df/d1e/l5d 0 2024-06-03T01:21:01.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/732: write df/d13/d1b/f5e [958755,129330] 0 2024-06-03T01:21:01.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/733: creat df/d1e/d2a/d70/ff0 x:0 0 0 2024-06-03T01:21:01.436 INFO:tasks.workunit.client.1.smithi044.stdout:5/789: unlink d2/d11/d1e/d21/d24/d93/d9d/d28/fd3 0 2024-06-03T01:21:01.436 INFO:tasks.workunit.client.1.smithi044.stdout:5/790: chown d2/d11/d1e/d21/d24/d9e/db1 11366 1 2024-06-03T01:21:01.437 INFO:tasks.workunit.client.1.smithi044.stdout:5/791: getdents d2/d11/d1e/d21/d24/d93/d9d/d28/dfe 0 2024-06-03T01:21:01.437 INFO:tasks.workunit.client.1.smithi044.stdout:5/792: read - d2/d11/fb2 zero size 2024-06-03T01:21:01.437 INFO:tasks.workunit.client.1.smithi044.stdout:5/793: write d2/d11/d1e/d3b/da6/fd6 [919027,102150] 0 2024-06-03T01:21:01.437 INFO:tasks.workunit.client.1.smithi044.stdout:6/959: rmdir d2/d23/d55/d5f/d99 39 2024-06-03T01:21:01.437 INFO:tasks.workunit.client.1.smithi044.stdout:6/960: fdatasync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/f121 0 2024-06-03T01:21:01.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/865: sync 2024-06-03T01:21:01.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/866: dread - d17/d1c/d7e/ffa zero size 2024-06-03T01:21:01.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/867: fdatasync d17/d1c/d75/d7a/db3/fd9 0 2024-06-03T01:21:01.439 INFO:tasks.workunit.client.1.smithi044.stdout:7/843: dwrite d0/d16/d23/fd2 [0,4194304] 0 2024-06-03T01:21:01.439 INFO:tasks.workunit.client.1.smithi044.stdout:7/844: chown d0/d16/d23/c70 156 1 2024-06-03T01:21:01.439 INFO:tasks.workunit.client.1.smithi044.stdout:1/994: mknod d0/dd2/dfe/dc2/c151 0 2024-06-03T01:21:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.437+0000 7fc191e27700 1 -- 172.21.15.32:0/1345848307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 msgr2=0x7fc18c0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.437+0000 7fc191e27700 1 --2- 172.21.15.32:0/1345848307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c0ff380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009910 tx=0x7fc17c009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.439+0000 7fc191e27700 1 -- 172.21.15.32:0/1345848307 shutdown_connections 2024-06-03T01:21:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.439+0000 7fc191e27700 1 --2- 172.21.15.32:0/1345848307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.439+0000 7fc191e27700 1 --2- 172.21.15.32:0/1345848307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c0ff380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.439+0000 7fc191e27700 1 -- 172.21.15.32:0/1345848307 >> 172.21.15.32:0/1345848307 conn(0x7fc18c0fa8c0 msgr2=0x7fc18c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:01.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.440+0000 7fc191e27700 1 -- 172.21.15.32:0/1345848307 shutdown_connections 2024-06-03T01:21:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.441+0000 7fc191e27700 1 -- 172.21.15.32:0/1345848307 wait complete. 2024-06-03T01:21:01.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/210: dwrite d18/d27/f35 [0,4194304] 0 2024-06-03T01:21:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.442+0000 7fc191e27700 1 Processor -- start 2024-06-03T01:21:01.443 INFO:tasks.workunit.client.1.smithi044.stdout:5/794: mkdir d2/d11/d12/d2b/d108 0 2024-06-03T01:21:01.443 INFO:tasks.workunit.client.1.smithi044.stdout:7/845: write d0/d3/d8/f1b [4561500,19061] 0 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.443+0000 7fc191e27700 1 -- start start 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.443+0000 7fc191e27700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.443+0000 7fc191e27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.443+0000 7fc191e27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c072950 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.443+0000 7fc191e27700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1aa910 con 0x7fc18c0fef60 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.444+0000 7fc18affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.444+0000 7fc18affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49056/0 (socket says 172.21.15.32:49056) 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.444+0000 7fc18affd700 1 -- 172.21.15.32:0/2319973343 learned_addr learned my addr 172.21.15.32:0/2319973343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.444+0000 7fc18b7fe700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:01.445 INFO:tasks.workunit.client.1.smithi044.stdout:6/961: symlink d2/d23/d30/dfe/l14b 0 2024-06-03T01:21:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.445+0000 7fc18affd700 1 -- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 msgr2=0x7fc18c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.445+0000 7fc18affd700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.445+0000 7fc18affd700 1 -- 172.21.15.32:0/2319973343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0095f0 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.445+0000 7fc18affd700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b4d0 tx=0x7fc18000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.446+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc180011800 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.446+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc180011e10 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.446+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc18000f2e0 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.446+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c1aab10 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.446+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c1ab010 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.448+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) v1 ==== 46426+0+0 (secure 0 0 0) 0x7fc18000f440 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/332: dwrite de/d29/d2e/d31/d3c/d50/d56/f67 [0,4194304] 0 2024-06-03T01:21:01.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/355: rmdir d8/d11 39 2024-06-03T01:21:01.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/795: creat d2/d11/d1e/d2f/df5/f109 x:0 0 0 2024-06-03T01:21:01.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/796: truncate d2/d11/d12/d2b/f72 4683873 0 2024-06-03T01:21:01.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/797: write d2/d11/d1e/d21/d24/d93/d9d/f8c [1623730,51510] 0 2024-06-03T01:21:01.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/798: chown d2/d11/d12/f6c 7997158 1 2024-06-03T01:21:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.448+0000 7fc188ff9700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 0x7fc17403b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.448+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc18004e0c0 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.449+0000 7fc18b7fe700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 0x7fc17403b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:01.453 INFO:tasks.workunit.client.1.smithi044.stdout:4/868: rmdir d17/d63 39 2024-06-03T01:21:01.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.453+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c10feb0 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.455 INFO:tasks.workunit.client.1.smithi044.stdout:0/984: dwrite d3/d9/d102/d159/f43 [4194304,4194304] 0 2024-06-03T01:21:01.455 INFO:tasks.workunit.client.1.smithi044.stdout:8/874: write d5/d20/f9e [285595,1650] 0 2024-06-03T01:21:01.456 INFO:tasks.workunit.client.1.smithi044.stdout:0/985: write d3/da6/dc1/dc2/de4/ff4 [883867,11726] 0 2024-06-03T01:21:01.456 INFO:tasks.workunit.client.1.smithi044.stdout:8/875: creat d5/d16/d1d/d32/f124 x:0 0 0 2024-06-03T01:21:01.457 INFO:tasks.workunit.client.1.smithi044.stdout:6/962: mkdir d2/d23/d55/d5f/db2/d3f/d14c 0 2024-06-03T01:21:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.457+0000 7fc18b7fe700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 0x7fc17403b8c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009fd0 tx=0x7fc17c00b540 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:01.458 INFO:tasks.workunit.client.0.smithi032.stdout:3/333: creat de/d29/d2e/d31/d3c/d50/d66/f68 x:0 0 0 2024-06-03T01:21:01.459 INFO:tasks.workunit.client.1.smithi044.stdout:5/799: rename d2/d11/d1e/d21/d24/l68 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/l10a 0 2024-06-03T01:21:01.460 INFO:tasks.workunit.client.0.smithi032.stdout:6/356: rename d8/d11/d20/d21/f24 to d8/d55/f6a 0 2024-06-03T01:21:01.461 INFO:tasks.workunit.client.1.smithi044.stdout:4/869: mkdir d17/d1c/d75/d7a/db8/d35/d124 0 2024-06-03T01:21:01.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/211: dread d18/d27/f31 [0,4194304] 0 2024-06-03T01:21:01.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/212: chown d18/l28 598 1 2024-06-03T01:21:01.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/213: creat d18/d27/f41 x:0 0 0 2024-06-03T01:21:01.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/214: chown d18/f25 34786 1 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/846: dread d0/d3/f47 [0,4194304] 0 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:2/714: dwrite db/d1b/d27/f39 [0,4194304] 0 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/847: creat d0/d3/d8/d95/d63/d89/da3/d9a/f111 x:0 0 0 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/848: read d0/d3/d8/d95/d63/f7a [577618,56940] 0 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/849: fdatasync d0/d16/d23/fd2 0 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/850: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/f9b zero size 2024-06-03T01:21:01.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/851: write d0/f9f [5196442,5384] 0 2024-06-03T01:21:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.464+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fc180017890 con 0x7fc18c0ffcd0 2024-06-03T01:21:01.467 INFO:tasks.workunit.client.0.smithi032.stdout:1/215: unlink f15 0 2024-06-03T01:21:01.467 INFO:tasks.workunit.client.0.smithi032.stdout:1/216: fsync f17 0 2024-06-03T01:21:01.467 INFO:tasks.workunit.client.1.smithi044.stdout:8/876: rename d5/f2d to d5/d55/d74/db4/dba/f125 0 2024-06-03T01:21:01.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/217: unlink d18/d27/f35 0 2024-06-03T01:21:01.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/218: chown d18/d32 1 1 2024-06-03T01:21:01.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/219: fsync f17 0 2024-06-03T01:21:01.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/220: stat d18/c36 0 2024-06-03T01:21:01.470 INFO:tasks.workunit.client.1.smithi044.stdout:5/800: mknod d2/d11/d1e/d21/c10b 0 2024-06-03T01:21:01.472 INFO:tasks.workunit.client.1.smithi044.stdout:8/877: symlink d5/d55/d56/d5c/d68/de5/dea/de2/l126 0 2024-06-03T01:21:01.472 INFO:tasks.workunit.client.0.smithi032.stdout:1/221: mknod d18/d32/c42 0 2024-06-03T01:21:01.475 INFO:tasks.workunit.client.1.smithi044.stdout:4/870: dread d17/d1c/d9c/fc1 [0,4194304] 0 2024-06-03T01:21:01.482 INFO:tasks.workunit.client.1.smithi044.stdout:0/986: dread d3/d9/d102/d159/fb4 [0,4194304] 0 2024-06-03T01:21:01.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/357: dwrite d8/d11/f3e [4194304,4194304] 0 2024-06-03T01:21:01.487 INFO:tasks.workunit.client.0.smithi032.stdout:3/334: dread de/d29/d2e/d31/d3c/f61 [0,4194304] 0 2024-06-03T01:21:01.487 INFO:tasks.workunit.client.0.smithi032.stdout:3/335: readlink de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:01.489 INFO:tasks.workunit.client.1.smithi044.stdout:7/852: dread d0/d3/d8/d95/f65 [0,4194304] 0 2024-06-03T01:21:01.489 INFO:tasks.workunit.client.1.smithi044.stdout:7/853: chown d0/d3/d8/d95/d63/d89/da3 60 1 2024-06-03T01:21:01.491 INFO:tasks.workunit.client.1.smithi044.stdout:2/715: dread db/d1b/d2a/f43 [0,4194304] 0 2024-06-03T01:21:01.496 INFO:tasks.workunit.client.0.smithi032.stdout:3/336: unlink de/d29/f35 0 2024-06-03T01:21:01.497 INFO:tasks.workunit.client.0.smithi032.stdout:1/222: dread d18/f19 [0,4194304] 0 2024-06-03T01:21:01.497 INFO:tasks.workunit.client.0.smithi032.stdout:6/358: link d8/c4b d8/d11/d20/c6b 0 2024-06-03T01:21:01.498 INFO:tasks.workunit.client.1.smithi044.stdout:7/854: dread d0/d3/ddf/ff4 [0,4194304] 0 2024-06-03T01:21:01.502 INFO:tasks.workunit.client.0.smithi032.stdout:3/337: mkdir de/d29/d2e/d31/d23/d53/d69 0 2024-06-03T01:21:01.502 INFO:tasks.workunit.client.0.smithi032.stdout:6/359: creat d8/d11/d20/d31/f6c x:0 0 0 2024-06-03T01:21:01.502 INFO:tasks.workunit.client.0.smithi032.stdout:1/223: write d18/d27/f31 [3868882,62708] 0 2024-06-03T01:21:01.503 INFO:tasks.workunit.client.1.smithi044.stdout:1/995: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/f67 [0,4194304] 0 2024-06-03T01:21:01.503 INFO:tasks.workunit.client.1.smithi044.stdout:1/996: stat d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/fbb 0 2024-06-03T01:21:01.503 INFO:tasks.workunit.client.0.smithi032.stdout:6/360: creat d8/d11/d20/d3b/f6d x:0 0 0 2024-06-03T01:21:01.506 INFO:tasks.workunit.client.0.smithi032.stdout:0/327: dwrite d1/da/f4d [0,4194304] 0 2024-06-03T01:21:01.506 INFO:tasks.workunit.client.0.smithi032.stdout:1/224: mknod d18/d3f/c43 0 2024-06-03T01:21:01.506 INFO:tasks.workunit.client.0.smithi032.stdout:1/225: fdatasync d18/f25 0 2024-06-03T01:21:01.508 INFO:tasks.workunit.client.0.smithi032.stdout:6/361: rmdir d8/d4c/d52 0 2024-06-03T01:21:01.512 INFO:tasks.workunit.client.1.smithi044.stdout:3/734: dwrite df/d13/d39/d3d/d43/f5b [4194304,4194304] 0 2024-06-03T01:21:01.513 INFO:tasks.workunit.client.1.smithi044.stdout:9/783: dwrite d6/dd/d51/d16/d27/d5e/d71/dc1/fc7 [0,4194304] 0 2024-06-03T01:21:01.516 INFO:tasks.workunit.client.1.smithi044.stdout:6/963: dwrite d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/fac [0,4194304] 0 2024-06-03T01:21:01.516 INFO:tasks.workunit.client.1.smithi044.stdout:6/964: chown d2/d23/d55/d5f/db2/d3f/d14c 2278209 1 2024-06-03T01:21:01.516 INFO:tasks.workunit.client.1.smithi044.stdout:9/784: dread d6/dd/f87 [0,4194304] 0 2024-06-03T01:21:01.517 INFO:tasks.workunit.client.0.smithi032.stdout:9/437: dwrite d0/d11/d1d/d38/d4f/f60 [0,4194304] 0 2024-06-03T01:21:01.517 INFO:tasks.workunit.client.0.smithi032.stdout:9/438: fdatasync d0/fa 0 2024-06-03T01:21:01.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/439: write d0/d23/d25/f62 [1010317,70082] 0 2024-06-03T01:21:01.518 INFO:tasks.workunit.client.1.smithi044.stdout:5/801: dwrite d2/d11/d12/f104 [0,4194304] 0 2024-06-03T01:21:01.519 INFO:tasks.workunit.client.0.smithi032.stdout:1/226: rename d18/l2a to d18/d27/d2b/l44 0 2024-06-03T01:21:01.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/871: dwrite d17/d1c/d75/d7a/db8/d28/d2b/f93 [0,4194304] 0 2024-06-03T01:21:01.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/872: chown d17/d1c/d75/d7a/db8/d28/l8c 2554 1 2024-06-03T01:21:01.521 INFO:tasks.workunit.client.0.smithi032.stdout:6/362: symlink d8/d50/l6e 0 2024-06-03T01:21:01.525 INFO:tasks.workunit.client.0.smithi032.stdout:3/338: dwrite de/d29/f65 [0,4194304] 0 2024-06-03T01:21:01.525 INFO:tasks.workunit.client.0.smithi032.stdout:3/339: dread - de/d29/d2e/d31/f1e zero size 2024-06-03T01:21:01.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/440: link d0/d11/d76/l93 d0/d11/d79/l98 0 2024-06-03T01:21:01.527 INFO:tasks.workunit.client.1.smithi044.stdout:5/802: write d2/d11/d12/d2b/f34 [1552681,55991] 0 2024-06-03T01:21:01.529 INFO:tasks.workunit.client.0.smithi032.stdout:1/227: truncate d18/d29/f2e 1344302 0 2024-06-03T01:21:01.530 INFO:tasks.workunit.client.1.smithi044.stdout:6/965: dread d2/d23/d30/d1e/d38/d77/f7c [0,4194304] 0 2024-06-03T01:21:01.530 INFO:tasks.workunit.client.1.smithi044.stdout:6/966: readlink d2/d23/d30/d1e/d38/lb3 0 2024-06-03T01:21:01.530 INFO:tasks.workunit.client.1.smithi044.stdout:6/967: write d2/d23/d55/d5f/db2/d3f/fc5 [850698,105554] 0 2024-06-03T01:21:01.530 INFO:tasks.workunit.client.1.smithi044.stdout:6/968: dread - d2/d23/d44/d4d/f107 zero size 2024-06-03T01:21:01.532 INFO:tasks.workunit.client.0.smithi032.stdout:6/363: link d8/fb d8/f6f 0 2024-06-03T01:21:01.534 INFO:tasks.workunit.client.1.smithi044.stdout:4/873: dread d17/d1c/d75/d7a/db8/d59/f81 [0,4194304] 0 2024-06-03T01:21:01.537 INFO:tasks.workunit.client.0.smithi032.stdout:3/340: symlink de/d29/d2e/d31/d24/l6a 0 2024-06-03T01:21:01.540 INFO:tasks.workunit.client.0.smithi032.stdout:9/441: unlink d0/d23/c40 0 2024-06-03T01:21:01.544 INFO:tasks.workunit.client.1.smithi044.stdout:4/874: write d17/d63/dae/fbf [477795,48817] 0 2024-06-03T01:21:01.546 INFO:tasks.workunit.client.0.smithi032.stdout:3/341: dread de/d29/d2e/d31/d3c/d50/d56/f67 [0,4194304] 0 2024-06-03T01:21:01.551 INFO:tasks.workunit.client.1.smithi044.stdout:4/875: dread d17/d1c/d75/d7a/db8/f9b [4194304,4194304] 0 2024-06-03T01:21:01.551 INFO:tasks.workunit.client.1.smithi044.stdout:4/876: chown d17/d1c/d70/c106 2684047 1 2024-06-03T01:21:01.556 INFO:tasks.workunit.client.0.smithi032.stdout:1/228: rename d18/f1e to d18/d29/f45 0 2024-06-03T01:21:01.557 INFO:tasks.workunit.client.1.smithi044.stdout:0/987: getdents d3/d6b/d154/d15b 0 2024-06-03T01:21:01.558 INFO:tasks.workunit.client.0.smithi032.stdout:6/364: mkdir d8/d11/d20/d3b/d66/d70 0 2024-06-03T01:21:01.558 INFO:tasks.workunit.client.0.smithi032.stdout:6/365: fdatasync d8/d4c/f58 0 2024-06-03T01:21:01.560 INFO:tasks.workunit.client.0.smithi032.stdout:1/229: mknod d18/d3f/c46 0 2024-06-03T01:21:01.563 INFO:tasks.workunit.client.0.smithi032.stdout:9/442: getdents d0/d11/d1d/d38 0 2024-06-03T01:21:01.565 INFO:tasks.workunit.client.0.smithi032.stdout:1/230: rename d18/d2d/d2f to d18/d29/d3d/d47 0 2024-06-03T01:21:01.565 INFO:tasks.workunit.client.0.smithi032.stdout:1/231: dread - d18/f34 zero size 2024-06-03T01:21:01.566 INFO:tasks.workunit.client.0.smithi032.stdout:6/366: symlink d8/d55/l71 0 2024-06-03T01:21:01.568 INFO:tasks.workunit.client.0.smithi032.stdout:1/232: mknod d18/d2d/c48 0 2024-06-03T01:21:01.568 INFO:tasks.workunit.client.0.smithi032.stdout:1/233: readlink d18/d32/l37 0 2024-06-03T01:21:01.568 INFO:tasks.workunit.client.0.smithi032.stdout:1/234: fdatasync d18/d27/f31 0 2024-06-03T01:21:01.568 INFO:tasks.workunit.client.0.smithi032.stdout:1/235: read d18/d27/f31 [963671,105657] 0 2024-06-03T01:21:01.569 INFO:tasks.workunit.client.0.smithi032.stdout:6/367: truncate d8/f6f 6747378 0 2024-06-03T01:21:01.571 INFO:tasks.workunit.client.0.smithi032.stdout:6/368: truncate d8/d11/f49 277225 0 2024-06-03T01:21:01.572 INFO:tasks.workunit.client.0.smithi032.stdout:6/369: symlink d8/d5f/l72 0 2024-06-03T01:21:01.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/236: write d18/f22 [1942707,118383] 0 2024-06-03T01:21:01.580 INFO:tasks.workunit.client.1.smithi044.stdout:7/855: write d0/d3/d8/d95/d63/fce [636547,77406] 0 2024-06-03T01:21:01.580 INFO:tasks.workunit.client.1.smithi044.stdout:7/856: chown d0/d3/l88 3272 1 2024-06-03T01:21:01.581 INFO:tasks.workunit.client.1.smithi044.stdout:7/857: chown d0/d3/d8/d95/f46 118 1 2024-06-03T01:21:01.582 INFO:tasks.workunit.client.0.smithi032.stdout:6/370: write d8/d11/d20/d3b/f41 [1888569,129302] 0 2024-06-03T01:21:01.582 INFO:tasks.workunit.client.0.smithi032.stdout:6/371: readlink d8/d11/d20/d31/l33 0 2024-06-03T01:21:01.584 INFO:tasks.workunit.client.0.smithi032.stdout:6/372: mknod d8/d55/c73 0 2024-06-03T01:21:01.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/373: link d8/d11/d20/d2b/c2e d8/d4c/c74 0 2024-06-03T01:21:01.589 INFO:tasks.workunit.client.1.smithi044.stdout:7/858: read d0/d3/d8/f37 [3098014,80134] 0 2024-06-03T01:21:01.593 INFO:tasks.workunit.client.1.smithi044.stdout:1/997: mknod d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/d30/dd6/c152 0 2024-06-03T01:21:01.593 INFO:tasks.workunit.client.1.smithi044.stdout:1/998: write d0/dd2/dfe/dbe/d9e/d10d/d45/d6b/d134/dae/f110 [768841,98867] 0 2024-06-03T01:21:01.597 INFO:tasks.workunit.client.0.smithi032.stdout:1/237: dwrite d18/f1c [4194304,4194304] 0 2024-06-03T01:21:01.598 INFO:tasks.workunit.client.0.smithi032.stdout:1/238: symlink d18/d32/d3c/l49 0 2024-06-03T01:21:01.598 INFO:tasks.workunit.client.0.smithi032.stdout:1/239: creat d18/d32/d39/f4a x:0 0 0 2024-06-03T01:21:01.599 INFO:tasks.workunit.client.0.smithi032.stdout:1/240: link d18/d27/f41 d18/d32/f4b 0 2024-06-03T01:21:01.600 INFO:tasks.workunit.client.1.smithi044.stdout:7/859: write d0/d3/d8/d32/d33/fb1 [925399,64038] 0 2024-06-03T01:21:01.604 INFO:tasks.workunit.client.1.smithi044.stdout:2/716: mkdir db/d1b/d2a/d67/de4 0 2024-06-03T01:21:01.614 INFO:tasks.workunit.client.1.smithi044.stdout:9/785: truncate d6/dd/d51/d16/d27/d5e/d71/fba 532775 0 2024-06-03T01:21:01.614 INFO:tasks.workunit.client.1.smithi044.stdout:9/786: chown d6/dd/d51 85 1 2024-06-03T01:21:01.615 INFO:tasks.workunit.client.1.smithi044.stdout:9/787: truncate d6/dd/d51/d16/f1b 1298312 0 2024-06-03T01:21:01.620 INFO:tasks.workunit.client.0.smithi032.stdout:1/241: dwrite d18/f19 [0,4194304] 0 2024-06-03T01:21:01.621 INFO:tasks.workunit.client.0.smithi032.stdout:1/242: truncate d18/f1a 2742003 0 2024-06-03T01:21:01.626 INFO:tasks.workunit.client.1.smithi044.stdout:9/788: dwrite d6/dd/d51/f13 [0,4194304] 0 2024-06-03T01:21:01.626 INFO:tasks.workunit.client.1.smithi044.stdout:5/803: mknod d2/d11/d3e/dce/dd1/c10c 0 2024-06-03T01:21:01.632 INFO:tasks.workunit.client.1.smithi044.stdout:4/877: symlink d17/d1c/d75/d7a/db8/d59/de5/l125 0 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/860: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/df6/f112 x:0 0 0 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/861: write d0/d3/d8/d95/d63/d89/da3/d9a/fe1 [125709,85255] 0 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:2/717: mkdir db/d15/d49/d6c/d72/d8f/de5 0 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:9/789: symlink d6/dd/d51/d16/d5f/d8b/l100 0 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:9/790: chown d6/dd/d51/fe 196279312 1 2024-06-03T01:21:01.638 INFO:tasks.workunit.client.1.smithi044.stdout:9/791: readlink d6/dd/d51/l6d 0 2024-06-03T01:21:01.639 INFO:tasks.workunit.client.1.smithi044.stdout:5/804: rename d2/d11/d3e/f40 to d2/d11/d3e/f10d 0 2024-06-03T01:21:01.640 INFO:tasks.workunit.client.1.smithi044.stdout:4/878: creat d17/d63/dae/f126 x:0 0 0 2024-06-03T01:21:01.641 INFO:tasks.workunit.client.1.smithi044.stdout:9/792: mkdir d6/dd/d51/d16/d27/d5e/db7/dfd/d101 0 2024-06-03T01:21:01.641 INFO:tasks.workunit.client.1.smithi044.stdout:9/793: write d6/dd/f17 [2902134,103654] 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:7/862: read d0/d3/d8/d32/d33/f57 [1812156,2285] 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/805: write d2/d11/d12/d2b/d4a/d57/f6b [809741,64873] 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:7/863: truncate d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/fd8 440569 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/806: write d2/d11/d1e/d21/d24/f69 [1417481,128326] 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/807: write d2/d11/d1e/d21/d24/fa7 [953401,15383] 0 2024-06-03T01:21:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/808: creat d2/d11/d12/dad/f10e x:0 0 0 2024-06-03T01:21:01.643 INFO:tasks.workunit.client.1.smithi044.stdout:4/879: symlink d17/d1c/d75/d7a/db8/d35/d124/l127 0 2024-06-03T01:21:01.646 INFO:tasks.workunit.client.1.smithi044.stdout:9/794: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [2435878,107584] 0 2024-06-03T01:21:01.646 INFO:tasks.workunit.client.1.smithi044.stdout:7/864: rmdir d0/d3/da8/de5/d101 0 2024-06-03T01:21:01.646 INFO:tasks.workunit.client.1.smithi044.stdout:7/865: readlink d0/d3/d8/l4e 0 2024-06-03T01:21:01.650 INFO:tasks.workunit.client.1.smithi044.stdout:9/795: rmdir d6/dd/d51/d16/d27/d5e/db7/dfd/d101 0 2024-06-03T01:21:01.652 INFO:tasks.workunit.client.1.smithi044.stdout:9/796: mknod d6/dd/d45/c102 0 2024-06-03T01:21:01.655 INFO:tasks.workunit.client.1.smithi044.stdout:2/718: dwrite db/d1b/d2a/d52/fb4 [4194304,4194304] 0 2024-06-03T01:21:01.666 INFO:tasks.workunit.client.1.smithi044.stdout:2/719: dread db/d1b/f99 [0,4194304] 0 2024-06-03T01:21:01.666 INFO:tasks.workunit.client.1.smithi044.stdout:2/720: write db/d15/d49/d6c/d72/fd1 [957906,29729] 0 2024-06-03T01:21:01.666 INFO:tasks.workunit.client.1.smithi044.stdout:2/721: chown db/d15/l20 2654 1 2024-06-03T01:21:01.668 INFO:tasks.workunit.client.1.smithi044.stdout:2/722: mkdir db/d1b/d2a/d67/d53/d9b/de6 0 2024-06-03T01:21:01.674 INFO:tasks.workunit.client.1.smithi044.stdout:2/723: write db/d1b/d5d/f62 [2916261,10478] 0 2024-06-03T01:21:01.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/724: mknod db/d1b/d2a/d52/d54/db6/ce7 0 2024-06-03T01:21:01.699 INFO:tasks.workunit.client.0.smithi032.stdout:9/443: read d0/d23/d25/f8e [2602518,98032] 0 2024-06-03T01:21:01.705 INFO:tasks.workunit.client.1.smithi044.stdout:0/988: mknod d3/d9/d55/d73/c162 0 2024-06-03T01:21:01.705 INFO:tasks.workunit.client.1.smithi044.stdout:0/989: read - d3/da6/d21/d94/da3/dad/fbc zero size 2024-06-03T01:21:01.707 INFO:tasks.workunit.client.0.smithi032.stdout:6/374: dwrite d8/d11/d20/d2b/d39/d19/f4d [0,4194304] 0 2024-06-03T01:21:01.709 INFO:tasks.workunit.client.1.smithi044.stdout:2/725: read db/d15/d49/d6c/f8c [4079152,70087] 0 2024-06-03T01:21:01.709 INFO:tasks.workunit.client.1.smithi044.stdout:2/726: fdatasync db/f12 0 2024-06-03T01:21:01.711 INFO:tasks.workunit.client.1.smithi044.stdout:0/990: dread d3/da6/f3e [0,4194304] 0 2024-06-03T01:21:01.712 INFO:tasks.workunit.client.0.smithi032.stdout:6/375: symlink d8/d11/d20/d21/l75 0 2024-06-03T01:21:01.712 INFO:tasks.workunit.client.0.smithi032.stdout:6/376: read - d8/d50/f62 zero size 2024-06-03T01:21:01.712 INFO:tasks.workunit.client.0.smithi032.stdout:6/377: chown d8/d11/d20/d2b/d39/d19/f56 20842 1 2024-06-03T01:21:01.714 INFO:tasks.workunit.client.0.smithi032.stdout:6/378: dread d8/d11/f34 [0,4194304] 0 2024-06-03T01:21:01.715 INFO:tasks.workunit.client.1.smithi044.stdout:7/866: dwrite d0/d3/d8/d32/f90 [0,4194304] 0 2024-06-03T01:21:01.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/867: truncate d0/f9f 2688525 0 2024-06-03T01:21:01.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/868: write d0/f62 [939801,48943] 0 2024-06-03T01:21:01.717 INFO:tasks.workunit.client.0.smithi032.stdout:1/243: rmdir d18 39 2024-06-03T01:21:01.720 INFO:tasks.workunit.client.0.smithi032.stdout:6/379: write d8/d11/d20/d31/f45 [3183951,8670] 0 2024-06-03T01:21:01.726 INFO:tasks.workunit.client.1.smithi044.stdout:1/999: dwrite d0/dd2/dfe/dbe/d9e/d10d/d45/d6f/d13e/d108/f3a [0,4194304] 0 2024-06-03T01:21:01.726 INFO:tasks.workunit.client.0.smithi032.stdout:9/444: dwrite d0/ff [0,4194304] 0 2024-06-03T01:21:01.729 INFO:tasks.workunit.client.0.smithi032.stdout:9/445: creat d0/d11/d1d/f99 x:0 0 0 2024-06-03T01:21:01.733 INFO:tasks.workunit.client.0.smithi032.stdout:9/446: write d0/d23/d25/f62 [960023,50164] 0 2024-06-03T01:21:01.733 INFO:tasks.workunit.client.0.smithi032.stdout:1/244: read d18/d27/d2b/f3e [4051560,32579] 0 2024-06-03T01:21:01.734 INFO:tasks.workunit.client.1.smithi044.stdout:0/991: dwrite d3/d9/d55/f14c [4194304,4194304] 0 2024-06-03T01:21:01.737 INFO:tasks.workunit.client.0.smithi032.stdout:1/245: rename d18/d27/d2b/f3e to d18/d29/d3d/d47/f4c 0 2024-06-03T01:21:01.739 INFO:tasks.workunit.client.0.smithi032.stdout:1/246: symlink d18/d29/d3d/l4d 0 2024-06-03T01:21:01.747 INFO:tasks.workunit.client.1.smithi044.stdout:0/992: symlink d3/da6/d56/d62/l163 0 2024-06-03T01:21:01.747 INFO:tasks.workunit.client.1.smithi044.stdout:0/993: read - d3/da6/fb6 zero size 2024-06-03T01:21:01.747 INFO:tasks.workunit.client.1.smithi044.stdout:0/994: stat d3/d5c/fbd 0 2024-06-03T01:21:01.750 INFO:tasks.workunit.client.0.smithi032.stdout:6/380: dwrite d8/d11/d20/d2b/f4a [0,4194304] 0 2024-06-03T01:21:01.755 INFO:tasks.workunit.client.0.smithi032.stdout:6/381: dread d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:01.756 INFO:tasks.workunit.client.0.smithi032.stdout:6/382: truncate d8/d11/d20/d2b/d39/d19/f40 2964662 0 2024-06-03T01:21:01.759 INFO:tasks.workunit.client.0.smithi032.stdout:6/383: creat d8/d11/d20/d3b/d66/f76 x:0 0 0 2024-06-03T01:21:01.762 INFO:tasks.workunit.client.0.smithi032.stdout:1/247: dwrite f17 [0,4194304] 0 2024-06-03T01:21:01.765 INFO:tasks.workunit.client.0.smithi032.stdout:1/248: stat d18/l1d 0 2024-06-03T01:21:01.768 INFO:tasks.workunit.client.0.smithi032.stdout:9/447: dwrite d0/d11/d1d/f49 [0,4194304] 0 2024-06-03T01:21:01.778 INFO:tasks.workunit.client.0.smithi032.stdout:9/448: dread d0/f19 [0,4194304] 0 2024-06-03T01:21:01.780 INFO:tasks.workunit.client.0.smithi032.stdout:9/449: creat d0/f9a x:0 0 0 2024-06-03T01:21:01.780 INFO:tasks.workunit.client.0.smithi032.stdout:9/450: readlink d0/d23/d25/l2b 0 2024-06-03T01:21:01.789 INFO:tasks.workunit.client.0.smithi032.stdout:6/384: dwrite d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:21:01.790 INFO:tasks.workunit.client.0.smithi032.stdout:1/249: dwrite fa [0,4194304] 0 2024-06-03T01:21:01.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/969: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/l14d 0 2024-06-03T01:21:01.792 INFO:tasks.workunit.client.0.smithi032.stdout:1/250: symlink d18/d32/l4e 0 2024-06-03T01:21:01.794 INFO:tasks.workunit.client.1.smithi044.stdout:6/970: dread d2/d23/d30/d1e/d38/d6c/f84 [0,4194304] 0 2024-06-03T01:21:01.796 INFO:tasks.workunit.client.1.smithi044.stdout:6/971: symlink d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/dc7/l14e 0 2024-06-03T01:21:01.796 INFO:tasks.workunit.client.1.smithi044.stdout:6/972: fsync d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d64/f9e 0 2024-06-03T01:21:01.798 INFO:tasks.workunit.client.1.smithi044.stdout:6/973: link d2/d23/d55/f140 d2/d23/d30/d1e/f14f 0 2024-06-03T01:21:01.801 INFO:tasks.workunit.client.1.smithi044.stdout:6/974: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/f150 x:0 0 0 2024-06-03T01:21:01.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/975: truncate d2/d23/d55/d5f/db2/fec 3372783 0 2024-06-03T01:21:01.806 INFO:tasks.workunit.client.0.smithi032.stdout:9/451: dwrite d0/d11/d1d/d4a/d71/f82 [4194304,4194304] 0 2024-06-03T01:21:01.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/251: dread d18/d29/f45 [0,4194304] 0 2024-06-03T01:21:01.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/252: readlink d18/d29/d3d/d47/l38 0 2024-06-03T01:21:01.807 INFO:tasks.workunit.client.1.smithi044.stdout:6/976: unlink d2/d23/d30/d1e/d38/fa5 0 2024-06-03T01:21:01.809 INFO:tasks.workunit.client.1.smithi044.stdout:6/977: mkdir d2/d23/d55/d5f/db2/d3f/d106/d116/d151 0 2024-06-03T01:21:01.810 INFO:tasks.workunit.client.0.smithi032.stdout:1/253: rename cb to d18/d32/d39/c4f 0 2024-06-03T01:21:01.812 INFO:tasks.workunit.client.1.smithi044.stdout:6/978: symlink d2/d23/d30/dfe/d132/df3/l152 0 2024-06-03T01:21:01.812 INFO:tasks.workunit.client.1.smithi044.stdout:6/979: write d2/d23/d30/d1e/d38/d6c/fa3 [3028907,103424] 0 2024-06-03T01:21:01.812 INFO:tasks.workunit.client.1.smithi044.stdout:6/980: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/da7/df8/f153 x:0 0 0 2024-06-03T01:21:01.817 INFO:tasks.workunit.client.1.smithi044.stdout:6/981: symlink d2/d23/d55/d5f/db2/d3f/d14c/l154 0 2024-06-03T01:21:01.818 INFO:tasks.workunit.client.0.smithi032.stdout:9/452: write d0/d11/f48 [3468805,79297] 0 2024-06-03T01:21:01.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/982: dread d2/d23/d55/d5f/f59 [0,4194304] 0 2024-06-03T01:21:01.828 INFO:tasks.workunit.client.0.smithi032.stdout:1/254: dwrite d18/d32/d39/f4a [0,4194304] 0 2024-06-03T01:21:01.828 INFO:tasks.workunit.client.0.smithi032.stdout:1/255: chown d18/d32/d39 32062 1 2024-06-03T01:21:01.828 INFO:tasks.workunit.client.0.smithi032.stdout:1/256: chown d18/f34 14996 1 2024-06-03T01:21:01.829 INFO:tasks.workunit.client.1.smithi044.stdout:6/983: link d2/d23/d55/d5f/d99/l11c d2/d23/d44/d4d/d11d/l155 0 2024-06-03T01:21:01.829 INFO:tasks.workunit.client.0.smithi032.stdout:9/453: write d0/fa [392619,77267] 0 2024-06-03T01:21:01.829 INFO:tasks.workunit.client.0.smithi032.stdout:9/454: chown d0/d23 1641820 1 2024-06-03T01:21:01.831 INFO:tasks.workunit.client.1.smithi044.stdout:6/984: mknod d2/d23/d30/d1e/d38/d6c/dc2/d104/c156 0 2024-06-03T01:21:01.831 INFO:tasks.workunit.client.0.smithi032.stdout:9/455: mkdir d0/d23/d24/d26/d41/d9b 0 2024-06-03T01:21:01.831 INFO:tasks.workunit.client.0.smithi032.stdout:9/456: chown d0/d23/d25/f8d 15168 1 2024-06-03T01:21:01.833 INFO:tasks.workunit.client.0.smithi032.stdout:1/257: dread d18/d29/f45 [0,4194304] 0 2024-06-03T01:21:01.833 INFO:tasks.workunit.client.0.smithi032.stdout:1/258: write f17 [5915442,72398] 0 2024-06-03T01:21:01.833 INFO:tasks.workunit.client.0.smithi032.stdout:1/259: fdatasync d18/f34 0 2024-06-03T01:21:01.834 INFO:tasks.workunit.client.0.smithi032.stdout:1/260: write d18/d27/f31 [4226572,115267] 0 2024-06-03T01:21:01.835 INFO:tasks.workunit.client.0.smithi032.stdout:1/261: rmdir d18/d32/d3c 39 2024-06-03T01:21:01.837 INFO:tasks.workunit.client.0.smithi032.stdout:1/262: creat d18/d2d/f50 x:0 0 0 2024-06-03T01:21:01.838 INFO:tasks.workunit.client.0.smithi032.stdout:1/263: readlink l13 0 2024-06-03T01:21:01.840 INFO:tasks.workunit.client.1.smithi044.stdout:0/995: dwrite d3/d9/f44 [0,4194304] 0 2024-06-03T01:21:01.840 INFO:tasks.workunit.client.1.smithi044.stdout:0/996: write d3/da6/d20/fdc [2100086,89786] 0 2024-06-03T01:21:01.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/264: creat d18/d29/d3d/d47/f51 x:0 0 0 2024-06-03T01:21:01.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/265: readlink d18/d27/l3a 0 2024-06-03T01:21:01.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/266: stat d18/d27/l3a 0 2024-06-03T01:21:01.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/267: readlink d18/d32/d3c/l49 0 2024-06-03T01:21:01.845 INFO:tasks.workunit.client.0.smithi032.stdout:1/268: link f17 d18/d32/f52 0 2024-06-03T01:21:01.845 INFO:tasks.workunit.client.0.smithi032.stdout:1/269: stat d18/d3f 0 2024-06-03T01:21:01.845 INFO:tasks.workunit.client.1.smithi044.stdout:0/997: mkdir d3/da6/d56/d62/de7/d164 0 2024-06-03T01:21:01.849 INFO:tasks.workunit.client.1.smithi044.stdout:0/998: dread d3/d9/d102/d159/fb4 [0,4194304] 0 2024-06-03T01:21:01.853 INFO:tasks.workunit.client.0.smithi032.stdout:1/270: write f17 [3783688,66336] 0 2024-06-03T01:21:01.856 INFO:tasks.workunit.client.0.smithi032.stdout:1/271: unlink d18/d27/f31 0 2024-06-03T01:21:01.860 INFO:tasks.workunit.client.0.smithi032.stdout:1/272: unlink d18/f22 0 2024-06-03T01:21:01.862 INFO:tasks.workunit.client.0.smithi032.stdout:1/273: creat d18/d3f/f53 x:0 0 0 2024-06-03T01:21:01.862 INFO:tasks.workunit.client.0.smithi032.stdout:1/274: dread - d18/d29/d3d/d47/f51 zero size 2024-06-03T01:21:01.862 INFO:tasks.workunit.client.0.smithi032.stdout:1/275: chown d18/d32/c42 4049717 1 2024-06-03T01:21:01.863 INFO:tasks.workunit.client.1.smithi044.stdout:9/797: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d103 0 2024-06-03T01:21:01.863 INFO:tasks.workunit.client.1.smithi044.stdout:9/798: fdatasync d6/dd/f14 0 2024-06-03T01:21:01.863 INFO:tasks.workunit.client.1.smithi044.stdout:9/799: fsync d6/dd/d51/d16/d27/f64 0 2024-06-03T01:21:01.870 INFO:tasks.workunit.client.1.smithi044.stdout:8/878: rename d5/d55/d56/d5c/d68/de5/dea/d116 to d5/d55/d74/db4/d127 0 2024-06-03T01:21:01.872 INFO:tasks.workunit.client.1.smithi044.stdout:0/999: dwrite d3/da6/d21/d94/da3/dad/f158 [0,4194304] 0 2024-06-03T01:21:01.872 INFO:tasks.workunit.client.1.smithi044.stdout:2/727: rename db/d15/d49/d6c/d72/d7a/la0 to db/d15/d49/d6c/d72/d7a/d4e/d7e/le8 0 2024-06-03T01:21:01.872 INFO:tasks.workunit.client.1.smithi044.stdout:2/728: truncate db/d15/d49/d6c/d72/dcd/fcf 623730 0 2024-06-03T01:21:01.873 INFO:tasks.workunit.client.1.smithi044.stdout:9/800: dread d6/dd/d51/d16/f34 [0,4194304] 0 2024-06-03T01:21:01.873 INFO:tasks.workunit.client.0.smithi032.stdout:8/282: sync 2024-06-03T01:21:01.873 INFO:tasks.workunit.client.0.smithi032.stdout:8/283: creat dc/d25/d28/d32/f60 x:0 0 0 2024-06-03T01:21:01.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/879: symlink d5/d55/d74/db4/d127/l128 0 2024-06-03T01:21:01.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/880: chown d5/d16/d81/de3/d58/fa3 55034 1 2024-06-03T01:21:01.877 INFO:tasks.workunit.client.0.smithi032.stdout:8/284: link dc/d25/d28/d32/f50 dc/de/d1e/f61 0 2024-06-03T01:21:01.881 INFO:tasks.workunit.client.1.smithi044.stdout:7/869: rename d0/d3/d8/d32/d33/f57 to d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/f113 0 2024-06-03T01:21:01.881 INFO:tasks.workunit.client.1.smithi044.stdout:9/801: mknod d6/dd/d51/d46/c104 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:8/881: write d5/d55/d56/d5c/fae [2918666,19971] 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/985: rename d2/d23/d55/d5f/d85/c91 to d2/d23/d55/d5f/d99/dcc/de0/c157 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:2/729: dread db/f1f [4194304,4194304] 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/986: dread - d2/d23/d30/d1e/d38/d77/f12e zero size 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:2/730: creat db/d15/d49/d6c/d72/d8f/fe9 x:0 0 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:2/731: dread - db/fd4 zero size 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.1.smithi044.stdout:2/732: readlink l4 0 2024-06-03T01:21:01.886 INFO:tasks.workunit.client.0.smithi032.stdout:1/276: getdents d18 0 2024-06-03T01:21:01.887 INFO:tasks.workunit.client.0.smithi032.stdout:1/277: chown d18/d2d/f50 191 1 2024-06-03T01:21:01.892 INFO:tasks.workunit.client.1.smithi044.stdout:8/882: symlink d5/d16/d1d/d32/d109/l129 0 2024-06-03T01:21:01.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/987: link d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/d7d/f9d d2/d23/d44/d4d/d11d/f158 0 2024-06-03T01:21:01.896 INFO:tasks.workunit.client.1.smithi044.stdout:6/988: dread - d2/d23/df4/ff9 zero size 2024-06-03T01:21:01.897 INFO:tasks.workunit.client.1.smithi044.stdout:2/733: rmdir db/d1b/d2a/d2f/d4c 39 2024-06-03T01:21:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.897+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc18c1ab680 con 0x7fc174039400 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/802: rmdir d6/dd/d51/d16/d27/d5e/db7/de1 39 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/803: stat d6/dd/f3d 0 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/804: write d6/dd/d51/d16/d5f/fa8 [138581,111159] 0 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/805: stat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/c7d 0 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/806: chown d6/dd/d51/d16/d5f/d8b/ceb 126 1 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/807: fdatasync d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/fb1 0 2024-06-03T01:21:01.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/808: readlink d6/dd/d51/d16/d27/d5e/da0/l9a 0 2024-06-03T01:21:01.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.899+0000 7fc188ff9700 1 -- 172.21.15.32:0/2319973343 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc18c1ab680 con 0x7fc174039400 2024-06-03T01:21:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 msgr2=0x7fc17403b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 0x7fc17403b8c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009fd0 tx=0x7fc17c00b540 comp rx=0 tx=0).stop 2024-06-03T01:21:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 msgr2=0x7fc18c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b4d0 tx=0x7fc18000b890 comp rx=0 tx=0).stop 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 shutdown_connections 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fc174039400 0x7fc17403b8c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c0fef60 0x7fc18c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 --2- 172.21.15.32:0/2319973343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18c0ffcd0 0x7fc18c072330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 >> 172.21.15.32:0/2319973343 conn(0x7fc18c0fa8c0 msgr2=0x7fc18c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 shutdown_connections 2024-06-03T01:21:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:01.901+0000 7fc191e27700 1 -- 172.21.15.32:0/2319973343 wait complete. 2024-06-03T01:21:01.902 INFO:tasks.workunit.client.1.smithi044.stdout:8/883: symlink d5/d16/d81/de3/d98/ddd/l12a 0 2024-06-03T01:21:01.905 INFO:tasks.workunit.client.0.smithi032.stdout:1/278: dwrite d18/d2d/f50 [0,4194304] 0 2024-06-03T01:21:01.905 INFO:tasks.workunit.client.0.smithi032.stdout:1/279: chown d18/d32/d3c/l49 9 1 2024-06-03T01:21:01.905 INFO:tasks.workunit.client.1.smithi044.stdout:6/989: symlink d2/d23/l159 0 2024-06-03T01:21:01.905 INFO:tasks.workunit.client.1.smithi044.stdout:6/990: fsync d2/d23/d30/f45 0 2024-06-03T01:21:01.905 INFO:tasks.workunit.client.1.smithi044.stdout:6/991: read d2/d23/d55/d5f/db2/d3f/d106/d116/fce [948549,115997] 0 2024-06-03T01:21:01.906 INFO:tasks.workunit.client.0.smithi032.stdout:8/285: dwrite dc/d25/d28/d32/f50 [0,4194304] 0 2024-06-03T01:21:01.906 INFO:tasks.workunit.client.0.smithi032.stdout:8/286: getdents dc/de/d1e/d43/d4e/d59 0 2024-06-03T01:21:01.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/734: symlink db/lea 0 2024-06-03T01:21:01.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/735: read db/dd/f11 [3466701,46086] 0 2024-06-03T01:21:01.914 INFO:tasks.workunit.client.0.smithi032.stdout:1/280: creat d18/d27/d2b/f54 x:0 0 0 2024-06-03T01:21:01.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/809: mkdir d6/dd/d51/d16/d27/d105 0 2024-06-03T01:21:01.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/810: chown d6/caa 4 1 2024-06-03T01:21:01.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/992: creat d2/d23/d55/d5f/d99/dcc/f15a x:0 0 0 2024-06-03T01:21:01.916 INFO:tasks.workunit.client.1.smithi044.stdout:2/736: mkdir db/d1b/d2a/d52/d54/db6/deb 0 2024-06-03T01:21:01.917 INFO:tasks.workunit.client.1.smithi044.stdout:2/737: mkdir db/d1b/d5d/dec 0 2024-06-03T01:21:01.917 INFO:tasks.workunit.client.1.smithi044.stdout:2/738: chown db/d1b/d2a/d52/d54/c6e 122632 1 2024-06-03T01:21:01.919 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:21:01.921 INFO:tasks.workunit.client.1.smithi044.stdout:8/884: dwrite d5/d16/d81/de3/d58/f106 [0,4194304] 0 2024-06-03T01:21:01.922 INFO:tasks.workunit.client.1.smithi044.stdout:8/885: dread - d5/d55/d74/f78 zero size 2024-06-03T01:21:01.922 INFO:tasks.workunit.client.1.smithi044.stdout:8/886: chown d5/d55/d56/d5c/d68/de5/dea/d79/lb2 7 1 2024-06-03T01:21:01.924 INFO:tasks.workunit.client.0.smithi032.stdout:1/281: read d18/d2d/f50 [2873714,24095] 0 2024-06-03T01:21:01.927 INFO:tasks.workunit.client.0.smithi032.stdout:1/282: rename d18/d29/d3d/d47/f4c to d18/d3f/f55 0 2024-06-03T01:21:01.927 INFO:tasks.workunit.client.0.smithi032.stdout:1/283: truncate d18/d32/f4b 714532 0 2024-06-03T01:21:01.928 INFO:tasks.workunit.client.0.smithi032.stdout:1/284: symlink d18/d32/l56 0 2024-06-03T01:21:01.928 INFO:tasks.workunit.client.1.smithi044.stdout:8/887: mkdir d5/d20/d47/d64/dd0/d12b 0 2024-06-03T01:21:01.928 INFO:tasks.workunit.client.1.smithi044.stdout:9/811: dread d6/dd/d51/d16/f3f [0,4194304] 0 2024-06-03T01:21:01.929 INFO:tasks.workunit.client.1.smithi044.stdout:4/880: sync 2024-06-03T01:21:01.929 INFO:tasks.workunit.client.1.smithi044.stdout:5/809: sync 2024-06-03T01:21:01.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/735: sync 2024-06-03T01:21:01.929 INFO:tasks.workunit.client.1.smithi044.stdout:4/881: chown d17/d63/dae 251 1 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.1.smithi044.stdout:5/810: read d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [1688992,13174] 0 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/285: write d18/d32/f52 [9070720,109427] 0 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/286: fsync d18/f25 0 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/287: chown l4 2 1 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/288: creat d18/d27/d2b/f57 x:0 0 0 2024-06-03T01:21:01.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/289: dread - d18/d29/d3d/d47/f51 zero size 2024-06-03T01:21:01.931 INFO:tasks.workunit.client.0.smithi032.stdout:1/290: rmdir d18/d32/d3c 39 2024-06-03T01:21:01.931 INFO:tasks.workunit.client.1.smithi044.stdout:2/739: read db/d15/d49/d6c/d72/d7a/f68 [4140766,125652] 0 2024-06-03T01:21:01.931 INFO:tasks.workunit.client.1.smithi044.stdout:2/740: write db/d15/d49/d82/fac [850721,63549] 0 2024-06-03T01:21:01.931 INFO:tasks.workunit.client.1.smithi044.stdout:5/811: write d2/d11/d1e/d3b/da6/fd6 [427119,36309] 0 2024-06-03T01:21:01.932 INFO:tasks.workunit.client.0.smithi032.stdout:8/287: dwrite dc/de/d1e/f61 [0,4194304] 0 2024-06-03T01:21:01.932 INFO:tasks.workunit.client.0.smithi032.stdout:1/291: creat d18/d32/d39/f58 x:0 0 0 2024-06-03T01:21:01.933 INFO:tasks.workunit.client.0.smithi032.stdout:8/288: dread dc/f13 [0,4194304] 0 2024-06-03T01:21:01.933 INFO:tasks.workunit.client.0.smithi032.stdout:8/289: chown dc/d25 7626954 1 2024-06-03T01:21:01.933 INFO:tasks.workunit.client.0.smithi032.stdout:1/292: mknod d18/d32/d3c/c59 0 2024-06-03T01:21:01.934 INFO:tasks.workunit.client.1.smithi044.stdout:8/888: creat d5/d16/d81/de3/f12c x:0 0 0 2024-06-03T01:21:01.934 INFO:tasks.workunit.client.1.smithi044.stdout:8/889: dread - d5/d55/ff7 zero size 2024-06-03T01:21:01.934 INFO:tasks.workunit.client.1.smithi044.stdout:8/890: chown d5/d20/d47/d64/fa0 274 1 2024-06-03T01:21:01.936 INFO:tasks.workunit.client.0.smithi032.stdout:8/290: unlink dc/d14/d31/f38 0 2024-06-03T01:21:01.936 INFO:tasks.workunit.client.1.smithi044.stdout:9/812: link d6/dd/d51/fe d6/dd/d51/dce/f106 0 2024-06-03T01:21:01.936 INFO:tasks.workunit.client.1.smithi044.stdout:9/813: write d6/dd/f21 [5862618,2700] 0 2024-06-03T01:21:01.937 INFO:tasks.workunit.client.1.smithi044.stdout:3/736: rmdir df/d13/d1b/d4b/d69 39 2024-06-03T01:21:01.938 INFO:tasks.workunit.client.0.smithi032.stdout:1/293: rename d18/d32/d3c/c59 to d18/d29/c5a 0 2024-06-03T01:21:01.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/291: truncate f1 757687 0 2024-06-03T01:21:01.940 INFO:tasks.workunit.client.0.smithi032.stdout:1/294: mkdir d18/d27/d2b/d5b 0 2024-06-03T01:21:01.941 INFO:tasks.workunit.client.1.smithi044.stdout:4/882: getdents d17/d1c/d75/d7a/db8/d28/d2b/d8e 0 2024-06-03T01:21:01.941 INFO:tasks.workunit.client.0.smithi032.stdout:8/292: mknod dc/c62 0 2024-06-03T01:21:01.943 INFO:tasks.workunit.client.1.smithi044.stdout:2/741: mkdir db/d1b/d2a/d2f/dbe/ded 0 2024-06-03T01:21:01.943 INFO:tasks.workunit.client.1.smithi044.stdout:2/742: write db/fa2 [362454,51694] 0 2024-06-03T01:21:01.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/295: rename d18/d27/d2b/f57 to d18/d29/f5c 0 2024-06-03T01:21:01.946 INFO:tasks.workunit.client.0.smithi032.stdout:8/293: symlink dc/d25/d28/d33/l63 0 2024-06-03T01:21:01.946 INFO:tasks.workunit.client.0.smithi032.stdout:8/294: write dc/f15 [8194855,12557] 0 2024-06-03T01:21:01.948 INFO:tasks.workunit.client.1.smithi044.stdout:5/812: unlink d2/d11/d1e/d21/d24/d93/d9d/fb6 0 2024-06-03T01:21:01.949 INFO:tasks.workunit.client.0.smithi032.stdout:1/296: creat d18/f5d x:0 0 0 2024-06-03T01:21:01.949 INFO:tasks.workunit.client.0.smithi032.stdout:1/297: fdatasync fa 0 2024-06-03T01:21:01.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/993: dwrite d2/d23/d44/d4d/d11d/f158 [0,4194304] 0 2024-06-03T01:21:01.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/994: fsync d2/f134 0 2024-06-03T01:21:01.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/995: truncate d2/d23/d55/d5f/db2/d3f/d106/d116/da4/dd5/f148 916147 0 2024-06-03T01:21:01.953 INFO:tasks.workunit.client.0.smithi032.stdout:8/295: mkdir dc/d25/d64 0 2024-06-03T01:21:01.954 INFO:tasks.workunit.client.1.smithi044.stdout:4/883: write d17/d1c/d75/d7a/db3/f96 [7814268,125570] 0 2024-06-03T01:21:01.954 INFO:tasks.workunit.client.1.smithi044.stdout:9/814: rmdir d6/dd/d51/d16/d5f/d8b/d8d/dac 39 2024-06-03T01:21:01.956 INFO:tasks.workunit.client.0.smithi032.stdout:1/298: mkdir d18/d2d/d5e 0 2024-06-03T01:21:01.956 INFO:tasks.workunit.client.0.smithi032.stdout:1/299: chown lf 107784 1 2024-06-03T01:21:01.956 INFO:tasks.workunit.client.0.smithi032.stdout:1/300: read f17 [673586,31792] 0 2024-06-03T01:21:01.957 INFO:tasks.workunit.client.1.smithi044.stdout:3/737: symlink df/d1e/lf1 0 2024-06-03T01:21:01.957 INFO:tasks.workunit.client.1.smithi044.stdout:3/738: creat df/d13/d39/d3d/d6a/d9c/ff2 x:0 0 0 2024-06-03T01:21:01.959 INFO:tasks.workunit.client.0.smithi032.stdout:8/296: truncate f2 4094597 0 2024-06-03T01:21:01.961 INFO:tasks.workunit.client.1.smithi044.stdout:2/743: dwrite db/d1b/d2a/f2b [0,4194304] 0 2024-06-03T01:21:01.962 INFO:tasks.workunit.client.1.smithi044.stdout:3/739: read df/d13/d39/d3d/f44 [1056072,50348] 0 2024-06-03T01:21:01.963 INFO:tasks.workunit.client.0.smithi032.stdout:1/301: mkdir d18/d3f/d5f 0 2024-06-03T01:21:01.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/302: mkdir d18/d32/d39/d60 0 2024-06-03T01:21:01.966 INFO:tasks.workunit.client.0.smithi032.stdout:1/303: dread - d18/d32/d39/f58 zero size 2024-06-03T01:21:01.966 INFO:tasks.workunit.client.1.smithi044.stdout:6/996: creat d2/d23/d30/d1e/d38/d6c/d5c/d79/dfa/d39/dcd/f15b x:0 0 0 2024-06-03T01:21:01.966 INFO:tasks.workunit.client.0.smithi032.stdout:8/297: symlink dc/de/l65 0 2024-06-03T01:21:01.967 INFO:tasks.workunit.client.0.smithi032.stdout:8/298: write dc/d25/f55 [933415,29383] 0 2024-06-03T01:21:01.969 INFO:tasks.workunit.client.1.smithi044.stdout:4/884: mkdir d17/d1c/d75/d7a/db8/d128 0 2024-06-03T01:21:01.969 INFO:tasks.workunit.client.1.smithi044.stdout:4/885: fsync d17/d1c/d75/d7a/db8/d8d/ded/f11a 0 2024-06-03T01:21:01.970 INFO:tasks.workunit.client.0.smithi032.stdout:1/304: rename d18/d32/d39/f58 to d18/d32/d3c/f61 0 2024-06-03T01:21:01.970 INFO:tasks.workunit.client.0.smithi032.stdout:8/299: symlink dc/d25/d28/d33/l66 0 2024-06-03T01:21:01.971 INFO:tasks.workunit.client.1.smithi044.stdout:2/744: dread db/d1b/d3f/f66 [0,4194304] 0 2024-06-03T01:21:01.971 INFO:tasks.workunit.client.1.smithi044.stdout:2/745: creat db/fee x:0 0 0 2024-06-03T01:21:01.971 INFO:tasks.workunit.client.0.smithi032.stdout:1/305: truncate d18/d29/f45 3412591 0 2024-06-03T01:21:01.973 INFO:tasks.workunit.client.0.smithi032.stdout:1/306: mkdir d18/d32/d3c/d62 0 2024-06-03T01:21:01.975 INFO:tasks.workunit.client.1.smithi044.stdout:3/740: dread df/f58 [0,4194304] 0 2024-06-03T01:21:01.975 INFO:tasks.workunit.client.1.smithi044.stdout:3/741: write df/d1e/f45 [57115,15831] 0 2024-06-03T01:21:01.976 INFO:tasks.workunit.client.0.smithi032.stdout:1/307: mknod d18/d2d/c63 0 2024-06-03T01:21:01.976 INFO:tasks.workunit.client.0.smithi032.stdout:1/308: write d18/d29/d3d/d47/f51 [909308,36677] 0 2024-06-03T01:21:01.977 INFO:tasks.workunit.client.0.smithi032.stdout:1/309: creat d18/d2d/d5e/f64 x:0 0 0 2024-06-03T01:21:01.977 INFO:tasks.workunit.client.0.smithi032.stdout:1/310: write d18/d32/f52 [3639541,80736] 0 2024-06-03T01:21:01.979 INFO:tasks.workunit.client.1.smithi044.stdout:5/813: rename d2/d11/d1e/d21/d24/d93/d9d/d28/f7c to d2/f10f 0 2024-06-03T01:21:01.979 INFO:tasks.workunit.client.1.smithi044.stdout:5/814: dread - d2/d11/d1e/d21/fe6 zero size 2024-06-03T01:21:01.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/886: symlink d17/d1c/d75/d7a/db3/de9/l129 0 2024-06-03T01:21:01.984 INFO:tasks.workunit.client.1.smithi044.stdout:9/815: dwrite d6/dd/d51/d16/f37 [0,4194304] 0 2024-06-03T01:21:01.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/887: write d17/d1c/d75/d7a/db8/d28/f9d [1946975,72165] 0 2024-06-03T01:21:01.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/888: write d17/d1c/d75/d7a/db8/d28/f9d [1726460,6890] 0 2024-06-03T01:21:01.988 INFO:tasks.workunit.client.1.smithi044.stdout:6/997: rename d2/d23/d30/d1e/d38/d6c/fa3 to d2/d23/d55/d5f/d99/dcc/de0/f15c 0 2024-06-03T01:21:01.988 INFO:tasks.workunit.client.1.smithi044.stdout:6/998: write d2/d23/d55/d5f/f6e [2433022,129862] 0 2024-06-03T01:21:01.988 INFO:tasks.workunit.client.1.smithi044.stdout:6/999: read - d2/d23/d55/d5f/d99/f127 zero size 2024-06-03T01:21:01.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/889: creat d17/d1c/d7e/f12a x:0 0 0 2024-06-03T01:21:01.994 INFO:tasks.workunit.client.1.smithi044.stdout:9/816: truncate d6/dd/d51/dce/f106 3133320 0 2024-06-03T01:21:01.994 INFO:tasks.workunit.client.1.smithi044.stdout:9/817: write d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf [797038,104108] 0 2024-06-03T01:21:01.994 INFO:tasks.workunit.client.1.smithi044.stdout:9/818: creat d6/dd/d51/d16/d5f/d8b/f107 x:0 0 0 2024-06-03T01:21:01.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/890: creat d17/d1c/d70/d7d/f12b x:0 0 0 2024-06-03T01:21:01.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/891: chown d17/d1c/d75/d7a/db3/lba 951032629 1 2024-06-03T01:21:01.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/892: chown d17/d1c/d75/c64 487254220 1 2024-06-03T01:21:01.996 INFO:tasks.workunit.client.1.smithi044.stdout:2/746: dwrite db/fd4 [0,4194304] 0 2024-06-03T01:21:01.997 INFO:tasks.workunit.client.1.smithi044.stdout:4/893: creat d17/d1c/d7e/f12c x:0 0 0 2024-06-03T01:21:01.999 INFO:tasks.workunit.client.1.smithi044.stdout:9/819: rename d6/dd/d51/d46/f48 to d6/dd/f108 0 2024-06-03T01:21:02.000 INFO:tasks.workunit.client.0.smithi032.stdout:1/311: dread d18/d29/d3d/d47/f51 [0,4194304] 0 2024-06-03T01:21:02.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/312: mknod d18/d27/d2b/c65 0 2024-06-03T01:21:02.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/313: fdatasync d18/d32/d3c/f61 0 2024-06-03T01:21:02.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/314: write d18/d3f/f53 [153943,70870] 0 2024-06-03T01:21:02.003 INFO:tasks.workunit.client.0.smithi032.stdout:1/315: read - d18/d2d/d5e/f64 zero size 2024-06-03T01:21:02.003 INFO:tasks.workunit.client.0.smithi032.stdout:8/300: dwrite dc/d14/f2d [0,4194304] 0 2024-06-03T01:21:02.003 INFO:tasks.workunit.client.0.smithi032.stdout:1/316: rmdir d18/d32/d3c 39 2024-06-03T01:21:02.003 INFO:tasks.workunit.client.0.smithi032.stdout:1/317: chown d18/d29/d3d/d47 14692 1 2024-06-03T01:21:02.007 INFO:tasks.workunit.client.1.smithi044.stdout:3/742: dwrite df/d13/d1b/d93/fa7 [0,4194304] 0 2024-06-03T01:21:02.007 INFO:tasks.workunit.client.1.smithi044.stdout:3/743: creat df/d13/d39/d3d/d43/d71/ff3 x:0 0 0 2024-06-03T01:21:02.008 INFO:tasks.workunit.client.1.smithi044.stdout:5/815: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [0,4194304] 0 2024-06-03T01:21:02.008 INFO:tasks.workunit.client.1.smithi044.stdout:5/816: truncate d2/d11/d12/dad/f10e 188982 0 2024-06-03T01:21:02.008 INFO:tasks.workunit.client.1.smithi044.stdout:5/817: read - d2/d11/f100 zero size 2024-06-03T01:21:02.008 INFO:tasks.workunit.client.1.smithi044.stdout:5/818: stat d2/d11/d1e/d2f/fe5 0 2024-06-03T01:21:02.009 INFO:tasks.workunit.client.1.smithi044.stdout:4/894: unlink d17/d1c/d75/d7a/db8/d35/fb4 0 2024-06-03T01:21:02.012 INFO:tasks.workunit.client.0.smithi032.stdout:8/301: rename lb to dc/d25/d28/l67 0 2024-06-03T01:21:02.015 INFO:tasks.workunit.client.0.smithi032.stdout:1/318: mknod d18/d29/d3d/c66 0 2024-06-03T01:21:02.019 INFO:tasks.workunit.client.0.smithi032.stdout:4/319: sync 2024-06-03T01:21:02.020 INFO:tasks.workunit.client.0.smithi032.stdout:8/302: dread dc/d14/f22 [0,4194304] 0 2024-06-03T01:21:02.021 INFO:tasks.workunit.client.0.smithi032.stdout:4/320: creat d4/db/da/d37/f70 x:0 0 0 2024-06-03T01:21:02.022 INFO:tasks.workunit.client.0.smithi032.stdout:4/321: mknod d4/db/da/c71 0 2024-06-03T01:21:02.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/747: truncate db/d1b/d2a/f2b 1624504 0 2024-06-03T01:21:02.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/748: readlink db/d15/d49/d6c/d72/d7a/d4e/laf 0 2024-06-03T01:21:02.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/322: link d4/db/le d4/db/da/d2d/d27/d53/l72 0 2024-06-03T01:21:02.027 INFO:tasks.workunit.client.0.smithi032.stdout:1/319: dread f17 [0,4194304] 0 2024-06-03T01:21:02.031 INFO:tasks.workunit.client.0.smithi032.stdout:8/303: write dc/de/d1e/f61 [2236698,491] 0 2024-06-03T01:21:02.033 INFO:tasks.workunit.client.0.smithi032.stdout:1/320: creat d18/d32/d3c/f67 x:0 0 0 2024-06-03T01:21:02.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/304: write f8 [78925,69148] 0 2024-06-03T01:21:02.045 INFO:tasks.workunit.client.0.smithi032.stdout:4/323: dwrite d4/db/da/d2d/f55 [0,4194304] 0 2024-06-03T01:21:02.049 INFO:tasks.workunit.client.1.smithi044.stdout:5/819: stat d2/lc2 0 2024-06-03T01:21:02.051 INFO:tasks.workunit.client.1.smithi044.stdout:4/895: creat d17/d1c/d9c/f12d x:0 0 0 2024-06-03T01:21:02.055 INFO:tasks.workunit.client.0.smithi032.stdout:1/321: dwrite d18/d32/f52 [8388608,4194304] 0 2024-06-03T01:21:02.055 INFO:tasks.workunit.client.0.smithi032.stdout:1/322: write d18/d2d/d5e/f64 [206467,32351] 0 2024-06-03T01:21:02.055 INFO:tasks.workunit.client.0.smithi032.stdout:8/305: dwrite dc/d25/d28/d3c/f3f [0,4194304] 0 2024-06-03T01:21:02.056 INFO:tasks.workunit.client.1.smithi044.stdout:9/820: rmdir d6/dd/d51/db/dd5 39 2024-06-03T01:21:02.056 INFO:tasks.workunit.client.1.smithi044.stdout:9/821: fdatasync d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68/f70 0 2024-06-03T01:21:02.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/324: dwrite d4/f41 [0,4194304] 0 2024-06-03T01:21:02.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/325: creat d4/d57/d6c/f73 x:0 0 0 2024-06-03T01:21:02.064 INFO:tasks.workunit.client.0.smithi032.stdout:1/323: getdents d18/d32/d39 0 2024-06-03T01:21:02.065 INFO:tasks.workunit.client.1.smithi044.stdout:5/820: link d2/d11/d1e/d2f/c9c d2/d11/d1e/d21/d24/d93/d9d/c110 0 2024-06-03T01:21:02.067 INFO:tasks.workunit.client.1.smithi044.stdout:4/896: rmdir d17/d1c/de0 39 2024-06-03T01:21:02.067 INFO:tasks.workunit.client.1.smithi044.stdout:4/897: stat f2 0 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.1.smithi044.stdout:4/898: truncate d17/d63/dae/f126 1001665 0 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.1.smithi044.stdout:9/822: dread d6/dd/d51/fe [0,4194304] 0 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.1.smithi044.stdout:9/823: readlink d6/dd/d51/d16/d27/d5e/l6f 0 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.1.smithi044.stdout:9/824: dread - d6/dd/d51/d16/d27/d5e/db7/de1/fdb zero size 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.0.smithi032.stdout:4/326: rename d4/db/da/d2d/c63 to d4/d57/c74 0 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.0.smithi032.stdout:4/327: chown d4/db/da/c12 1081601 1 2024-06-03T01:21:02.068 INFO:tasks.workunit.client.0.smithi032.stdout:8/306: unlink dc/de/c1b 0 2024-06-03T01:21:02.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/324: getdents d18/d27 0 2024-06-03T01:21:02.072 INFO:tasks.workunit.client.1.smithi044.stdout:3/744: rename df/d13/d39/d3d/d43/d72 to df/d13/d1b/d56/d5c/d98/df4 0 2024-06-03T01:21:02.074 INFO:tasks.workunit.client.1.smithi044.stdout:3/745: dread df/d13/d39/d3d/f68 [0,4194304] 0 2024-06-03T01:21:02.075 INFO:tasks.workunit.client.1.smithi044.stdout:4/899: unlink d17/d1c/d7e/lc5 0 2024-06-03T01:21:02.080 INFO:tasks.workunit.client.1.smithi044.stdout:3/746: write df/d13/f16 [122401,19529] 0 2024-06-03T01:21:02.080 INFO:tasks.workunit.client.1.smithi044.stdout:3/747: chown df/d1e/d2a/d70/fd2 300 1 2024-06-03T01:21:02.080 INFO:tasks.workunit.client.1.smithi044.stdout:3/748: stat df/d1e/d2a/d70/fbe 0 2024-06-03T01:21:02.081 INFO:tasks.workunit.client.1.smithi044.stdout:2/749: write db/d1b/d27/f50 [884254,118776] 0 2024-06-03T01:21:02.081 INFO:tasks.workunit.client.1.smithi044.stdout:3/749: dread df/d13/d6d/d9a/dad/fd6 [0,4194304] 0 2024-06-03T01:21:02.082 INFO:tasks.workunit.client.1.smithi044.stdout:5/821: rename d2/d11/d1e/d21/l3f to d2/d11/d3e/dce/l111 0 2024-06-03T01:21:02.083 INFO:tasks.workunit.client.0.smithi032.stdout:4/328: write d4/db/f36 [2022427,83164] 0 2024-06-03T01:21:02.083 INFO:tasks.workunit.client.1.smithi044.stdout:3/750: read df/d13/d1b/f5e [501630,115003] 0 2024-06-03T01:21:02.085 INFO:tasks.workunit.client.0.smithi032.stdout:4/329: creat d4/db/da/d1d/f75 x:0 0 0 2024-06-03T01:21:02.086 INFO:tasks.workunit.client.0.smithi032.stdout:4/330: chown d4/db/da/d2d/c52 46 1 2024-06-03T01:21:02.086 INFO:tasks.workunit.client.1.smithi044.stdout:3/751: mkdir df/d13/d6d/d9a/dad/df5 0 2024-06-03T01:21:02.087 INFO:tasks.workunit.client.1.smithi044.stdout:5/822: creat d2/d11/d12/d2b/d4a/d57/d106/f112 x:0 0 0 2024-06-03T01:21:02.088 INFO:tasks.workunit.client.0.smithi032.stdout:4/331: link d4/l6d d4/db/da/d37/l76 0 2024-06-03T01:21:02.089 INFO:tasks.workunit.client.1.smithi044.stdout:3/752: mknod df/d13/d39/d3d/d6a/d9c/cf6 0 2024-06-03T01:21:02.089 INFO:tasks.workunit.client.1.smithi044.stdout:3/753: write df/d13/d1b/f34 [865947,20283] 0 2024-06-03T01:21:02.089 INFO:tasks.workunit.client.1.smithi044.stdout:3/754: chown df/d1e/d3a/d84/lab 1998393 1 2024-06-03T01:21:02.090 INFO:tasks.workunit.client.1.smithi044.stdout:5/823: creat d2/d11/d12/d2b/d4a/d57/f113 x:0 0 0 2024-06-03T01:21:02.090 INFO:tasks.workunit.client.1.smithi044.stdout:5/824: write d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f7e [1335657,91237] 0 2024-06-03T01:21:02.090 INFO:tasks.workunit.client.1.smithi044.stdout:9/825: dwrite d6/dd/d51/d46/fb0 [4194304,4194304] 0 2024-06-03T01:21:02.090 INFO:tasks.workunit.client.0.smithi032.stdout:4/332: symlink d4/db/da/l77 0 2024-06-03T01:21:02.091 INFO:tasks.workunit.client.0.smithi032.stdout:4/333: chown d4/db/da/c12 3304 1 2024-06-03T01:21:02.091 INFO:tasks.workunit.client.1.smithi044.stdout:9/826: dread d6/dd/d51/d16/f5a [0,4194304] 0 2024-06-03T01:21:02.095 INFO:tasks.workunit.client.0.smithi032.stdout:1/325: dread d18/f25 [4194304,4194304] 0 2024-06-03T01:21:02.100 INFO:tasks.workunit.client.1.smithi044.stdout:8/891: sync 2024-06-03T01:21:02.100 INFO:tasks.workunit.client.1.smithi044.stdout:7/870: sync 2024-06-03T01:21:02.102 INFO:tasks.workunit.client.0.smithi032.stdout:4/334: write d4/db/da/d1d/f33 [2764483,105934] 0 2024-06-03T01:21:02.102 INFO:tasks.workunit.client.0.smithi032.stdout:4/335: dread - d4/db/da/d37/f70 zero size 2024-06-03T01:21:02.102 INFO:tasks.workunit.client.1.smithi044.stdout:7/871: write d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/f113 [1279377,8856] 0 2024-06-03T01:21:02.103 INFO:tasks.workunit.client.1.smithi044.stdout:4/900: dwrite d17/d1c/d75/d7a/db8/d59/f6d [0,4194304] 0 2024-06-03T01:21:02.103 INFO:tasks.workunit.client.1.smithi044.stdout:4/901: fsync d17/d1c/d75/d7a/db8/d28/d2b/d8e/fc9 0 2024-06-03T01:21:02.103 INFO:tasks.workunit.client.1.smithi044.stdout:4/902: readlink d17/d1c/d75/d7a/db8/d6a/l88 0 2024-06-03T01:21:02.103 INFO:tasks.workunit.client.0.smithi032.stdout:4/336: mkdir d4/d78 0 2024-06-03T01:21:02.104 INFO:tasks.workunit.client.1.smithi044.stdout:9/827: read d6/dd/d51/d16/d27/f39 [3529657,32686] 0 2024-06-03T01:21:02.107 INFO:tasks.workunit.client.0.smithi032.stdout:4/337: symlink d4/db/da/d2d/d27/d53/l79 0 2024-06-03T01:21:02.109 INFO:tasks.workunit.client.0.smithi032.stdout:4/338: link d4/db/da/d37/l47 d4/db/da/d5d/l7a 0 2024-06-03T01:21:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 -- 172.21.15.32:0/557598543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e01011b0 msgr2=0x7f20e01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 --2- 172.21.15.32:0/557598543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e01011b0 0x7f20e01015d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009910 tx=0x7f20d8009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 -- 172.21.15.32:0/557598543 shutdown_connections 2024-06-03T01:21:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 --2- 172.21.15.32:0/557598543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00ffe10 0x7f20e0100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 --2- 172.21.15.32:0/557598543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e01011b0 0x7f20e01015d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 -- 172.21.15.32:0/557598543 >> 172.21.15.32:0/557598543 conn(0x7f20e00fa8c0 msgr2=0x7f20e00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 -- 172.21.15.32:0/557598543 shutdown_connections 2024-06-03T01:21:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.110+0000 7f20e73f5700 1 -- 172.21.15.32:0/557598543 wait complete. 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 Processor -- start 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 -- start start 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e01011b0 0x7f20e0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0199270 con 0x7f20e00ffe10 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.111+0000 7f20e73f5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e01993b0 con 0x7f20e01011b0 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49076/0 (socket says 172.21.15.32:49076) 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 -- 172.21.15.32:0/2540350477 learned_addr learned my addr 172.21.15.32:0/2540350477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e4990700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e01011b0 0x7f20e0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/750: dwrite db/fd4 [0,4194304] 0 2024-06-03T01:21:02.113 INFO:tasks.workunit.client.1.smithi044.stdout:7/872: write d0/f74 [3533414,48150] 0 2024-06-03T01:21:02.114 INFO:tasks.workunit.client.1.smithi044.stdout:7/873: fsync d0/d3/d8/d95/d63/d7f/f91 0 2024-06-03T01:21:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 -- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e01011b0 msgr2=0x7f20e0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e01011b0 0x7f20e0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.112+0000 7f20e5191700 1 -- 172.21.15.32:0/2540350477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80095f0 con 0x7f20e00ffe10 2024-06-03T01:21:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.113+0000 7f20e5191700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.113+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d801d070 con 0x7f20e00ffe10 2024-06-03T01:21:02.118 INFO:tasks.workunit.client.0.smithi032.stdout:1/326: dwrite d18/d32/d39/f4a [4194304,4194304] 0 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.114+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f20d800fe90 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.114+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e019de00 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.114+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e019e260 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.114+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d8017630 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.115+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4005320 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.116+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) v1 ==== 46426+0+0 (secure 0 0 0) 0x7f20d80050d0 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.117+0000 7f20d67fc700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 0x7f20cc03b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.117+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f20d804e9c0 con 0x7f20e00ffe10 2024-06-03T01:21:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.117+0000 7f20e4990700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 0x7f20cc03b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.120 INFO:tasks.workunit.client.1.smithi044.stdout:5/825: creat d2/d11/d3e/dce/dd1/f114 x:0 0 0 2024-06-03T01:21:02.120 INFO:tasks.workunit.client.1.smithi044.stdout:3/755: getdents df/d1e/d2a/d70 0 2024-06-03T01:21:02.120 INFO:tasks.workunit.client.1.smithi044.stdout:3/756: write df/d1e/d2a/d70/fd2 [65914,118287] 0 2024-06-03T01:21:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.120+0000 7f20e4990700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 0x7f20cc03b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bf0 tx=0x7f20d0009260 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:02.123 INFO:tasks.workunit.client.1.smithi044.stdout:4/903: mkdir d17/d12e 0 2024-06-03T01:21:02.123 INFO:tasks.workunit.client.1.smithi044.stdout:4/904: chown d17/d1c/d75/d7a/db8/d59/ca7 1004 1 2024-06-03T01:21:02.125 INFO:tasks.workunit.client.1.smithi044.stdout:7/874: dread d0/d3/d8/d32/f38 [0,4194304] 0 2024-06-03T01:21:02.125 INFO:tasks.workunit.client.1.smithi044.stdout:7/875: chown d0/d3/d8/d95/d63/d89/fa5 17 1 2024-06-03T01:21:02.126 INFO:tasks.workunit.client.1.smithi044.stdout:9/828: link d6/dd/d51/c4c d6/dd/d51/d16/c109 0 2024-06-03T01:21:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.126+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f20d80167a0 con 0x7f20e00ffe10 2024-06-03T01:21:02.130 INFO:tasks.workunit.client.1.smithi044.stdout:2/751: link db/d1b/d2a/d2f/d4c/c4d db/d15/d49/d6c/cef 0 2024-06-03T01:21:02.132 INFO:tasks.workunit.client.0.smithi032.stdout:4/339: dwrite d4/db/f68 [0,4194304] 0 2024-06-03T01:21:02.136 INFO:tasks.workunit.client.0.smithi032.stdout:1/327: dwrite d18/d29/d3d/d47/f51 [0,4194304] 0 2024-06-03T01:21:02.136 INFO:tasks.workunit.client.0.smithi032.stdout:1/328: dread - d18/d27/d2b/f54 zero size 2024-06-03T01:21:02.139 INFO:tasks.workunit.client.1.smithi044.stdout:5/826: symlink d2/d11/d1e/d21/d24/d9e/l115 0 2024-06-03T01:21:02.146 INFO:tasks.workunit.client.0.smithi032.stdout:4/340: read d4/f41 [2366818,14832] 0 2024-06-03T01:21:02.146 INFO:tasks.workunit.client.1.smithi044.stdout:4/905: read d17/d1c/f3a [7081532,85528] 0 2024-06-03T01:21:02.148 INFO:tasks.workunit.client.1.smithi044.stdout:7/876: mknod d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/c114 0 2024-06-03T01:21:02.150 INFO:tasks.workunit.client.1.smithi044.stdout:9/829: creat d6/dd/d51/d16/d27/d5e/d71/de0/f10a x:0 0 0 2024-06-03T01:21:02.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/752: dwrite db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:21:02.151 INFO:tasks.workunit.client.1.smithi044.stdout:5/827: mkdir d2/d11/d1e/d3b/da6/d116 0 2024-06-03T01:21:02.152 INFO:tasks.workunit.client.1.smithi044.stdout:5/828: dread - d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f61 zero size 2024-06-03T01:21:02.154 INFO:tasks.workunit.client.1.smithi044.stdout:7/877: creat d0/d3/d8/d95/d63/d7f/d9c/da4/f115 x:0 0 0 2024-06-03T01:21:02.156 INFO:tasks.workunit.client.1.smithi044.stdout:9/830: creat d6/dd/d51/d16/dee/f10b x:0 0 0 2024-06-03T01:21:02.158 INFO:tasks.workunit.client.1.smithi044.stdout:2/753: mkdir db/d1b/d2a/d52/df0 0 2024-06-03T01:21:02.158 INFO:tasks.workunit.client.1.smithi044.stdout:2/754: dread - db/d1b/d5d/fb1 zero size 2024-06-03T01:21:02.158 INFO:tasks.workunit.client.1.smithi044.stdout:2/755: dread - db/d1b/d2a/d52/fdc zero size 2024-06-03T01:21:02.159 INFO:tasks.workunit.client.1.smithi044.stdout:5/829: mkdir d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117 0 2024-06-03T01:21:02.160 INFO:tasks.workunit.client.1.smithi044.stdout:7/878: symlink d0/d3/d8/l116 0 2024-06-03T01:21:02.162 INFO:tasks.workunit.client.1.smithi044.stdout:9/831: link d6/dd/d51/d16/d27/d5e/db7/dd6/cf3 d6/dd/d51/d16/d27/d5e/c10c 0 2024-06-03T01:21:02.162 INFO:tasks.workunit.client.1.smithi044.stdout:9/832: write d6/dd/d51/d16/d27/d5e/db7/de1/fc9 [612730,93390] 0 2024-06-03T01:21:02.162 INFO:tasks.workunit.client.0.smithi032.stdout:4/341: dread d4/db/da/f2f [0,4194304] 0 2024-06-03T01:21:02.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/756: dread db/dd/f60 [0,4194304] 0 2024-06-03T01:21:02.166 INFO:tasks.workunit.client.1.smithi044.stdout:3/757: dwrite df/d13/f29 [0,4194304] 0 2024-06-03T01:21:02.176 INFO:tasks.workunit.client.1.smithi044.stdout:2/757: write db/d1b/d3f/f4a [3447999,4264] 0 2024-06-03T01:21:02.176 INFO:tasks.workunit.client.1.smithi044.stdout:2/758: chown db/d1b/d2a/lc7 18631 1 2024-06-03T01:21:02.178 INFO:tasks.workunit.client.1.smithi044.stdout:4/906: dwrite d17/d1c/d75/d7a/db8/fa5 [0,4194304] 0 2024-06-03T01:21:02.180 INFO:tasks.workunit.client.1.smithi044.stdout:4/907: link d17/d1c/d70/d7d/l83 d17/d63/d10c/l12f 0 2024-06-03T01:21:02.187 INFO:tasks.workunit.client.1.smithi044.stdout:4/908: dread d17/d1c/d75/d7a/db8/d35/f41 [0,4194304] 0 2024-06-03T01:21:02.187 INFO:tasks.workunit.client.1.smithi044.stdout:4/909: rmdir d17/d1c/d70/dfb 39 2024-06-03T01:21:02.189 INFO:tasks.workunit.client.1.smithi044.stdout:4/910: mknod d17/d1c/d75/d7a/db8/d28/d2b/c130 0 2024-06-03T01:21:02.190 INFO:tasks.workunit.client.1.smithi044.stdout:4/911: mknod d17/d63/c131 0 2024-06-03T01:21:02.190 INFO:tasks.workunit.client.1.smithi044.stdout:2/759: dread db/d15/d49/d7b/f88 [0,4194304] 0 2024-06-03T01:21:02.191 INFO:tasks.workunit.client.1.smithi044.stdout:4/912: creat d17/d1c/d75/d7a/f132 x:0 0 0 2024-06-03T01:21:02.192 INFO:tasks.workunit.client.1.smithi044.stdout:5/830: dwrite d2/d11/d1e/f95 [0,4194304] 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:7/879: dwrite d0/d16/d23/fd2 [0,4194304] 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:5/831: creat d2/d11/d1e/d3b/da6/f118 x:0 0 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:7/880: stat d0/d3/d8/d95/d63/d89/da3 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:7/881: fsync d0/d3/d8/d95/d63/d89/da3/f5e 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:5/832: rename d2/d11/d1e/d21/d24/d93 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/d119 22 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:5/833: stat d2/d11 0 2024-06-03T01:21:02.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/342: dwrite d4/db/da/d1d/f75 [0,4194304] 0 2024-06-03T01:21:02.200 INFO:tasks.workunit.client.1.smithi044.stdout:7/882: mknod d0/d16/c117 0 2024-06-03T01:21:02.201 INFO:tasks.workunit.client.1.smithi044.stdout:4/913: write d17/d1c/d75/d7a/db8/d6a/f9a [2073081,47008] 0 2024-06-03T01:21:02.203 INFO:tasks.workunit.client.0.smithi032.stdout:4/343: truncate d4/db/f36 3977233 0 2024-06-03T01:21:02.203 INFO:tasks.workunit.client.0.smithi032.stdout:4/344: write d4/d57/d6c/f6e [519041,82231] 0 2024-06-03T01:21:02.204 INFO:tasks.workunit.client.1.smithi044.stdout:5/834: write d2/d11/d1e/f95 [822495,40420] 0 2024-06-03T01:21:02.205 INFO:tasks.workunit.client.0.smithi032.stdout:4/345: unlink d4/db/da/d37/f3e 0 2024-06-03T01:21:02.205 INFO:tasks.workunit.client.0.smithi032.stdout:4/346: chown d4/db/da/d1d 94 1 2024-06-03T01:21:02.205 INFO:tasks.workunit.client.0.smithi032.stdout:4/347: fdatasync d4/db/da/d1d/f1f 0 2024-06-03T01:21:02.205 INFO:tasks.workunit.client.0.smithi032.stdout:4/348: write d4/db/da/d2d/d27/f60 [432833,75030] 0 2024-06-03T01:21:02.205 INFO:tasks.workunit.client.0.smithi032.stdout:4/349: creat d4/db/da/d37/f7b x:0 0 0 2024-06-03T01:21:02.208 INFO:tasks.workunit.client.0.smithi032.stdout:4/350: creat d4/db/da/d1d/d6b/f7c x:0 0 0 2024-06-03T01:21:02.212 INFO:tasks.workunit.client.0.smithi032.stdout:4/351: rename d4/f3a to d4/db/da/f7d 0 2024-06-03T01:21:02.213 INFO:tasks.workunit.client.1.smithi044.stdout:2/760: dwrite db/d1b/d2a/f2b [0,4194304] 0 2024-06-03T01:21:02.214 INFO:tasks.workunit.client.1.smithi044.stdout:4/914: dread d17/d1c/d75/d7a/db8/d59/f82 [0,4194304] 0 2024-06-03T01:21:02.214 INFO:tasks.workunit.client.0.smithi032.stdout:4/352: creat d4/db/da/d1d/f7e x:0 0 0 2024-06-03T01:21:02.214 INFO:tasks.workunit.client.0.smithi032.stdout:4/353: dread - d4/db/da/f6f zero size 2024-06-03T01:21:02.214 INFO:tasks.workunit.client.0.smithi032.stdout:4/354: dread - d4/db/da/d1d/f7e zero size 2024-06-03T01:21:02.214 INFO:tasks.workunit.client.0.smithi032.stdout:4/355: fsync d4/db/da/d1d/f23 0 2024-06-03T01:21:02.215 INFO:tasks.workunit.client.0.smithi032.stdout:7/342: sync 2024-06-03T01:21:02.216 INFO:tasks.workunit.client.0.smithi032.stdout:4/356: mkdir d4/d7f 0 2024-06-03T01:21:02.216 INFO:tasks.workunit.client.0.smithi032.stdout:4/357: creat d4/db/da/f80 x:0 0 0 2024-06-03T01:21:02.217 INFO:tasks.workunit.client.1.smithi044.stdout:4/915: link d17/d1c/f52 d17/d1c/d9c/f133 0 2024-06-03T01:21:02.219 INFO:tasks.workunit.client.0.smithi032.stdout:7/343: rename dc/d12/d22/d42/d4e to dc/d12/d2a/d2f/d63/d69 0 2024-06-03T01:21:02.220 INFO:tasks.workunit.client.1.smithi044.stdout:4/916: mknod d17/d1c/d75/d7a/db8/c134 0 2024-06-03T01:21:02.220 INFO:tasks.workunit.client.1.smithi044.stdout:4/917: readlink d17/d1c/d75/d7a/db3/de9/l129 0 2024-06-03T01:21:02.221 INFO:tasks.workunit.client.1.smithi044.stdout:4/918: readlink d17/d1c/d70/l7f 0 2024-06-03T01:21:02.222 INFO:tasks.workunit.client.1.smithi044.stdout:4/919: mkdir d17/d1c/d70/d7d/d135 0 2024-06-03T01:21:02.223 INFO:tasks.workunit.client.0.smithi032.stdout:4/358: symlink d4/d78/l81 0 2024-06-03T01:21:02.223 INFO:tasks.workunit.client.1.smithi044.stdout:4/920: symlink d17/d63/d10c/l136 0 2024-06-03T01:21:02.225 INFO:tasks.workunit.client.1.smithi044.stdout:5/835: dwrite d2/d11/d12/f37 [4194304,4194304] 0 2024-06-03T01:21:02.225 INFO:tasks.workunit.client.1.smithi044.stdout:5/836: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f11a x:0 0 0 2024-06-03T01:21:02.227 INFO:tasks.workunit.client.0.smithi032.stdout:7/344: unlink dc/dd/f11 0 2024-06-03T01:21:02.231 INFO:tasks.workunit.client.0.smithi032.stdout:4/359: link d4/db/da/f15 d4/db/da/f82 0 2024-06-03T01:21:02.232 INFO:tasks.workunit.client.1.smithi044.stdout:4/921: dread d17/d1c/d75/d7a/db8/d28/d2b/d8e/fc9 [0,4194304] 0 2024-06-03T01:21:02.233 INFO:tasks.workunit.client.1.smithi044.stdout:2/761: dwrite db/d15/d49/d6c/f8c [0,4194304] 0 2024-06-03T01:21:02.234 INFO:tasks.workunit.client.1.smithi044.stdout:4/922: creat d17/d1c/d75/d7a/db8/d35/d124/f137 x:0 0 0 2024-06-03T01:21:02.235 INFO:tasks.workunit.client.0.smithi032.stdout:4/360: symlink d4/db/da/d2d/d27/d50/l83 0 2024-06-03T01:21:02.238 INFO:tasks.workunit.client.1.smithi044.stdout:2/762: mknod db/d1b/d2a/d52/d54/db6/cf1 0 2024-06-03T01:21:02.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/923: dread d17/d1c/d70/fe4 [0,4194304] 0 2024-06-03T01:21:02.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/924: chown d17/d1c/d9c 740156 1 2024-06-03T01:21:02.241 INFO:tasks.workunit.client.1.smithi044.stdout:2/763: rename db/d15/d49/d6c/d72/d7a/d4e/l7f to db/d15/d49/d6c/d72/d7a/d4e/lf2 0 2024-06-03T01:21:02.246 INFO:tasks.workunit.client.1.smithi044.stdout:4/925: unlink d17/d1c/d75/d7a/db8/d6a/f73 0 2024-06-03T01:21:02.246 INFO:tasks.workunit.client.1.smithi044.stdout:5/837: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/f74 [0,4194304] 0 2024-06-03T01:21:02.248 INFO:tasks.workunit.client.1.smithi044.stdout:8/892: sync 2024-06-03T01:21:02.253 INFO:tasks.workunit.client.1.smithi044.stdout:2/764: unlink db/d15/d49/d6c/d72/d7a/f81 0 2024-06-03T01:21:02.253 INFO:tasks.workunit.client.1.smithi044.stdout:2/765: read - db/d15/d49/d6c/d72/d8f/fe9 zero size 2024-06-03T01:21:02.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/766: mkdir db/d15/d49/d6c/df3 0 2024-06-03T01:21:02.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/767: chown db/d1b/d27/d79 31398948 1 2024-06-03T01:21:02.256 INFO:tasks.workunit.client.1.smithi044.stdout:2/768: write db/d1b/d3f/f66 [4084737,66308] 0 2024-06-03T01:21:02.256 INFO:tasks.workunit.client.0.smithi032.stdout:7/345: dwrite dc/d12/d24/d33/f4b [0,4194304] 0 2024-06-03T01:21:02.260 INFO:tasks.workunit.client.0.smithi032.stdout:7/346: mkdir dc/d12/d6a 0 2024-06-03T01:21:02.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/361: dwrite d4/db/da/d2d/d27/f2b [0,4194304] 0 2024-06-03T01:21:02.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/362: dread - d4/db/f2e zero size 2024-06-03T01:21:02.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/363: read - d4/db/da/f3d zero size 2024-06-03T01:21:02.264 INFO:tasks.workunit.client.0.smithi032.stdout:4/364: dread d4/f29 [0,4194304] 0 2024-06-03T01:21:02.264 INFO:tasks.workunit.client.0.smithi032.stdout:4/365: creat d4/db/da/d37/f84 x:0 0 0 2024-06-03T01:21:02.286 INFO:tasks.workunit.client.1.smithi044.stdout:4/926: dwrite d17/d1c/d75/d7a/db8/d28/fd1 [0,4194304] 0 2024-06-03T01:21:02.286 INFO:tasks.workunit.client.1.smithi044.stdout:5/838: dwrite d2/d11/d1e/d21/d24/d9e/dbb/ff7 [0,4194304] 0 2024-06-03T01:21:02.287 INFO:tasks.workunit.client.1.smithi044.stdout:8/893: dwrite d5/d16/d1d/f31 [0,4194304] 0 2024-06-03T01:21:02.287 INFO:tasks.workunit.client.0.smithi032.stdout:4/366: dwrite d4/db/da/f80 [0,4194304] 0 2024-06-03T01:21:02.291 INFO:tasks.workunit.client.1.smithi044.stdout:4/927: mkdir d17/d1c/d70/d7d/d138 0 2024-06-03T01:21:02.292 INFO:tasks.workunit.client.0.smithi032.stdout:4/367: symlink d4/d57/l85 0 2024-06-03T01:21:02.292 INFO:tasks.workunit.client.0.smithi032.stdout:4/368: chown d4/db/le 384 1 2024-06-03T01:21:02.292 INFO:tasks.workunit.client.1.smithi044.stdout:5/839: truncate d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/fc1 376383 0 2024-06-03T01:21:02.296 INFO:tasks.workunit.client.0.smithi032.stdout:4/369: rename d4/db/da/f6f to d4/d5c/f86 0 2024-06-03T01:21:02.302 INFO:tasks.workunit.client.1.smithi044.stdout:8/894: rmdir d5/dd4/df9/d117 0 2024-06-03T01:21:02.302 INFO:tasks.workunit.client.0.smithi032.stdout:7/347: getdents dc/d12 0 2024-06-03T01:21:02.303 INFO:tasks.workunit.client.1.smithi044.stdout:4/928: unlink l10 0 2024-06-03T01:21:02.303 INFO:tasks.workunit.client.1.smithi044.stdout:4/929: chown d17/d63/dae/fbf 1 1 2024-06-03T01:21:02.304 INFO:tasks.workunit.client.0.smithi032.stdout:7/348: dread dc/f3d [0,4194304] 0 2024-06-03T01:21:02.304 INFO:tasks.workunit.client.0.smithi032.stdout:7/349: write dc/d12/d22/f47 [779964,119837] 0 2024-06-03T01:21:02.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/350: write f6 [5011510,63891] 0 2024-06-03T01:21:02.307 INFO:tasks.workunit.client.0.smithi032.stdout:4/370: rename d4/db/da/d2d/c42 to d4/db/c87 0 2024-06-03T01:21:02.310 INFO:tasks.workunit.client.1.smithi044.stdout:8/895: creat d5/d55/f12d x:0 0 0 2024-06-03T01:21:02.310 INFO:tasks.workunit.client.0.smithi032.stdout:4/371: symlink d4/db/da/d37/l88 0 2024-06-03T01:21:02.313 INFO:tasks.workunit.client.1.smithi044.stdout:4/930: mknod d17/d1c/d75/d7a/db8/d28/d2b/c139 0 2024-06-03T01:21:02.313 INFO:tasks.workunit.client.1.smithi044.stdout:4/931: getdents d17/d1c/d75/d7a/db8/d35/d117 0 2024-06-03T01:21:02.314 INFO:tasks.workunit.client.1.smithi044.stdout:4/932: read d17/d1c/d75/d7a/db8/d59/fda [162394,68508] 0 2024-06-03T01:21:02.317 INFO:tasks.workunit.client.1.smithi044.stdout:9/833: sync 2024-06-03T01:21:02.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/840: dwrite d2/d11/d12/d2b/d4a/d57/f6b [0,4194304] 0 2024-06-03T01:21:02.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/841: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f11b x:0 0 0 2024-06-03T01:21:02.321 INFO:tasks.workunit.client.1.smithi044.stdout:5/842: write d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f7e [744316,34649] 0 2024-06-03T01:21:02.322 INFO:tasks.workunit.client.1.smithi044.stdout:9/834: creat d6/dd/d51/d16/d27/d5e/d71/d89/f10d x:0 0 0 2024-06-03T01:21:02.322 INFO:tasks.workunit.client.1.smithi044.stdout:9/835: chown d6/dd/d51/d16/d27/d5e/d71/de0/f10a 8523 1 2024-06-03T01:21:02.327 INFO:tasks.workunit.client.0.smithi032.stdout:7/351: dwrite dc/d12/d2a/f43 [0,4194304] 0 2024-06-03T01:21:02.333 INFO:tasks.workunit.client.1.smithi044.stdout:8/896: dread d5/d16/f7c [0,4194304] 0 2024-06-03T01:21:02.335 INFO:tasks.workunit.client.1.smithi044.stdout:5/843: rename d2/d11/d1e/d2f to d2/d11c 0 2024-06-03T01:21:02.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/372: dwrite d4/db/da/d1d/f1f [0,4194304] 0 2024-06-03T01:21:02.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/373: creat d4/db/da/d2d/d27/f89 x:0 0 0 2024-06-03T01:21:02.336 INFO:tasks.workunit.client.1.smithi044.stdout:8/897: mkdir d5/d55/d74/db4/d12e 0 2024-06-03T01:21:02.340 INFO:tasks.workunit.client.0.smithi032.stdout:7/352: dread dc/d12/d22/d42/f51 [0,4194304] 0 2024-06-03T01:21:02.341 INFO:tasks.workunit.client.1.smithi044.stdout:4/933: dwrite d17/d1c/d75/d7a/db8/d59/fda [0,4194304] 0 2024-06-03T01:21:02.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/836: rename d6/dd/d51/db/fb4 to d6/dd/d51/d16/d5f/d8b/f10e 0 2024-06-03T01:21:02.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/837: write d6/dd/d51/d16/d5f/d8b/f107 [71005,55781] 0 2024-06-03T01:21:02.342 INFO:tasks.workunit.client.1.smithi044.stdout:3/758: sync 2024-06-03T01:21:02.342 INFO:tasks.workunit.client.1.smithi044.stdout:7/883: sync 2024-06-03T01:21:02.343 INFO:tasks.workunit.client.1.smithi044.stdout:8/898: mknod d5/d55/d74/db4/d127/c12f 0 2024-06-03T01:21:02.343 INFO:tasks.workunit.client.1.smithi044.stdout:8/899: stat d5/d16/d81/de3/d98/ddd 0 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/353: rename dc/d12/d22/f2e to dc/d12/f6b 0 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/354: chown dc/d12/d24/d33/l49 53222 1 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/355: write dc/d12/d22/f47 [1274277,60335] 0 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/356: readlink dc/d12/d22/l58 0 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/357: getdents dc/d12/d2a 0 2024-06-03T01:21:02.344 INFO:tasks.workunit.client.0.smithi032.stdout:4/374: dread d4/db/da/d2d/d27/f4f [0,4194304] 0 2024-06-03T01:21:02.345 INFO:tasks.workunit.client.1.smithi044.stdout:5/844: symlink d2/d11/d12/d2b/d4a/d57/l11d 0 2024-06-03T01:21:02.355 INFO:tasks.workunit.client.1.smithi044.stdout:4/934: symlink d17/d63/d10c/l13a 0 2024-06-03T01:21:02.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/838: truncate f4 5634997 0 2024-06-03T01:21:02.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/839: readlink d6/dd/d51/d16/d27/d5e/l91 0 2024-06-03T01:21:02.359 INFO:tasks.workunit.client.1.smithi044.stdout:3/759: mkdir df/d13/d1b/d4b/d69/dd1/df7 0 2024-06-03T01:21:02.364 INFO:tasks.workunit.client.1.smithi044.stdout:9/840: symlink d6/dd/d51/d16/d27/d5e/l10f 0 2024-06-03T01:21:02.365 INFO:tasks.workunit.client.1.smithi044.stdout:2/769: sync 2024-06-03T01:21:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/900: rmdir d5/d55/d56/d5c/dee 39 2024-06-03T01:21:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/901: readlink d5/d55/d74/db4/d127/l128 0 2024-06-03T01:21:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/902: fsync d5/fa4 0 2024-06-03T01:21:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/903: fdatasync d5/d16/d81/de3/d58/f107 0 2024-06-03T01:21:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/904: creat d5/d55/d56/d5c/d68/de5/f130 x:0 0 0 2024-06-03T01:21:02.367 INFO:tasks.workunit.client.1.smithi044.stdout:8/905: chown d5/fe4 11027 1 2024-06-03T01:21:02.367 INFO:tasks.workunit.client.1.smithi044.stdout:8/906: dread - d5/d55/d56/d5c/d68/de5/dea/f7f zero size 2024-06-03T01:21:02.367 INFO:tasks.workunit.client.1.smithi044.stdout:5/845: dwrite d2/d11/d1e/d21/d24/d93/d9d/f8c [0,4194304] 0 2024-06-03T01:21:02.367 INFO:tasks.workunit.client.1.smithi044.stdout:4/935: link d17/d1c/d75/d7a/db8/d59/de5/l125 d17/d1c/d7e/l13b 0 2024-06-03T01:21:02.368 INFO:tasks.workunit.client.1.smithi044.stdout:9/841: mkdir d6/dd/d51/d16/d27/d5e/d110 0 2024-06-03T01:21:02.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/760: rmdir df/d13/d6d/d9a 39 2024-06-03T01:21:02.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/761: stat df/d1e/d3a/f88 0 2024-06-03T01:21:02.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/762: stat df/d1e/d3a/d84/d90 0 2024-06-03T01:21:02.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/763: dread - df/d1e/d3a/fb6 zero size 2024-06-03T01:21:02.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/764: fsync df/d13/d1b/fc6 0 2024-06-03T01:21:02.376 INFO:tasks.workunit.client.1.smithi044.stdout:9/842: creat d6/dd/d45/f111 x:0 0 0 2024-06-03T01:21:02.376 INFO:tasks.workunit.client.1.smithi044.stdout:9/843: creat d6/dd/d51/d16/d27/d5e/d71/d89/f112 x:0 0 0 2024-06-03T01:21:02.376 INFO:tasks.workunit.client.1.smithi044.stdout:9/844: fdatasync d6/dd/d51/d16/d5f/d8b/f10e 0 2024-06-03T01:21:02.376 INFO:tasks.workunit.client.1.smithi044.stdout:9/845: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fb6 [263820,88364] 0 2024-06-03T01:21:02.377 INFO:tasks.workunit.client.1.smithi044.stdout:3/765: mkdir df/d13/d6d/d9a/db7/df8 0 2024-06-03T01:21:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/884: dwrite d0/d3/d8/d95/d63/fc1 [0,4194304] 0 2024-06-03T01:21:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/885: write d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/fd7 [5368192,31654] 0 2024-06-03T01:21:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/886: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/df6/f112 zero size 2024-06-03T01:21:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/887: chown d0/d3/d8/d95/d63/d89/d8b/da9 88296936 1 2024-06-03T01:21:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/888: write d0/d3/d8/d95/d63/d7f/d9c/f92 [893540,85995] 0 2024-06-03T01:21:02.380 INFO:tasks.workunit.client.0.smithi032.stdout:7/358: dwrite dc/f30 [4194304,4194304] 0 2024-06-03T01:21:02.380 INFO:tasks.workunit.client.1.smithi044.stdout:3/766: unlink df/d1e/d78/cee 0 2024-06-03T01:21:02.380 INFO:tasks.workunit.client.1.smithi044.stdout:8/907: read d5/d55/d74/f94 [812244,3665] 0 2024-06-03T01:21:02.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/936: rmdir d17/d1c/d75/d7a 39 2024-06-03T01:21:02.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/937: chown d17/d1c/d75/d7a/db8 4135022 1 2024-06-03T01:21:02.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/938: write d17/d1c/d75/d7a/db8/d35/f3c [666494,5151] 0 2024-06-03T01:21:02.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/939: truncate d17/d1c/d75/d7a/db8/db9/fc3 5051240 0 2024-06-03T01:21:02.383 INFO:tasks.workunit.client.1.smithi044.stdout:4/940: fsync d17/d1c/d7e/fa8 0 2024-06-03T01:21:02.384 INFO:tasks.workunit.client.1.smithi044.stdout:2/770: write db/d15/d49/f5c [2093848,23205] 0 2024-06-03T01:21:02.385 INFO:tasks.workunit.client.1.smithi044.stdout:7/889: creat d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/f118 x:0 0 0 2024-06-03T01:21:02.387 INFO:tasks.workunit.client.1.smithi044.stdout:9/846: dread d6/f44 [0,4194304] 0 2024-06-03T01:21:02.388 INFO:tasks.workunit.client.1.smithi044.stdout:3/767: creat df/d13/d1b/d56/d5c/d98/ff9 x:0 0 0 2024-06-03T01:21:02.388 INFO:tasks.workunit.client.1.smithi044.stdout:3/768: write df/d13/d39/d3d/d43/f5b [9428551,109068] 0 2024-06-03T01:21:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:8/908: symlink d5/d20/d47/d64/l131 0 2024-06-03T01:21:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:8/909: write d5/d55/d56/d5c/d68/de5/ffb [986726,20639] 0 2024-06-03T01:21:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:8/910: fdatasync d5/d16/d81/de3/d58/d89/fab 0 2024-06-03T01:21:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:8/911: truncate d5/d55/ff7 244192 0 2024-06-03T01:21:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:5/846: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/ff2 [0,4194304] 0 2024-06-03T01:21:02.392 INFO:tasks.workunit.client.1.smithi044.stdout:4/941: symlink d17/d1c/d9c/l13c 0 2024-06-03T01:21:02.392 INFO:tasks.workunit.client.1.smithi044.stdout:9/847: dread d6/dd/d51/d16/d5f/d8b/d8d/fb9 [0,4194304] 0 2024-06-03T01:21:02.394 INFO:tasks.workunit.client.1.smithi044.stdout:3/769: creat df/d1e/d3a/d84/d90/ffa x:0 0 0 2024-06-03T01:21:02.397 INFO:tasks.workunit.client.1.smithi044.stdout:8/912: truncate d5/d20/db3/fb7 2711268 0 2024-06-03T01:21:02.406 INFO:tasks.workunit.client.1.smithi044.stdout:4/942: symlink d17/d1c/d75/d7a/db3/l13d 0 2024-06-03T01:21:02.409 INFO:tasks.workunit.client.1.smithi044.stdout:9/848: link d6/dd/f87 d6/dd/d51/d16/d27/d5e/f113 0 2024-06-03T01:21:02.415 INFO:tasks.workunit.client.1.smithi044.stdout:9/849: creat d6/dd/d51/d16/d27/d5e/db7/de1/f114 x:0 0 0 2024-06-03T01:21:02.415 INFO:tasks.workunit.client.1.smithi044.stdout:9/850: write d6/f12 [558751,104261] 0 2024-06-03T01:21:02.416 INFO:tasks.workunit.client.1.smithi044.stdout:2/771: dwrite db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:21:02.417 INFO:tasks.workunit.client.1.smithi044.stdout:5/847: dwrite d2/d11/d1e/d21/d24/d93/d9d/f22 [0,4194304] 0 2024-06-03T01:21:02.417 INFO:tasks.workunit.client.1.smithi044.stdout:5/848: readlink d2/d11/d12/d2b/d4a/l7b 0 2024-06-03T01:21:02.421 INFO:tasks.workunit.client.1.smithi044.stdout:5/849: mkdir d2/d11/d1e/d21/d24/d93/d11e 0 2024-06-03T01:21:02.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/850: truncate d2/d11/d1e/d21/d24/d93/d9d/f49 1111491 0 2024-06-03T01:21:02.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/851: chown d2/d11/fd8 0 1 2024-06-03T01:21:02.424 INFO:tasks.workunit.client.1.smithi044.stdout:5/852: stat d2/lc2 0 2024-06-03T01:21:02.425 INFO:tasks.workunit.client.1.smithi044.stdout:2/772: getdents db/d1b/d27 0 2024-06-03T01:21:02.426 INFO:tasks.workunit.client.1.smithi044.stdout:8/913: dwrite d5/d55/d56/d5c/d68/de5/dea/d79/d88/fc2 [0,4194304] 0 2024-06-03T01:21:02.427 INFO:tasks.workunit.client.1.smithi044.stdout:3/770: dwrite df/f49 [0,4194304] 0 2024-06-03T01:21:02.427 INFO:tasks.workunit.client.1.smithi044.stdout:3/771: chown df/d13/f3f 1071031543 1 2024-06-03T01:21:02.428 INFO:tasks.workunit.client.1.smithi044.stdout:5/853: link d2/d11/d1e/d21/d24/f77 d2/d11/df1/f11f 0 2024-06-03T01:21:02.428 INFO:tasks.workunit.client.1.smithi044.stdout:5/854: fsync d2/d11/d1e/d21/d24/d9e/d107/f5f 0 2024-06-03T01:21:02.428 INFO:tasks.workunit.client.1.smithi044.stdout:5/855: read d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f7e [1073442,9879] 0 2024-06-03T01:21:02.430 INFO:tasks.workunit.client.1.smithi044.stdout:2/773: creat db/d15/d49/d82/db5/ff4 x:0 0 0 2024-06-03T01:21:02.430 INFO:tasks.workunit.client.1.smithi044.stdout:2/774: chown db/d15/d49/d82 25784704 1 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/914: creat d5/d20/d47/d64/dd0/f132 x:0 0 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/915: fsync d5/d55/d56/d5c/d68/de5/dea/d79/d88/f101 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/916: chown d5/d16/f24 5330 1 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/917: truncate d5/d16/d1d/d32/f124 174636 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/918: creat d5/d16/d81/d9b/f133 x:0 0 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/919: write d5/d55/d56/d5c/d68/de5/dea/f7f [800714,59668] 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/920: chown d5/d55/d56/d5c/d68/de5/dea/f57 243 1 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/921: creat d5/d16/d81/de3/d98/f134 x:0 0 0 2024-06-03T01:21:02.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/922: chown d5/f10d 3051 1 2024-06-03T01:21:02.435 INFO:tasks.workunit.client.1.smithi044.stdout:8/923: read - d5/d16/d81/d9b/fc1 zero size 2024-06-03T01:21:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:3/772: creat df/d13/d1b/d56/ffb x:0 0 0 2024-06-03T01:21:02.439 INFO:tasks.workunit.client.1.smithi044.stdout:5/856: mknod d2/d11/d1e/d3b/da6/d116/c120 0 2024-06-03T01:21:02.442 INFO:tasks.workunit.client.1.smithi044.stdout:2/775: mkdir db/d1b/d2a/d2f/d4c/df5 0 2024-06-03T01:21:02.446 INFO:tasks.workunit.client.1.smithi044.stdout:8/924: creat d5/d16/d81/de3/d58/d89/dbf/dce/f135 x:0 0 0 2024-06-03T01:21:02.449 INFO:tasks.workunit.client.1.smithi044.stdout:2/776: creat db/d15/d49/d6c/d72/d8f/de5/ff6 x:0 0 0 2024-06-03T01:21:02.449 INFO:tasks.workunit.client.1.smithi044.stdout:9/851: getdents d6/dd/d51/d16/d27/d5e/db7/de1 0 2024-06-03T01:21:02.450 INFO:tasks.workunit.client.1.smithi044.stdout:2/777: dread db/d15/d49/d6c/d72/dcd/fcf [0,4194304] 0 2024-06-03T01:21:02.455 INFO:tasks.workunit.client.1.smithi044.stdout:8/925: dread d5/d16/d1d/f42 [0,4194304] 0 2024-06-03T01:21:02.457 INFO:tasks.workunit.client.1.smithi044.stdout:3/773: rename c8 to df/d1e/d78/cfc 0 2024-06-03T01:21:02.457 INFO:tasks.workunit.client.1.smithi044.stdout:3/774: truncate df/d13/d1b/f38 610148 0 2024-06-03T01:21:02.457 INFO:tasks.workunit.client.1.smithi044.stdout:3/775: write df/d13/d1b/d4b/fea [48130,17081] 0 2024-06-03T01:21:02.458 INFO:tasks.workunit.client.1.smithi044.stdout:2/778: mknod db/d15/d49/d82/dc8/cf7 0 2024-06-03T01:21:02.458 INFO:tasks.workunit.client.1.smithi044.stdout:2/779: chown db/d15/cd8 8 1 2024-06-03T01:21:02.459 INFO:tasks.workunit.client.1.smithi044.stdout:9/852: dread d6/dd/fa3 [0,4194304] 0 2024-06-03T01:21:02.459 INFO:tasks.workunit.client.1.smithi044.stdout:9/853: chown d6/dd/d51/d16/d5f/d8b 883 1 2024-06-03T01:21:02.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/926: symlink d5/d20/d33/dfd/l136 0 2024-06-03T01:21:02.463 INFO:tasks.workunit.client.1.smithi044.stdout:3/776: unlink df/d13/d1b/d56/le1 0 2024-06-03T01:21:02.464 INFO:tasks.workunit.client.1.smithi044.stdout:3/777: readlink df/d13/d39/d3d/d50/l57 0 2024-06-03T01:21:02.466 INFO:tasks.workunit.client.1.smithi044.stdout:2/780: mkdir db/d15/d49/d7b/df8 0 2024-06-03T01:21:02.468 INFO:tasks.workunit.client.1.smithi044.stdout:9/854: creat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/f115 x:0 0 0 2024-06-03T01:21:02.468 INFO:tasks.workunit.client.1.smithi044.stdout:9/855: chown d6/d38/cb5 12096005 1 2024-06-03T01:21:02.470 INFO:tasks.workunit.client.1.smithi044.stdout:5/857: rename d2/d11/d12/d2b/d4a/d57/cba to d2/d11/d1e/d21/d24/d9e/d107/c121 0 2024-06-03T01:21:02.471 INFO:tasks.workunit.client.1.smithi044.stdout:3/778: rmdir df/d13/d6d/d9a/db7 39 2024-06-03T01:21:02.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/856: mkdir d6/dd/d51/d16/d116 0 2024-06-03T01:21:02.475 INFO:tasks.workunit.client.1.smithi044.stdout:8/927: dwrite d5/d16/d1d/d32/f7a [0,4194304] 0 2024-06-03T01:21:02.476 INFO:tasks.workunit.client.1.smithi044.stdout:8/928: read d5/d20/d47/f49 [345217,14827] 0 2024-06-03T01:21:02.476 INFO:tasks.workunit.client.1.smithi044.stdout:8/929: dread - d5/d16/d1d/d59/f105 zero size 2024-06-03T01:21:02.477 INFO:tasks.workunit.client.1.smithi044.stdout:5/858: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117/f122 x:0 0 0 2024-06-03T01:21:02.477 INFO:tasks.workunit.client.1.smithi044.stdout:5/859: chown d2/d11/d1e/d21/fe6 1883266343 1 2024-06-03T01:21:02.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/779: creat df/d13/d39/d3d/d50/ffd x:0 0 0 2024-06-03T01:21:02.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/780: fdatasync df/d13/f3f 0 2024-06-03T01:21:02.480 INFO:tasks.workunit.client.1.smithi044.stdout:2/781: mkdir db/dd/df9 0 2024-06-03T01:21:02.483 INFO:tasks.workunit.client.1.smithi044.stdout:9/857: link d6/dd/d51/d16/d27/d5e/db7/de1/fc9 d6/dd/d51/f117 0 2024-06-03T01:21:02.485 INFO:tasks.workunit.client.1.smithi044.stdout:8/930: dread d5/d16/d81/de3/d98/ddd/fe1 [0,4194304] 0 2024-06-03T01:21:02.488 INFO:tasks.workunit.client.1.smithi044.stdout:5/860: link d2/d11/d1e/d21/d24/d9e/d107/f5f d2/d11/d1e/d21/d24/d93/d11e/f123 0 2024-06-03T01:21:02.490 INFO:tasks.workunit.client.1.smithi044.stdout:3/781: mknod df/d13/d1b/d56/d5c/d98/df4/cfe 0 2024-06-03T01:21:02.493 INFO:tasks.workunit.client.1.smithi044.stdout:8/931: creat d5/d55/db5/f137 x:0 0 0 2024-06-03T01:21:02.498 INFO:tasks.workunit.client.1.smithi044.stdout:3/782: link df/d13/d1b/d56/d5c/d98/la1 df/d13/d6d/d9a/db7/lff 0 2024-06-03T01:21:02.498 INFO:tasks.workunit.client.1.smithi044.stdout:3/783: creat df/d1e/d78/dda/f100 x:0 0 0 2024-06-03T01:21:02.499 INFO:tasks.workunit.client.1.smithi044.stdout:9/858: getdents d6/dd/d51/d16/d27/d5e/d71/d89 0 2024-06-03T01:21:02.500 INFO:tasks.workunit.client.1.smithi044.stdout:8/932: truncate d5/d55/d74/d87/f7b 98630 0 2024-06-03T01:21:02.501 INFO:tasks.workunit.client.1.smithi044.stdout:5/861: dwrite d2/d11/d1e/d21/d24/f38 [4194304,4194304] 0 2024-06-03T01:21:02.501 INFO:tasks.workunit.client.1.smithi044.stdout:5/862: write d2/d11/d12/d2b/f34 [4180627,70345] 0 2024-06-03T01:21:02.504 INFO:tasks.workunit.client.1.smithi044.stdout:3/784: mknod df/d1e/d2a/d70/c101 0 2024-06-03T01:21:02.507 INFO:tasks.workunit.client.1.smithi044.stdout:5/863: mkdir d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/d124 0 2024-06-03T01:21:02.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/785: getdents df/d1e 0 2024-06-03T01:21:02.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/786: write df/d1e/d2a/d70/fba [377423,52397] 0 2024-06-03T01:21:02.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/787: chown df/d13/d6d/d9a/fcc 2224570 1 2024-06-03T01:21:02.510 INFO:tasks.workunit.client.1.smithi044.stdout:7/890: sync 2024-06-03T01:21:02.513 INFO:tasks.workunit.client.1.smithi044.stdout:7/891: creat d0/d3/d8/d95/d63/d89/da3/dde/d10a/f119 x:0 0 0 2024-06-03T01:21:02.513 INFO:tasks.workunit.client.1.smithi044.stdout:7/892: fsync d0/fd 0 2024-06-03T01:21:02.513 INFO:tasks.workunit.client.1.smithi044.stdout:7/893: chown d0/d3/d8/d95/d84/dd5 107962 1 2024-06-03T01:21:02.517 INFO:tasks.workunit.client.1.smithi044.stdout:7/894: rmdir d0/d3/d8/d95/d63/d7f/d9c/da4/dc9 0 2024-06-03T01:21:02.518 INFO:tasks.workunit.client.1.smithi044.stdout:3/788: dread df/d13/d1b/d56/d5c/d98/df4/f91 [0,4194304] 0 2024-06-03T01:21:02.520 INFO:tasks.workunit.client.1.smithi044.stdout:7/895: rename d0/d3/d8/d95/d63/d7f/d9c/da4/f115 to d0/d3/d8/d32/d33/dd1/f11a 0 2024-06-03T01:21:02.521 INFO:tasks.workunit.client.0.smithi032.stdout:2/370: sync 2024-06-03T01:21:02.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/426: sync 2024-06-03T01:21:02.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/427: stat da/c3f 0 2024-06-03T01:21:02.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/428: fsync da/dd/df/d10/d16/d18/f27 0 2024-06-03T01:21:02.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/789: mkdir df/d13/d39/d3d/d102 0 2024-06-03T01:21:02.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/790: mknod df/d13/d1b/d4b/c103 0 2024-06-03T01:21:02.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/791: chown df/d13/d39/d3d/d6a/d9c/ff2 108006809 1 2024-06-03T01:21:02.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/371: mknod d5/da/d2e/c7d 0 2024-06-03T01:21:02.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/372: stat d5/da/l5b 0 2024-06-03T01:21:02.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/792: symlink df/l104 0 2024-06-03T01:21:02.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/793: truncate df/d1e/d2a/d70/fbe 85811 0 2024-06-03T01:21:02.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/794: write df/d13/d1b/d59/fd8 [658308,32894] 0 2024-06-03T01:21:02.530 INFO:tasks.workunit.client.0.smithi032.stdout:2/373: dread d5/da/f1e [4194304,4194304] 0 2024-06-03T01:21:02.531 INFO:tasks.workunit.client.1.smithi044.stdout:3/795: dread df/d13/d1b/f34 [0,4194304] 0 2024-06-03T01:21:02.534 INFO:tasks.workunit.client.0.smithi032.stdout:2/374: rename d5/d38/d52 to d5/da/d1a/d2a/d49/d7e 0 2024-06-03T01:21:02.535 INFO:tasks.workunit.client.0.smithi032.stdout:2/375: mknod d5/d38/d46/d5c/d6c/c7f 0 2024-06-03T01:21:02.536 INFO:tasks.workunit.client.0.smithi032.stdout:2/376: mkdir d5/d38/d46/d5c/d6c/d80 0 2024-06-03T01:21:02.538 INFO:tasks.workunit.client.0.smithi032.stdout:2/377: getdents d5/da/d1a/d69 0 2024-06-03T01:21:02.540 INFO:tasks.workunit.client.0.smithi032.stdout:2/378: creat d5/d38/d46/f81 x:0 0 0 2024-06-03T01:21:02.540 INFO:tasks.workunit.client.1.smithi044.stdout:7/896: dwrite d0/d3/da8/fe0 [0,4194304] 0 2024-06-03T01:21:02.540 INFO:tasks.workunit.client.1.smithi044.stdout:7/897: write d0/d3/d30/f79 [275208,111274] 0 2024-06-03T01:21:02.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.541+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20c4000bf0 con 0x7f20cc0393b0 2024-06-03T01:21:02.543 INFO:tasks.workunit.client.1.smithi044.stdout:7/898: creat d0/d3/d8/da0/f11b x:0 0 0 2024-06-03T01:21:02.543 INFO:tasks.workunit.client.1.smithi044.stdout:7/899: rmdir d0/d3/d8/d32 39 2024-06-03T01:21:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:7/900: creat d0/d16/f11c x:0 0 0 2024-06-03T01:21:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:7/901: chown d0/d3/d8/d95/d63/d7f/d9c/f92 6 1 2024-06-03T01:21:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:7/902: fsync d0/d3/d8/d95/d63/d89/da3/f5e 0 2024-06-03T01:21:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:7/903: write d0/d3/d8/d95/d63/d89/da3/dde/d10a/f119 [243152,16098] 0 2024-06-03T01:21:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:7/904: write d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/ffd [483158,95191] 0 2024-06-03T01:21:02.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.546+0000 7f20d67fc700 1 -- 172.21.15.32:0/2540350477 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f20c4000bf0 con 0x7f20cc0393b0 2024-06-03T01:21:02.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 msgr2=0x7f20cc03b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 0x7f20cc03b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bf0 tx=0x7f20d0009260 comp rx=0 tx=0).stop 2024-06-03T01:21:02.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 msgr2=0x7f20e0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d800fab0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 shutdown_connections 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f20cc0393b0 0x7f20cc03b870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00ffe10 0x7f20e0198710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 --2- 172.21.15.32:0/2540350477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e01011b0 0x7f20e0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 >> 172.21.15.32:0/2540350477 conn(0x7f20e00fa8c0 msgr2=0x7f20e00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 shutdown_connections 2024-06-03T01:21:02.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.548+0000 7f20e73f5700 1 -- 172.21.15.32:0/2540350477 wait complete. 2024-06-03T01:21:02.556 INFO:tasks.workunit.client.1.smithi044.stdout:5/864: dwrite d2/d11/d3e/f59 [4194304,4194304] 0 2024-06-03T01:21:02.557 INFO:tasks.workunit.client.1.smithi044.stdout:5/865: read - d2/d11/fb2 zero size 2024-06-03T01:21:02.559 INFO:tasks.workunit.client.1.smithi044.stdout:5/866: rename d2/d11/d1e/d21/d24/d9e/d107/l9b to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117/l125 0 2024-06-03T01:21:02.567 INFO:tasks.workunit.client.1.smithi044.stdout:5/867: symlink d2/d11/d1e/d21/d24/d93/d11e/l126 0 2024-06-03T01:21:02.567 INFO:tasks.workunit.client.1.smithi044.stdout:5/868: fdatasync d2/d11/d1e/d21/d24/d93/d9d/d28/d66/fa2 0 2024-06-03T01:21:02.567 INFO:tasks.workunit.client.1.smithi044.stdout:5/869: write d2/d11/d12/f39 [1583477,22612] 0 2024-06-03T01:21:02.568 INFO:tasks.workunit.client.1.smithi044.stdout:5/870: getdents d2/d11/d1e 0 2024-06-03T01:21:02.568 INFO:tasks.workunit.client.1.smithi044.stdout:7/905: dwrite d0/d16/f22 [0,4194304] 0 2024-06-03T01:21:02.569 INFO:tasks.workunit.client.0.smithi032.stdout:2/379: dwrite d5/f1c [4194304,4194304] 0 2024-06-03T01:21:02.570 INFO:tasks.workunit.client.1.smithi044.stdout:5/871: dread d2/d11/d12/d2b/d4a/d57/fb4 [0,4194304] 0 2024-06-03T01:21:02.571 INFO:tasks.workunit.client.0.smithi032.stdout:2/380: symlink d5/d38/d46/d5c/d6c/d80/l82 0 2024-06-03T01:21:02.571 INFO:tasks.workunit.client.0.smithi032.stdout:2/381: fdatasync d5/da/f2c 0 2024-06-03T01:21:02.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/382: mkdir d5/da/d1a/d69/d83 0 2024-06-03T01:21:02.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/383: stat d5/da/d2e/f32 0 2024-06-03T01:21:02.575 INFO:tasks.workunit.client.1.smithi044.stdout:7/906: link d0/d3/d8/d95/f53 d0/d3/d8/d95/d63/f11d 0 2024-06-03T01:21:02.579 INFO:tasks.workunit.client.1.smithi044.stdout:7/907: truncate d0/d3/d8/f10 621364 0 2024-06-03T01:21:02.582 INFO:tasks.workunit.client.1.smithi044.stdout:7/908: creat d0/d3/d8/d32/f11e x:0 0 0 2024-06-03T01:21:02.584 INFO:tasks.workunit.client.1.smithi044.stdout:4/943: sync 2024-06-03T01:21:02.584 INFO:tasks.workunit.client.1.smithi044.stdout:4/944: stat d17/d1c/d75/d7a/db8/d6a/lcb 0 2024-06-03T01:21:02.586 INFO:tasks.workunit.client.1.smithi044.stdout:4/945: creat d17/d1c/d75/d7a/db8/db9/f13e x:0 0 0 2024-06-03T01:21:02.592 INFO:tasks.workunit.client.1.smithi044.stdout:5/872: dwrite d2/d11/d12/d2b/d4a/f5d [0,4194304] 0 2024-06-03T01:21:02.592 INFO:tasks.workunit.client.1.smithi044.stdout:5/873: write d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117/f122 [238865,88882] 0 2024-06-03T01:21:02.592 INFO:tasks.workunit.client.1.smithi044.stdout:5/874: chown d2/d11/d12/d2b/d4a/l53 19198158 1 2024-06-03T01:21:02.592 INFO:tasks.workunit.client.1.smithi044.stdout:5/875: fsync d2/d11/d12/f67 0 2024-06-03T01:21:02.595 INFO:tasks.workunit.client.0.smithi032.stdout:2/384: dwrite d5/f58 [0,4194304] 0 2024-06-03T01:21:02.595 INFO:tasks.workunit.client.0.smithi032.stdout:2/385: creat d5/da/d1a/d2a/d36/d3f/f84 x:0 0 0 2024-06-03T01:21:02.600 INFO:tasks.workunit.client.1.smithi044.stdout:4/946: write d17/d1c/d75/d7a/db8/d35/f4f [7468141,20584] 0 2024-06-03T01:21:02.602 INFO:tasks.workunit.client.1.smithi044.stdout:4/947: mkdir d17/d1c/d70/d7d/d135/d13f 0 2024-06-03T01:21:02.602 INFO:tasks.workunit.client.0.smithi032.stdout:2/386: symlink d5/d38/d46/d5c/d6c/d80/l85 0 2024-06-03T01:21:02.603 INFO:tasks.workunit.client.1.smithi044.stdout:4/948: fdatasync d17/d1c/d75/d7a/db8/d59/f6d 0 2024-06-03T01:21:02.604 INFO:tasks.workunit.client.1.smithi044.stdout:7/909: dwrite d0/d16/f3c [0,4194304] 0 2024-06-03T01:21:02.604 INFO:tasks.workunit.client.0.smithi032.stdout:2/387: mkdir d5/da/d1a/d2a/d49/d7e/d86 0 2024-06-03T01:21:02.606 INFO:tasks.workunit.client.0.smithi032.stdout:2/388: mknod d5/da/c87 0 2024-06-03T01:21:02.606 INFO:tasks.workunit.client.0.smithi032.stdout:2/389: write d5/ff [1922416,26794] 0 2024-06-03T01:21:02.606 INFO:tasks.workunit.client.1.smithi044.stdout:4/949: creat d17/d1c/d75/f140 x:0 0 0 2024-06-03T01:21:02.606 INFO:tasks.workunit.client.1.smithi044.stdout:4/950: write d17/d63/f65 [809918,96032] 0 2024-06-03T01:21:02.608 INFO:tasks.workunit.client.0.smithi032.stdout:0/328: sync 2024-06-03T01:21:02.608 INFO:tasks.workunit.client.0.smithi032.stdout:3/342: sync 2024-06-03T01:21:02.609 INFO:tasks.workunit.client.1.smithi044.stdout:4/951: mkdir d17/d1c/d75/d7a/db8/d35/d117/d141 0 2024-06-03T01:21:02.609 INFO:tasks.workunit.client.1.smithi044.stdout:4/952: creat d17/f142 x:0 0 0 2024-06-03T01:21:02.609 INFO:tasks.workunit.client.1.smithi044.stdout:4/953: write d17/d1c/d75/d7a/db8/f57 [5420556,100261] 0 2024-06-03T01:21:02.610 INFO:tasks.workunit.client.0.smithi032.stdout:0/329: mkdir d1/da/d28/d3b/d68 0 2024-06-03T01:21:02.611 INFO:tasks.workunit.client.0.smithi032.stdout:3/343: rmdir de/d29/d2e/d31 39 2024-06-03T01:21:02.613 INFO:tasks.workunit.client.1.smithi044.stdout:5/876: dwrite d2/d11/d12/d2b/d4a/f5d [4194304,4194304] 0 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:8/307: sync 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:6/385: sync 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:8/308: fsync f8 0 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:6/386: write d8/d11/d20/d2b/d39/d19/f56 [1009671,119286] 0 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:9/457: sync 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:9/458: getdents d0/d23/d24/d92 0 2024-06-03T01:21:02.617 INFO:tasks.workunit.client.0.smithi032.stdout:1/329: sync 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/330: dread d1/f17 [0,4194304] 0 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/331: dread - d1/d46/f5f zero size 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/332: chown d1/f9 11774 1 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:6/387: truncate d8/d4c/f58 3620826 0 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:6/388: stat d8/d11/d20/d3b/l61 0 2024-06-03T01:21:02.620 INFO:tasks.workunit.client.0.smithi032.stdout:6/389: read d8/d11/d20/d3b/f41 [2654144,4307] 0 2024-06-03T01:21:02.621 INFO:tasks.workunit.client.1.smithi044.stdout:5/877: symlink d2/d11c/df5/l127 0 2024-06-03T01:21:02.621 INFO:tasks.workunit.client.1.smithi044.stdout:5/878: write d2/d11/d1e/d21/d24/d93/d11e/f123 [5933594,22583] 0 2024-06-03T01:21:02.621 INFO:tasks.workunit.client.0.smithi032.stdout:2/390: read d5/f58 [190808,123978] 0 2024-06-03T01:21:02.621 INFO:tasks.workunit.client.1.smithi044.stdout:4/954: dread d17/d1c/fb7 [0,4194304] 0 2024-06-03T01:21:02.622 INFO:tasks.workunit.client.1.smithi044.stdout:4/955: creat d17/d1c/d75/d7a/db8/db9/f143 x:0 0 0 2024-06-03T01:21:02.622 INFO:tasks.workunit.client.0.smithi032.stdout:8/309: dread f1 [0,4194304] 0 2024-06-03T01:21:02.624 INFO:tasks.workunit.client.0.smithi032.stdout:1/330: mknod d18/d2d/d5e/c68 0 2024-06-03T01:21:02.625 INFO:tasks.workunit.client.1.smithi044.stdout:4/956: dread d17/d1c/d70/fe4 [0,4194304] 0 2024-06-03T01:21:02.625 INFO:tasks.workunit.client.0.smithi032.stdout:0/333: stat d1/da/d10/l15 0 2024-06-03T01:21:02.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/334: stat d1/d46/d3a 0 2024-06-03T01:21:02.626 INFO:tasks.workunit.client.0.smithi032.stdout:3/344: dread de/d29/d2e/d31/d3c/f61 [0,4194304] 0 2024-06-03T01:21:02.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/310: mknod dc/d25/d28/c68 0 2024-06-03T01:21:02.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/311: truncate dc/d25/d28/d3c/f42 319864 0 2024-06-03T01:21:02.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/312: chown dc/de 1 1 2024-06-03T01:21:02.627 INFO:tasks.workunit.client.1.smithi044.stdout:7/910: dwrite d0/d3/d8/d95/d63/d89/da3/dde/f102 [0,4194304] 0 2024-06-03T01:21:02.629 INFO:tasks.workunit.client.1.smithi044.stdout:4/957: creat d17/d1c/d75/d7a/db8/d6a/d84/df4/f144 x:0 0 0 2024-06-03T01:21:02.629 INFO:tasks.workunit.client.1.smithi044.stdout:4/958: fdatasync d17/d1c/d75/d7a/db8/f4d 0 2024-06-03T01:21:02.630 INFO:tasks.workunit.client.0.smithi032.stdout:9/459: rmdir d0/d11/d1d 39 2024-06-03T01:21:02.631 INFO:tasks.workunit.client.0.smithi032.stdout:1/331: stat c10 0 2024-06-03T01:21:02.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/335: truncate d1/da/f1a 1551959 0 2024-06-03T01:21:02.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/336: chown d1/da/d28/d3b/d68 496 1 2024-06-03T01:21:02.636 INFO:tasks.workunit.client.0.smithi032.stdout:3/345: mknod de/d29/d2e/d31/d23/d53/d69/c6b 0 2024-06-03T01:21:02.637 INFO:tasks.workunit.client.0.smithi032.stdout:9/460: mkdir d0/d11/d9c 0 2024-06-03T01:21:02.638 INFO:tasks.workunit.client.0.smithi032.stdout:1/332: truncate d18/d2d/f50 2525911 0 2024-06-03T01:21:02.640 INFO:tasks.workunit.client.1.smithi044.stdout:7/911: link d0/d3/d8/d95/d63/d7f/f99 d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/f11f 0 2024-06-03T01:21:02.641 INFO:tasks.workunit.client.0.smithi032.stdout:0/337: symlink d1/da/d28/l69 0 2024-06-03T01:21:02.641 INFO:tasks.workunit.client.0.smithi032.stdout:0/338: readlink d1/da/d28/l69 0 2024-06-03T01:21:02.642 INFO:tasks.workunit.client.1.smithi044.stdout:2/782: sync 2024-06-03T01:21:02.643 INFO:tasks.workunit.client.1.smithi044.stdout:2/783: chown db/d1b/d27/lb2 1021463 1 2024-06-03T01:21:02.643 INFO:tasks.workunit.client.0.smithi032.stdout:3/346: creat de/d29/d2e/d31/d23/d53/f6c x:0 0 0 2024-06-03T01:21:02.644 INFO:tasks.workunit.client.0.smithi032.stdout:9/461: creat d0/d23/d24/f9d x:0 0 0 2024-06-03T01:21:02.645 INFO:tasks.workunit.client.1.smithi044.stdout:7/912: unlink d0/d3/d8/d95/d63/d89/da3/dde/cfb 0 2024-06-03T01:21:02.646 INFO:tasks.workunit.client.1.smithi044.stdout:2/784: mkdir db/d1b/d2a/d2f/dbe/ded/dfa 0 2024-06-03T01:21:02.649 INFO:tasks.workunit.client.0.smithi032.stdout:3/347: mkdir de/d29/d2e/d6d 0 2024-06-03T01:21:02.649 INFO:tasks.workunit.client.0.smithi032.stdout:0/339: truncate d1/f64 201882 0 2024-06-03T01:21:02.650 INFO:tasks.workunit.client.0.smithi032.stdout:6/390: dwrite d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:02.651 INFO:tasks.workunit.client.0.smithi032.stdout:6/391: fdatasync d8/d11/d20/d2b/d39/d19/f1d 0 2024-06-03T01:21:02.651 INFO:tasks.workunit.client.0.smithi032.stdout:0/340: mkdir d1/da/d28/d6a 0 2024-06-03T01:21:02.651 INFO:tasks.workunit.client.0.smithi032.stdout:0/341: read d1/da/d10/f2a [905238,18695] 0 2024-06-03T01:21:02.651 INFO:tasks.workunit.client.0.smithi032.stdout:8/313: dwrite dc/de/f37 [0,4194304] 0 2024-06-03T01:21:02.654 INFO:tasks.workunit.client.1.smithi044.stdout:4/959: getdents d17/d63/dae 0 2024-06-03T01:21:02.656 INFO:tasks.workunit.client.1.smithi044.stdout:2/785: write db/f48 [5416850,128502] 0 2024-06-03T01:21:02.657 INFO:tasks.workunit.client.1.smithi044.stdout:7/913: dread d0/f9f [0,4194304] 0 2024-06-03T01:21:02.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/879: dwrite d2/f3c [0,4194304] 0 2024-06-03T01:21:02.657 INFO:tasks.workunit.client.1.smithi044.stdout:7/914: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fbd zero size 2024-06-03T01:21:02.657 INFO:tasks.workunit.client.0.smithi032.stdout:6/392: mknod d8/c77 0 2024-06-03T01:21:02.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/796: sync 2024-06-03T01:21:02.660 INFO:tasks.workunit.client.1.smithi044.stdout:9/859: sync 2024-06-03T01:21:02.660 INFO:tasks.workunit.client.1.smithi044.stdout:8/933: sync 2024-06-03T01:21:02.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/797: stat df/d13/d39/d3d/d43/fc1 0 2024-06-03T01:21:02.660 INFO:tasks.workunit.client.1.smithi044.stdout:2/786: write db/fee [902249,109710] 0 2024-06-03T01:21:02.662 INFO:tasks.workunit.client.1.smithi044.stdout:5/880: rmdir d2 39 2024-06-03T01:21:02.663 INFO:tasks.workunit.client.1.smithi044.stdout:4/960: dread d17/d1c/fb7 [0,4194304] 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.0.smithi032.stdout:8/314: creat dc/de/d1e/d43/d4e/d59/f69 x:0 0 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/881: write d2/d11/d1e/d21/d24/f69 [559866,58523] 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/882: stat d2/d11/d12/d2b/d4a/ld5 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/883: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/f128 x:0 0 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/884: fdatasync d2/d11/d1e/d21/d24/d93/d9d/f22 0 2024-06-03T01:21:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:8/934: mknod d5/d16/d1d/d32/d109/c138 0 2024-06-03T01:21:02.665 INFO:tasks.workunit.client.0.smithi032.stdout:0/342: read d1/da/f16 [57533,47630] 0 2024-06-03T01:21:02.665 INFO:tasks.workunit.client.0.smithi032.stdout:6/393: creat d8/d11/d20/d3b/d66/d70/f78 x:0 0 0 2024-06-03T01:21:02.667 INFO:tasks.workunit.client.1.smithi044.stdout:3/798: getdents df/d1e/d8a 0 2024-06-03T01:21:02.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/860: getdents d6 0 2024-06-03T01:21:02.667 INFO:tasks.workunit.client.1.smithi044.stdout:3/799: creat df/d13/d1b/d56/f105 x:0 0 0 2024-06-03T01:21:02.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/861: write d6/dd/d51/d16/d27/d5e/d71/dc1/fc4 [667927,44241] 0 2024-06-03T01:21:02.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/862: chown d6/dd/d51/d16/d5f 45359057 1 2024-06-03T01:21:02.668 INFO:tasks.workunit.client.0.smithi032.stdout:6/394: rmdir d8/d11/d20/d2b/d39 39 2024-06-03T01:21:02.671 INFO:tasks.workunit.client.0.smithi032.stdout:6/395: rmdir d8/d11 39 2024-06-03T01:21:02.673 INFO:tasks.workunit.client.1.smithi044.stdout:5/885: rename d2/d11/d1e/d3b/le8 to d2/d11/d12/d2b/d4a/l129 0 2024-06-03T01:21:02.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/935: getdents d5/d16 0 2024-06-03T01:21:02.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/936: chown d5/d20/f9e 1147 1 2024-06-03T01:21:02.676 INFO:tasks.workunit.client.1.smithi044.stdout:3/800: rmdir df/d1e/d2a/d70 39 2024-06-03T01:21:02.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/801: truncate df/d1e/f37 744373 0 2024-06-03T01:21:02.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/802: stat df/f23 0 2024-06-03T01:21:02.680 INFO:tasks.workunit.client.1.smithi044.stdout:9/863: read d6/f84 [1010967,52927] 0 2024-06-03T01:21:02.680 INFO:tasks.workunit.client.1.smithi044.stdout:5/886: getdents d2/d11/d12 0 2024-06-03T01:21:02.682 INFO:tasks.workunit.client.1.smithi044.stdout:3/803: symlink df/d1e/d2a/d9b/l106 0 2024-06-03T01:21:02.683 INFO:tasks.workunit.client.1.smithi044.stdout:3/804: fdatasync df/d13/d1b/d59/fd8 0 2024-06-03T01:21:02.684 INFO:tasks.workunit.client.1.smithi044.stdout:9/864: creat d6/dd/d51/dce/f118 x:0 0 0 2024-06-03T01:21:02.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/915: read d0/d3/d8/d95/d63/d7f/d9c/f83 [2687417,119183] 0 2024-06-03T01:21:02.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/916: dread - d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/de8/df6/f112 zero size 2024-06-03T01:21:02.686 INFO:tasks.workunit.client.0.smithi032.stdout:1/333: dwrite d18/d2d/d5e/f64 [0,4194304] 0 2024-06-03T01:21:02.686 INFO:tasks.workunit.client.0.smithi032.stdout:1/334: fdatasync d18/f1a 0 2024-06-03T01:21:02.687 INFO:tasks.workunit.client.1.smithi044.stdout:3/805: mknod df/d13/de5/c107 0 2024-06-03T01:21:02.687 INFO:tasks.workunit.client.1.smithi044.stdout:3/806: write df/d13/d1b/d4b/d69/f95 [1027880,56311] 0 2024-06-03T01:21:02.688 INFO:tasks.workunit.client.0.smithi032.stdout:9/462: dwrite d0/d23/d25/f8d [0,4194304] 0 2024-06-03T01:21:02.690 INFO:tasks.workunit.client.1.smithi044.stdout:9/865: rmdir d6/dd/d51/d16/d5f 39 2024-06-03T01:21:02.690 INFO:tasks.workunit.client.1.smithi044.stdout:7/917: mkdir d0/d3/d8/d95/d63/d89/d120 0 2024-06-03T01:21:02.690 INFO:tasks.workunit.client.1.smithi044.stdout:3/807: symlink df/d1e/d8a/l108 0 2024-06-03T01:21:02.690 INFO:tasks.workunit.client.1.smithi044.stdout:3/808: dread - df/d1e/d78/dda/fde zero size 2024-06-03T01:21:02.690 INFO:tasks.workunit.client.1.smithi044.stdout:2/787: dread db/f48 [4194304,4194304] 0 2024-06-03T01:21:02.691 INFO:tasks.workunit.client.0.smithi032.stdout:1/335: dread d18/f25 [4194304,4194304] 0 2024-06-03T01:21:02.691 INFO:tasks.workunit.client.0.smithi032.stdout:1/336: dread - d18/f5d zero size 2024-06-03T01:21:02.691 INFO:tasks.workunit.client.0.smithi032.stdout:1/337: chown d18/c20 922825 1 2024-06-03T01:21:02.691 INFO:tasks.workunit.client.0.smithi032.stdout:1/338: chown fa 3680 1 2024-06-03T01:21:02.692 INFO:tasks.workunit.client.1.smithi044.stdout:7/918: symlink d0/d3/da8/de5/d106/l121 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.0.smithi032.stdout:8/315: dwrite dc/fd [0,4194304] 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.0.smithi032.stdout:9/463: rename d0/c4 to d0/d11/d1d/d38/c9e 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/348: dwrite de/f10 [0,4194304] 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/349: write de/f5b [210488,113650] 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/350: creat de/d29/d2e/d31/d23/f6e x:0 0 0 2024-06-03T01:21:02.695 INFO:tasks.workunit.client.1.smithi044.stdout:8/937: dwrite d5/d55/d74/d87/f7b [0,4194304] 0 2024-06-03T01:21:02.696 INFO:tasks.workunit.client.0.smithi032.stdout:0/343: write d1/f64 [182874,22828] 0 2024-06-03T01:21:02.696 INFO:tasks.workunit.client.0.smithi032.stdout:0/344: creat d1/d46/f6b x:0 0 0 2024-06-03T01:21:02.696 INFO:tasks.workunit.client.1.smithi044.stdout:3/809: unlink df/d13/d1b/d56/f105 0 2024-06-03T01:21:02.696 INFO:tasks.workunit.client.0.smithi032.stdout:1/339: unlink d18/d29/d3d/d47/c30 0 2024-06-03T01:21:02.696 INFO:tasks.workunit.client.0.smithi032.stdout:1/340: chown d18/d32/d39/f4a 256536 1 2024-06-03T01:21:02.702 INFO:tasks.workunit.client.1.smithi044.stdout:2/788: creat db/d15/d49/d6c/d72/dce/ffb x:0 0 0 2024-06-03T01:21:02.704 INFO:tasks.workunit.client.0.smithi032.stdout:9/464: unlink d0/d11/d76/c90 0 2024-06-03T01:21:02.705 INFO:tasks.workunit.client.0.smithi032.stdout:8/316: link dc/de/d1e/c23 dc/d25/d3a/c6a 0 2024-06-03T01:21:02.706 INFO:tasks.workunit.client.1.smithi044.stdout:7/919: rmdir d0/d3/d8/d95/d63 39 2024-06-03T01:21:02.707 INFO:tasks.workunit.client.0.smithi032.stdout:0/345: read d1/da/f1a [778877,82133] 0 2024-06-03T01:21:02.707 INFO:tasks.workunit.client.0.smithi032.stdout:0/346: chown d1/da/d28/d3b/d5d/f67 8188179 1 2024-06-03T01:21:02.711 INFO:tasks.workunit.client.0.smithi032.stdout:1/341: getdents d18/d29 0 2024-06-03T01:21:02.712 INFO:tasks.workunit.client.0.smithi032.stdout:8/317: link dc/d25/d28/l4d dc/d14/d31/d3b/l6b 0 2024-06-03T01:21:02.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/810: mkdir df/d1e/d2a/d70/d109 0 2024-06-03T01:21:02.713 INFO:tasks.workunit.client.1.smithi044.stdout:2/789: rmdir db/d15/d49/d6c/d72/d7a/d4e/d7e 39 2024-06-03T01:21:02.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/347: truncate d1/f17 619595 0 2024-06-03T01:21:02.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/348: chown d1/f9 65 1 2024-06-03T01:21:02.714 INFO:tasks.workunit.client.1.smithi044.stdout:2/790: write db/d1b/d2a/d67/d53/f93 [1316097,127089] 0 2024-06-03T01:21:02.714 INFO:tasks.workunit.client.1.smithi044.stdout:2/791: stat db/d1b/d2a/d52/d54/db6/deb 0 2024-06-03T01:21:02.717 INFO:tasks.workunit.client.1.smithi044.stdout:3/811: rename df/d13/d39/d3d/d43/fc1 to df/d13/d6d/f10a 0 2024-06-03T01:21:02.719 INFO:tasks.workunit.client.0.smithi032.stdout:8/318: unlink dc/d25/f47 0 2024-06-03T01:21:02.719 INFO:tasks.workunit.client.0.smithi032.stdout:8/319: fdatasync dc/f15 0 2024-06-03T01:21:02.719 INFO:tasks.workunit.client.0.smithi032.stdout:8/320: creat dc/d25/f6c x:0 0 0 2024-06-03T01:21:02.719 INFO:tasks.workunit.client.0.smithi032.stdout:1/342: rmdir d18/d32/d3c/d62 0 2024-06-03T01:21:02.720 INFO:tasks.workunit.client.1.smithi044.stdout:7/920: getdents d0/d3/d8/d95/d63/d7f/d9c 0 2024-06-03T01:21:02.723 INFO:tasks.workunit.client.1.smithi044.stdout:3/812: creat df/f10b x:0 0 0 2024-06-03T01:21:02.724 INFO:tasks.workunit.client.0.smithi032.stdout:9/465: dread d0/d11/d1d/f47 [0,4194304] 0 2024-06-03T01:21:02.724 INFO:tasks.workunit.client.0.smithi032.stdout:9/466: chown d0/d23/f85 411 1 2024-06-03T01:21:02.724 INFO:tasks.workunit.client.0.smithi032.stdout:9/467: truncate d0/f57 915107 0 2024-06-03T01:21:02.724 INFO:tasks.workunit.client.1.smithi044.stdout:7/921: write d0/d3/d8/d95/d63/f6b [4659791,112876] 0 2024-06-03T01:21:02.725 INFO:tasks.workunit.client.1.smithi044.stdout:7/922: getdents d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d104 0 2024-06-03T01:21:02.726 INFO:tasks.workunit.client.0.smithi032.stdout:8/321: unlink dc/d25/d3a/c6a 0 2024-06-03T01:21:02.726 INFO:tasks.workunit.client.0.smithi032.stdout:8/322: stat f9 0 2024-06-03T01:21:02.726 INFO:tasks.workunit.client.0.smithi032.stdout:8/323: stat dc/de/d1e/d43/d4e/d59 0 2024-06-03T01:21:02.726 INFO:tasks.workunit.client.0.smithi032.stdout:8/324: read f8 [161385,25424] 0 2024-06-03T01:21:02.726 INFO:tasks.workunit.client.0.smithi032.stdout:8/325: chown dc/de/l2c 823895 1 2024-06-03T01:21:02.730 INFO:tasks.workunit.client.1.smithi044.stdout:3/813: creat df/d13/d6d/d9a/db7/f10c x:0 0 0 2024-06-03T01:21:02.733 INFO:tasks.workunit.client.1.smithi044.stdout:7/923: unlink d0/d3/d8/d95/d63/d89/da3/f5e 0 2024-06-03T01:21:02.734 INFO:tasks.workunit.client.0.smithi032.stdout:9/468: dread d0/d11/d76/f7c [0,4194304] 0 2024-06-03T01:21:02.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/351: dwrite de/d29/d2e/d31/f38 [0,4194304] 0 2024-06-03T01:21:02.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/352: fdatasync de/d29/d2e/d31/d24/f25 0 2024-06-03T01:21:02.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/353: fdatasync f8 0 2024-06-03T01:21:02.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/354: fdatasync f9 0 2024-06-03T01:21:02.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/355: truncate de/d29/f3b 4916642 0 2024-06-03T01:21:02.737 INFO:tasks.workunit.client.1.smithi044.stdout:5/887: dwrite d2/d11/d1e/d21/d24/d93/d11e/f123 [4194304,4194304] 0 2024-06-03T01:21:02.738 INFO:tasks.workunit.client.1.smithi044.stdout:5/888: fsync d2/fa 0 2024-06-03T01:21:02.739 INFO:tasks.workunit.client.1.smithi044.stdout:5/889: chown d2/d11/d1e/d21/d24/d9e/l115 42433707 1 2024-06-03T01:21:02.739 INFO:tasks.workunit.client.0.smithi032.stdout:9/469: link d0/d11/d1d/d38/f3e d0/d11/d79/f9f 0 2024-06-03T01:21:02.739 INFO:tasks.workunit.client.0.smithi032.stdout:9/470: truncate d0/d11/d1d/f8f 1382704 0 2024-06-03T01:21:02.744 INFO:tasks.workunit.client.1.smithi044.stdout:8/938: dwrite d5/d55/d74/d87/f99 [0,4194304] 0 2024-06-03T01:21:02.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/866: dwrite d6/dd/d51/d16/d27/d5e/d71/de0/f10a [0,4194304] 0 2024-06-03T01:21:02.747 INFO:tasks.workunit.client.0.smithi032.stdout:9/471: rename d0/d11/f46 to d0/d23/d24/d26/fa0 0 2024-06-03T01:21:02.747 INFO:tasks.workunit.client.1.smithi044.stdout:5/890: link d2/d11/d12/d2b/d4a/d57/l11d d2/d11/d1e/d21/l12a 0 2024-06-03T01:21:02.752 INFO:tasks.workunit.client.0.smithi032.stdout:3/356: write de/d29/d2e/d31/d24/f2f [752111,114349] 0 2024-06-03T01:21:02.753 INFO:tasks.workunit.client.0.smithi032.stdout:9/472: rename d0/d11/f31 to d0/d11/d1d/d38/d4f/fa1 0 2024-06-03T01:21:02.755 INFO:tasks.workunit.client.1.smithi044.stdout:9/867: dread d6/dd/f36 [0,4194304] 0 2024-06-03T01:21:02.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/343: dwrite d18/d27/d2b/f54 [0,4194304] 0 2024-06-03T01:21:02.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/344: dread - d18/d32/d3c/f61 zero size 2024-06-03T01:21:02.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/345: chown d18/c36 128609 1 2024-06-03T01:21:02.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/346: creat d18/d2d/f69 x:0 0 0 2024-06-03T01:21:02.760 INFO:tasks.workunit.client.0.smithi032.stdout:3/357: stat de/c41 0 2024-06-03T01:21:02.760 INFO:tasks.workunit.client.0.smithi032.stdout:3/358: dread - de/d29/d2e/d31/d23/f6e zero size 2024-06-03T01:21:02.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/939: getdents d5/d55/d56/d5c/d68/de5/dea/d79/d88 0 2024-06-03T01:21:02.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/940: chown d5/d20/d47/d64/dd0/l7e 314851 1 2024-06-03T01:21:02.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/941: creat d5/d20/d33/f139 x:0 0 0 2024-06-03T01:21:02.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/942: fsync d5/d16/d1d/d59/f10e 0 2024-06-03T01:21:02.763 INFO:tasks.workunit.client.0.smithi032.stdout:9/473: truncate d0/d11/d1d/d4a/d71/f96 3399362 0 2024-06-03T01:21:02.764 INFO:tasks.workunit.client.1.smithi044.stdout:2/792: dwrite db/d15/d49/d7b/f88 [4194304,4194304] 0 2024-06-03T01:21:02.766 INFO:tasks.workunit.client.0.smithi032.stdout:0/349: dwrite d1/f21 [0,4194304] 0 2024-06-03T01:21:02.766 INFO:tasks.workunit.client.0.smithi032.stdout:0/350: chown d1/d46 452 1 2024-06-03T01:21:02.767 INFO:tasks.workunit.client.1.smithi044.stdout:5/891: dread d2/d11/d1e/d21/d24/f77 [0,4194304] 0 2024-06-03T01:21:02.769 INFO:tasks.workunit.client.0.smithi032.stdout:1/347: truncate d18/f1a 1716860 0 2024-06-03T01:21:02.769 INFO:tasks.workunit.client.1.smithi044.stdout:8/943: creat d5/d16/d81/f13a x:0 0 0 2024-06-03T01:21:02.770 INFO:tasks.workunit.client.0.smithi032.stdout:8/326: dwrite dc/d25/d28/d33/f48 [0,4194304] 0 2024-06-03T01:21:02.770 INFO:tasks.workunit.client.0.smithi032.stdout:3/359: symlink de/d29/d2e/l6f 0 2024-06-03T01:21:02.770 INFO:tasks.workunit.client.0.smithi032.stdout:3/360: truncate de/f57 440832 0 2024-06-03T01:21:02.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/793: creat db/d1b/d2a/d2f/dbe/ded/dfa/ffc x:0 0 0 2024-06-03T01:21:02.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.771+0000 7f9a1df75700 1 -- 172.21.15.32:0/1153148911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 msgr2=0x7f9a18073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.771+0000 7f9a1df75700 1 --2- 172.21.15.32:0/1153148911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a18073220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009910 tx=0x7f9a08009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:02.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.772+0000 7f9a1df75700 1 -- 172.21.15.32:0/1153148911 shutdown_connections 2024-06-03T01:21:02.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.772+0000 7f9a1df75700 1 --2- 172.21.15.32:0/1153148911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 0x7f9a18073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.772+0000 7f9a1df75700 1 --2- 172.21.15.32:0/1153148911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a18073220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.772+0000 7f9a1df75700 1 -- 172.21.15.32:0/1153148911 >> 172.21.15.32:0/1153148911 conn(0x7f9a180fc230 msgr2=0x7f9a180fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:02.773 INFO:tasks.workunit.client.1.smithi044.stdout:7/924: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/faa [0,4194304] 0 2024-06-03T01:21:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.777+0000 7f9a1df75700 1 -- 172.21.15.32:0/1153148911 shutdown_connections 2024-06-03T01:21:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.777+0000 7f9a1df75700 1 -- 172.21.15.32:0/1153148911 wait complete. 2024-06-03T01:21:02.778 INFO:tasks.workunit.client.1.smithi044.stdout:7/925: rmdir d0/d3/d8 39 2024-06-03T01:21:02.778 INFO:tasks.workunit.client.1.smithi044.stdout:7/926: chown d0/d3/d8/d32/c8c 1582788 1 2024-06-03T01:21:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.779+0000 7f9a1df75700 1 Processor -- start 2024-06-03T01:21:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.779+0000 7f9a1df75700 1 -- start start 2024-06-03T01:21:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.780+0000 7f9a1df75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 0x7f9a181a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.780+0000 7f9a1df75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.780+0000 7f9a1df75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181a60c0 con 0x7f9a18074dc0 2024-06-03T01:21:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.780+0000 7f9a1df75700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181a6200 con 0x7f9a180737f0 2024-06-03T01:21:02.780 INFO:tasks.workunit.client.0.smithi032.stdout:9/474: creat d0/d11/d1d/d4a/fa2 x:0 0 0 2024-06-03T01:21:02.781 INFO:tasks.workunit.client.0.smithi032.stdout:9/475: write d0/d23/f3d [4703344,86289] 0 2024-06-03T01:21:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.780+0000 7f9a177fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 0x7f9a181a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.781+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.781+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49096/0 (socket says 172.21.15.32:49096) 2024-06-03T01:21:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.781+0000 7f9a16ffd700 1 -- 172.21.15.32:0/385645276 learned_addr learned my addr 172.21.15.32:0/385645276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.782+0000 7f9a16ffd700 1 -- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 msgr2=0x7f9a181a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.782+0000 7f9a16ffd700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 0x7f9a181a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.782+0000 7f9a16ffd700 1 -- 172.21.15.32:0/385645276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080095f0 con 0x7f9a18074dc0 2024-06-03T01:21:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.782+0000 7f9a16ffd700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b480 tx=0x7f9a0c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.783+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0c0118c0 con 0x7f9a18074dc0 2024-06-03T01:21:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.783+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c010380 con 0x7f9a18074dc0 2024-06-03T01:21:02.784 INFO:tasks.workunit.client.1.smithi044.stdout:5/892: symlink d2/d11/d12/d2b/d4a/d57/l12b 0 2024-06-03T01:21:02.784 INFO:tasks.workunit.client.1.smithi044.stdout:5/893: write d2/d11/d3e/dce/fec [1467091,118820] 0 2024-06-03T01:21:02.784 INFO:tasks.workunit.client.1.smithi044.stdout:3/814: dwrite df/f17 [0,4194304] 0 2024-06-03T01:21:02.785 INFO:tasks.workunit.client.1.smithi044.stdout:3/815: creat df/d1e/d3a/d84/f10d x:0 0 0 2024-06-03T01:21:02.785 INFO:tasks.workunit.client.1.smithi044.stdout:3/816: creat df/d1e/d3a/d84/f10e x:0 0 0 2024-06-03T01:21:02.785 INFO:tasks.workunit.client.1.smithi044.stdout:5/894: mkdir d2/d11/d12/d2b/d4a/d57/d106/d12c 0 2024-06-03T01:21:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.784+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0c00f4f0 con 0x7f9a18074dc0 2024-06-03T01:21:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.785+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a181aa8c0 con 0x7f9a18074dc0 2024-06-03T01:21:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.785+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a181aad30 con 0x7f9a18074dc0 2024-06-03T01:21:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.788+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) v1 ==== 46426+0+0 (secure 0 0 0) 0x7f9a0c0104f0 con 0x7f9a18074dc0 2024-06-03T01:21:02.788 INFO:tasks.workunit.client.0.smithi032.stdout:9/476: dread d0/d23/d24/d26/fa0 [0,4194304] 0 2024-06-03T01:21:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.788+0000 7f9a14ff9700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 0x7f9a0003b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.788+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f9a0c04cb70 con 0x7f9a18074dc0 2024-06-03T01:21:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.788+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1806b040 con 0x7f9a18074dc0 2024-06-03T01:21:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.789+0000 7f9a177fe700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 0x7f9a0003b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.789+0000 7f9a177fe700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 0x7f9a0003b700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009350 tx=0x7f9a0800b540 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:02.790 INFO:tasks.workunit.client.0.smithi032.stdout:0/351: mkdir d1/da/d10/d29/d6c 0 2024-06-03T01:21:02.790 INFO:tasks.workunit.client.1.smithi044.stdout:9/868: dwrite d6/dd/f14 [0,4194304] 0 2024-06-03T01:21:02.791 INFO:tasks.workunit.client.1.smithi044.stdout:2/794: dwrite db/d1b/d2a/f43 [0,4194304] 0 2024-06-03T01:21:02.792 INFO:tasks.workunit.client.0.smithi032.stdout:9/477: dread d0/d23/d24/f72 [0,4194304] 0 2024-06-03T01:21:02.792 INFO:tasks.workunit.client.0.smithi032.stdout:1/348: dwrite d18/d32/f4b [0,4194304] 0 2024-06-03T01:21:02.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/817: symlink df/d1e/d2a/l10f 0 2024-06-03T01:21:02.795 INFO:tasks.workunit.client.1.smithi044.stdout:5/895: rmdir d2/d11c 39 2024-06-03T01:21:02.795 INFO:tasks.workunit.client.1.smithi044.stdout:5/896: readlink d2/d11/d12/d2b/lde 0 2024-06-03T01:21:02.795 INFO:tasks.workunit.client.1.smithi044.stdout:5/897: write d2/d11/d1e/d21/d24/fa7 [1969087,119212] 0 2024-06-03T01:21:02.796 INFO:tasks.workunit.client.0.smithi032.stdout:8/327: unlink dc/de/l2a 0 2024-06-03T01:21:02.798 INFO:tasks.workunit.client.1.smithi044.stdout:9/869: creat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d103/f119 x:0 0 0 2024-06-03T01:21:02.799 INFO:tasks.workunit.client.1.smithi044.stdout:9/870: fsync d6/dd/d51/dce/f106 0 2024-06-03T01:21:02.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:02.799+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f9a0c015070 con 0x7f9a18074dc0 2024-06-03T01:21:02.799 INFO:tasks.workunit.client.0.smithi032.stdout:3/361: rename fa to de/d29/f70 0 2024-06-03T01:21:02.799 INFO:tasks.workunit.client.0.smithi032.stdout:3/362: fdatasync de/d29/d2e/d31/d23/f5a 0 2024-06-03T01:21:02.800 INFO:tasks.workunit.client.1.smithi044.stdout:2/795: mknod db/d1b/cfd 0 2024-06-03T01:21:02.801 INFO:tasks.workunit.client.1.smithi044.stdout:2/796: write db/d1b/d2a/d2f/dbe/fe2 [156492,75212] 0 2024-06-03T01:21:02.801 INFO:tasks.workunit.client.1.smithi044.stdout:2/797: chown db/f48 189323 1 2024-06-03T01:21:02.802 INFO:tasks.workunit.client.1.smithi044.stdout:3/818: symlink df/d1e/d3a/d84/l110 0 2024-06-03T01:21:02.803 INFO:tasks.workunit.client.1.smithi044.stdout:5/898: mknod d2/d11/d12/d2b/d108/c12d 0 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.1.smithi044.stdout:5/899: chown d2/d11/d12/dad/lf0 31 1 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.0.smithi032.stdout:0/352: creat d1/d46/d3a/f6d x:0 0 0 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/478: getdents d0/d11/d1d/d4a/d71 0 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/479: chown d0/d23 11550 1 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/480: truncate d0/d11/d1d/f99 1048428 0 2024-06-03T01:21:02.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/481: fsync d0/f57 0 2024-06-03T01:21:02.805 INFO:tasks.workunit.client.0.smithi032.stdout:9/482: chown d0/d23/d24/d26/c8b 2189 1 2024-06-03T01:21:02.805 INFO:tasks.workunit.client.1.smithi044.stdout:4/961: sync 2024-06-03T01:21:02.808 INFO:tasks.workunit.client.1.smithi044.stdout:5/900: dread d2/d11/d1e/d21/d24/f69 [0,4194304] 0 2024-06-03T01:21:02.811 INFO:tasks.workunit.client.1.smithi044.stdout:9/871: rename d6/dd/d51/d16/d61 to d6/d38/d11a 0 2024-06-03T01:21:02.813 INFO:tasks.workunit.client.0.smithi032.stdout:1/349: mkdir d18/d29/d3d/d6a 0 2024-06-03T01:21:02.814 INFO:tasks.workunit.client.0.smithi032.stdout:1/350: dread - d18/d29/f5c zero size 2024-06-03T01:21:02.815 INFO:tasks.workunit.client.1.smithi044.stdout:2/798: truncate db/d15/d49/d6c/d72/d7a/f8d 201808 0 2024-06-03T01:21:02.816 INFO:tasks.workunit.client.1.smithi044.stdout:2/799: write db/d15/d49/d82/db5/ff4 [46469,43922] 0 2024-06-03T01:21:02.816 INFO:tasks.workunit.client.1.smithi044.stdout:3/819: mkdir df/d1e/d2a/d111 0 2024-06-03T01:21:02.816 INFO:tasks.workunit.client.0.smithi032.stdout:8/328: creat dc/de/d1e/d43/d4e/f6d x:0 0 0 2024-06-03T01:21:02.817 INFO:tasks.workunit.client.1.smithi044.stdout:4/962: mknod d17/d1c/d70/d7d/c145 0 2024-06-03T01:21:02.818 INFO:tasks.workunit.client.1.smithi044.stdout:5/901: creat d2/d11c/df5/f12e x:0 0 0 2024-06-03T01:21:02.818 INFO:tasks.workunit.client.0.smithi032.stdout:3/363: rename de/c41 to de/d29/d2e/d31/d3c/d50/d56/c71 0 2024-06-03T01:21:02.819 INFO:tasks.workunit.client.1.smithi044.stdout:2/800: mknod db/d1b/d5d/cfe 0 2024-06-03T01:21:02.821 INFO:tasks.workunit.client.0.smithi032.stdout:1/351: mkdir d18/d3f/d5f/d6b 0 2024-06-03T01:21:02.821 INFO:tasks.workunit.client.0.smithi032.stdout:1/352: chown d18/l21 16722028 1 2024-06-03T01:21:02.821 INFO:tasks.workunit.client.1.smithi044.stdout:9/872: write d6/dd/d51/d16/f7b [622374,33871] 0 2024-06-03T01:21:02.821 INFO:tasks.workunit.client.1.smithi044.stdout:9/873: chown d6/dd/d45/c75 15829 1 2024-06-03T01:21:02.822 INFO:tasks.workunit.client.0.smithi032.stdout:8/329: symlink dc/de/d1e/d43/d4e/d59/l6e 0 2024-06-03T01:21:02.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/801: creat db/d15/d49/d6c/df3/fff x:0 0 0 2024-06-03T01:21:02.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/802: chown db/d15/d49/d6c/d72/d7a/d4e/c7d 484 1 2024-06-03T01:21:02.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/803: fdatasync db/d15/d49/d6c/d72/d7a/f8d 0 2024-06-03T01:21:02.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/804: write db/d1b/d27/d79/d91/f92 [288989,121258] 0 2024-06-03T01:21:02.824 INFO:tasks.workunit.client.0.smithi032.stdout:5/429: sync 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/396: sync 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:2/391: sync 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:7/359: sync 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:2/392: getdents d5/da/d1a/d69/d83 0 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:7/360: chown dc/d12/c3b 2211524 1 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:4/375: sync 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:7/361: fsync dc/d12/d22/d42/f50 0 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:4/376: creat d4/db/da/d2d/d27/f8a x:0 0 0 2024-06-03T01:21:02.825 INFO:tasks.workunit.client.0.smithi032.stdout:7/362: creat dc/d12/d24/f6c x:0 0 0 2024-06-03T01:21:02.828 INFO:tasks.workunit.client.0.smithi032.stdout:3/364: symlink de/d29/d2e/d31/d3c/d50/d56/l72 0 2024-06-03T01:21:02.828 INFO:tasks.workunit.client.0.smithi032.stdout:1/353: unlink d18/f25 0 2024-06-03T01:21:02.830 INFO:tasks.workunit.client.0.smithi032.stdout:8/330: truncate dc/f15 4267762 0 2024-06-03T01:21:02.830 INFO:tasks.workunit.client.0.smithi032.stdout:8/331: chown dc/d14/d31/l45 213871 1 2024-06-03T01:21:02.834 INFO:tasks.workunit.client.0.smithi032.stdout:0/353: dwrite d1/da/d28/d3b/d5d/f65 [0,4194304] 0 2024-06-03T01:21:02.835 INFO:tasks.workunit.client.0.smithi032.stdout:5/430: mknod da/c83 0 2024-06-03T01:21:02.839 INFO:tasks.workunit.client.1.smithi044.stdout:2/805: write db/d1b/d2a/d52/fb4 [6921412,36581] 0 2024-06-03T01:21:02.839 INFO:tasks.workunit.client.1.smithi044.stdout:2/806: truncate db/d1b/f99 4339300 0 2024-06-03T01:21:02.841 INFO:tasks.workunit.client.0.smithi032.stdout:6/397: unlink d8/d11/d20/d36/c43 0 2024-06-03T01:21:02.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/393: truncate d5/f19 5576342 0 2024-06-03T01:21:02.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/394: chown d5/d38/d46/d53/d57/d6d 535273 1 2024-06-03T01:21:02.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/395: chown d5/d38/d46/d53/d57 1147 1 2024-06-03T01:21:02.844 INFO:tasks.workunit.client.1.smithi044.stdout:2/807: read db/d1b/d27/f32 [2129614,4218] 0 2024-06-03T01:21:02.845 INFO:tasks.workunit.client.0.smithi032.stdout:7/363: fdatasync dc/d12/d1b/f21 0 2024-06-03T01:21:02.846 INFO:tasks.workunit.client.0.smithi032.stdout:1/354: symlink d18/d2d/d5e/l6c 0 2024-06-03T01:21:02.846 INFO:tasks.workunit.client.0.smithi032.stdout:1/355: chown d18/d3f/c46 109 1 2024-06-03T01:21:02.847 INFO:tasks.workunit.client.0.smithi032.stdout:3/365: read f8 [8248425,23889] 0 2024-06-03T01:21:02.851 INFO:tasks.workunit.client.1.smithi044.stdout:4/963: dwrite d17/d1c/d75/d7a/db8/d35/f4f [4194304,4194304] 0 2024-06-03T01:21:02.852 INFO:tasks.workunit.client.0.smithi032.stdout:7/364: write dc/d12/d24/d33/f4b [3142256,34028] 0 2024-06-03T01:21:02.852 INFO:tasks.workunit.client.0.smithi032.stdout:7/365: chown dc/d12/d2a/d2f/d63/d69/f5f 5 1 2024-06-03T01:21:02.853 INFO:tasks.workunit.client.0.smithi032.stdout:5/431: unlink da/dd/df/d10/d16/d18/d1a/d30/d58/l62 0 2024-06-03T01:21:02.853 INFO:tasks.workunit.client.0.smithi032.stdout:5/432: stat da/dd/df/d10/d16/d25/l45 0 2024-06-03T01:21:02.854 INFO:tasks.workunit.client.0.smithi032.stdout:2/396: truncate d5/f25 2555552 0 2024-06-03T01:21:02.854 INFO:tasks.workunit.client.0.smithi032.stdout:2/397: getdents d5/da/d1a/d2a/d49/d7e/d86 0 2024-06-03T01:21:02.856 INFO:tasks.workunit.client.1.smithi044.stdout:3/820: dwrite df/d13/d1b/f38 [0,4194304] 0 2024-06-03T01:21:02.857 INFO:tasks.workunit.client.1.smithi044.stdout:4/964: creat d17/d1c/d75/d7a/db3/de9/f146 x:0 0 0 2024-06-03T01:21:02.858 INFO:tasks.workunit.client.0.smithi032.stdout:6/398: dread f3 [4194304,4194304] 0 2024-06-03T01:21:02.858 INFO:tasks.workunit.client.0.smithi032.stdout:3/366: truncate de/d29/d2e/f37 2738429 0 2024-06-03T01:21:02.858 INFO:tasks.workunit.client.0.smithi032.stdout:4/377: dwrite d4/db/da/f15 [0,4194304] 0 2024-06-03T01:21:02.859 INFO:tasks.workunit.client.0.smithi032.stdout:0/354: getdents d1/d46 0 2024-06-03T01:21:02.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/367: dread de/f57 [0,4194304] 0 2024-06-03T01:21:02.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/368: read - de/d29/d2e/d31/d3c/f54 zero size 2024-06-03T01:21:02.863 INFO:tasks.workunit.client.1.smithi044.stdout:3/821: rename fd to df/d13/d39/d3d/d43/d71/dd7/f112 0 2024-06-03T01:21:02.865 INFO:tasks.workunit.client.1.smithi044.stdout:3/822: write df/d13/d1b/f73 [739851,120052] 0 2024-06-03T01:21:02.865 INFO:tasks.workunit.client.1.smithi044.stdout:3/823: stat df/d13/d39/d3d/d43/d71 0 2024-06-03T01:21:02.866 INFO:tasks.workunit.client.0.smithi032.stdout:5/433: creat da/dd/df/d10/d16/d18/d35/d70/f84 x:0 0 0 2024-06-03T01:21:02.867 INFO:tasks.workunit.client.0.smithi032.stdout:2/398: rmdir d5/da/d1a 39 2024-06-03T01:21:02.868 INFO:tasks.workunit.client.1.smithi044.stdout:3/824: creat df/d13/d1b/d4b/d69/dd1/df7/f113 x:0 0 0 2024-06-03T01:21:02.868 INFO:tasks.workunit.client.1.smithi044.stdout:3/825: truncate df/d13/d1b/d59/fd8 946888 0 2024-06-03T01:21:02.868 INFO:tasks.workunit.client.1.smithi044.stdout:3/826: truncate df/d13/d1b/d4b/fea 1049622 0 2024-06-03T01:21:02.868 INFO:tasks.workunit.client.1.smithi044.stdout:3/827: write df/d13/d39/d3d/d43/d71/f83 [5027929,6244] 0 2024-06-03T01:21:02.868 INFO:tasks.workunit.client.1.smithi044.stdout:3/828: fdatasync df/d13/d1b/f5e 0 2024-06-03T01:21:02.871 INFO:tasks.workunit.client.1.smithi044.stdout:3/829: symlink df/d13/d6d/d9a/db7/l114 0 2024-06-03T01:21:02.871 INFO:tasks.workunit.client.0.smithi032.stdout:4/378: creat d4/d7f/f8b x:0 0 0 2024-06-03T01:21:02.873 INFO:tasks.workunit.client.0.smithi032.stdout:3/369: dread f8 [0,4194304] 0 2024-06-03T01:21:02.873 INFO:tasks.workunit.client.1.smithi044.stdout:3/830: creat df/d13/d1b/d4b/d69/dd1/f115 x:0 0 0 2024-06-03T01:21:02.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/355: truncate d1/f9 4544164 0 2024-06-03T01:21:02.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/356: creat d1/da/d10/d53/f6e x:0 0 0 2024-06-03T01:21:02.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/357: creat d1/da/d10/f6f x:0 0 0 2024-06-03T01:21:02.875 INFO:tasks.workunit.client.1.smithi044.stdout:3/831: mkdir df/d1e/d78/da4/d116 0 2024-06-03T01:21:02.879 INFO:tasks.workunit.client.1.smithi044.stdout:3/832: mknod df/d13/d6d/d9a/dad/df5/c117 0 2024-06-03T01:21:02.879 INFO:tasks.workunit.client.1.smithi044.stdout:3/833: creat df/d13/d1b/d4b/d69/dd1/f118 x:0 0 0 2024-06-03T01:21:02.880 INFO:tasks.workunit.client.0.smithi032.stdout:3/370: write de/d29/d2e/d31/d3c/f61 [2077594,93283] 0 2024-06-03T01:21:02.880 INFO:tasks.workunit.client.0.smithi032.stdout:3/371: truncate de/d29/d2e/f45 3777422 0 2024-06-03T01:21:02.880 INFO:tasks.workunit.client.0.smithi032.stdout:3/372: fdatasync de/d29/d2e/d31/d3c/d50/d56/f67 0 2024-06-03T01:21:02.880 INFO:tasks.workunit.client.0.smithi032.stdout:3/373: read - de/d29/d2e/d31/d3c/f54 zero size 2024-06-03T01:21:02.881 INFO:tasks.workunit.client.0.smithi032.stdout:8/332: dwrite dc/de/d1e/d43/d4e/f6d [0,4194304] 0 2024-06-03T01:21:02.882 INFO:tasks.workunit.client.0.smithi032.stdout:1/356: unlink ce 0 2024-06-03T01:21:02.882 INFO:tasks.workunit.client.0.smithi032.stdout:1/357: write d18/f5d [172761,33901] 0 2024-06-03T01:21:02.886 INFO:tasks.workunit.client.1.smithi044.stdout:3/834: mkdir df/d13/d1b/d56/d119 0 2024-06-03T01:21:02.887 INFO:tasks.workunit.client.0.smithi032.stdout:5/434: write da/dd/df/d10/d16/d18/d1a/f6d [2693183,3919] 0 2024-06-03T01:21:02.897 INFO:tasks.workunit.client.1.smithi044.stdout:5/902: dwrite d2/d11/d1e/d21/d24/d93/d9d/f22 [0,4194304] 0 2024-06-03T01:21:02.900 INFO:tasks.workunit.client.0.smithi032.stdout:7/366: rmdir dc/d12/d24 39 2024-06-03T01:21:02.901 INFO:tasks.workunit.client.1.smithi044.stdout:3/835: dread df/d1e/f45 [0,4194304] 0 2024-06-03T01:21:02.901 INFO:tasks.workunit.client.1.smithi044.stdout:9/874: dwrite d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/fc5 [0,4194304] 0 2024-06-03T01:21:02.902 INFO:tasks.workunit.client.1.smithi044.stdout:5/903: read d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d9a/fc1 [62958,26130] 0 2024-06-03T01:21:02.902 INFO:tasks.workunit.client.1.smithi044.stdout:5/904: write d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f7e [117803,119086] 0 2024-06-03T01:21:02.903 INFO:tasks.workunit.client.0.smithi032.stdout:4/379: dread d4/db/da/f2c [0,4194304] 0 2024-06-03T01:21:02.903 INFO:tasks.workunit.client.0.smithi032.stdout:0/358: mknod d1/da/d10/c70 0 2024-06-03T01:21:02.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/399: dwrite d5/da/d1a/d2a/d49/d7e/f54 [0,4194304] 0 2024-06-03T01:21:02.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/400: getdents d5/da/d1a/d2a/d49/d7e/d86 0 2024-06-03T01:21:02.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/401: creat d5/da/d1a/d2a/d49/f88 x:0 0 0 2024-06-03T01:21:02.905 INFO:tasks.workunit.client.0.smithi032.stdout:6/399: dwrite d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:02.905 INFO:tasks.workunit.client.1.smithi044.stdout:4/965: dwrite d17/d1c/d75/d7a/db8/d28/d2b/d8e/f107 [0,4194304] 0 2024-06-03T01:21:02.908 INFO:tasks.workunit.client.0.smithi032.stdout:3/374: creat de/d29/d2e/d31/d24/f73 x:0 0 0 2024-06-03T01:21:02.908 INFO:tasks.workunit.client.0.smithi032.stdout:3/375: chown de/d29/d2e/l6f 22802 1 2024-06-03T01:21:02.908 INFO:tasks.workunit.client.0.smithi032.stdout:3/376: creat de/d29/f74 x:0 0 0 2024-06-03T01:21:02.908 INFO:tasks.workunit.client.0.smithi032.stdout:3/377: chown de/d29/l51 3 1 2024-06-03T01:21:02.908 INFO:tasks.workunit.client.0.smithi032.stdout:3/378: fsync f5 0 2024-06-03T01:21:02.913 INFO:tasks.workunit.client.1.smithi044.stdout:3/836: write df/d13/d39/d3d/d43/d71/f83 [539796,41798] 0 2024-06-03T01:21:02.914 INFO:tasks.workunit.client.0.smithi032.stdout:8/333: rename dc/d14/d31/d3b/l5c to dc/d25/d28/d58/l6f 0 2024-06-03T01:21:02.916 INFO:tasks.workunit.client.1.smithi044.stdout:4/966: dread d17/d1c/d75/d7a/db8/f9b [0,4194304] 0 2024-06-03T01:21:02.917 INFO:tasks.workunit.client.0.smithi032.stdout:1/358: truncate d18/d29/d3d/d47/f51 185853 0 2024-06-03T01:21:02.920 INFO:tasks.workunit.client.1.smithi044.stdout:9/875: rename d6/d38/cb5 to d6/dd/d51/d16/d27/d5e/da0/c11b 0 2024-06-03T01:21:02.923 INFO:tasks.workunit.client.0.smithi032.stdout:4/380: dwrite d4/db/da/f2f [0,4194304] 0 2024-06-03T01:21:02.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/837: truncate df/d13/d39/d3d/d43/d71/dd7/f112 1026296 0 2024-06-03T01:21:02.929 INFO:tasks.workunit.client.1.smithi044.stdout:4/967: link d17/d1c/d9c/ff3 d17/d1c/d75/d7a/db8/d28/d2b/d8e/f147 0 2024-06-03T01:21:02.929 INFO:tasks.workunit.client.1.smithi044.stdout:5/905: truncate d2/d11/d1e/d21/d24/d93/d9d/d28/d66/f8e 2058241 0 2024-06-03T01:21:02.930 INFO:tasks.workunit.client.0.smithi032.stdout:8/334: dread dc/f13 [0,4194304] 0 2024-06-03T01:21:02.930 INFO:tasks.workunit.client.0.smithi032.stdout:0/359: mknod d1/da/d28/d3b/d68/c71 0 2024-06-03T01:21:02.930 INFO:tasks.workunit.client.0.smithi032.stdout:7/367: mknod dc/d12/d2a/d2f/c6d 0 2024-06-03T01:21:02.930 INFO:tasks.workunit.client.1.smithi044.stdout:9/876: creat d6/dd/d51/d16/d27/d5e/d71/d89/f11c x:0 0 0 2024-06-03T01:21:02.932 INFO:tasks.workunit.client.1.smithi044.stdout:5/906: dread d2/d11/d12/d2b/f58 [0,4194304] 0 2024-06-03T01:21:02.933 INFO:tasks.workunit.client.0.smithi032.stdout:2/402: mkdir d5/d38/d46/d5c/d6c/d89 0 2024-06-03T01:21:02.934 INFO:tasks.workunit.client.0.smithi032.stdout:6/400: symlink d8/d11/l79 0 2024-06-03T01:21:02.937 INFO:tasks.workunit.client.1.smithi044.stdout:4/968: rename d17/d1c/d75/d7a/db8/d35/d117 to d17/d1c/de0/d148 0 2024-06-03T01:21:02.937 INFO:tasks.workunit.client.1.smithi044.stdout:4/969: write d17/d1c/d75/d7a/db8/db9/f13e [559899,64136] 0 2024-06-03T01:21:02.937 INFO:tasks.workunit.client.1.smithi044.stdout:4/970: chown d17/d1c/d7e/ffa 1 1 2024-06-03T01:21:02.939 INFO:tasks.workunit.client.1.smithi044.stdout:5/907: symlink d2/d11/d12/l12f 0 2024-06-03T01:21:02.939 INFO:tasks.workunit.client.0.smithi032.stdout:3/379: mknod de/d29/d2e/c75 0 2024-06-03T01:21:02.939 INFO:tasks.workunit.client.0.smithi032.stdout:3/380: readlink de/d29/d2e/d31/l22 0 2024-06-03T01:21:02.943 INFO:tasks.workunit.client.1.smithi044.stdout:9/877: rename d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/fe8 to d6/dd/f11d 0 2024-06-03T01:21:02.943 INFO:tasks.workunit.client.1.smithi044.stdout:9/878: read d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [1774410,19612] 0 2024-06-03T01:21:02.943 INFO:tasks.workunit.client.1.smithi044.stdout:9/879: truncate d6/dd/d51/d16/d27/d5e/db7/de1/f114 568915 0 2024-06-03T01:21:02.943 INFO:tasks.workunit.client.1.smithi044.stdout:9/880: write d6/dd/d51/d16/d27/d5e/d71/d89/f112 [1039594,94551] 0 2024-06-03T01:21:02.950 INFO:tasks.workunit.client.0.smithi032.stdout:3/381: write f9 [1394389,127095] 0 2024-06-03T01:21:02.950 INFO:tasks.workunit.client.0.smithi032.stdout:5/435: rmdir da/dd 39 2024-06-03T01:21:02.952 INFO:tasks.workunit.client.1.smithi044.stdout:3/838: dwrite df/f19 [0,4194304] 0 2024-06-03T01:21:02.953 INFO:tasks.workunit.client.0.smithi032.stdout:4/381: mkdir d4/db/da/d2d/d27/d8c 0 2024-06-03T01:21:02.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/360: mknod d1/da/d10/d29/d6c/c72 0 2024-06-03T01:21:02.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/361: creat d1/da/d10/d29/f73 x:0 0 0 2024-06-03T01:21:02.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/362: write d1/d46/f48 [4750805,75942] 0 2024-06-03T01:21:02.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/363: write d1/f64 [231897,74470] 0 2024-06-03T01:21:02.958 INFO:tasks.workunit.client.1.smithi044.stdout:7/927: sync 2024-06-03T01:21:02.958 INFO:tasks.workunit.client.1.smithi044.stdout:8/944: sync 2024-06-03T01:21:02.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/403: getdents d5/d38/d46/d53/d57 0 2024-06-03T01:21:02.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/404: write d5/da/f39 [1437461,85601] 0 2024-06-03T01:21:02.961 INFO:tasks.workunit.client.0.smithi032.stdout:3/382: dread de/d29/d2e/f4d [0,4194304] 0 2024-06-03T01:21:02.967 INFO:tasks.workunit.client.0.smithi032.stdout:6/401: creat d8/d55/f7a x:0 0 0 2024-06-03T01:21:02.969 INFO:tasks.workunit.client.1.smithi044.stdout:9/881: dread f4 [0,4194304] 0 2024-06-03T01:21:02.969 INFO:tasks.workunit.client.1.smithi044.stdout:9/882: stat d6/dd/l93 0 2024-06-03T01:21:02.969 INFO:tasks.workunit.client.1.smithi044.stdout:9/883: chown d6/d38/d11a/dd9/lf6 6242697 1 2024-06-03T01:21:02.970 INFO:tasks.workunit.client.1.smithi044.stdout:9/884: chown d6/dd/d51/d16/d5f/d8b/d8d/dd8 7 1 2024-06-03T01:21:02.970 INFO:tasks.workunit.client.1.smithi044.stdout:9/885: dread - d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d103/f119 zero size 2024-06-03T01:21:02.971 INFO:tasks.workunit.client.0.smithi032.stdout:5/436: mknod da/dd/df/d10/d16/d18/d1a/d30/d51/c85 0 2024-06-03T01:21:02.973 INFO:tasks.workunit.client.1.smithi044.stdout:4/971: dwrite d17/d1c/d70/d7d/fa0 [0,4194304] 0 2024-06-03T01:21:02.974 INFO:tasks.workunit.client.0.smithi032.stdout:1/359: dwrite f17 [8388608,4194304] 0 2024-06-03T01:21:02.974 INFO:tasks.workunit.client.0.smithi032.stdout:1/360: write d18/d3f/f55 [5179014,89658] 0 2024-06-03T01:21:02.975 INFO:tasks.workunit.client.0.smithi032.stdout:6/402: dread d8/d11/d20/d31/f45 [0,4194304] 0 2024-06-03T01:21:02.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/945: rmdir d5/d55 39 2024-06-03T01:21:02.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/928: write d0/d3/d8/d95/d63/d7f/f99 [1669572,19865] 0 2024-06-03T01:21:02.976 INFO:tasks.workunit.client.1.smithi044.stdout:9/886: symlink d6/d38/d11a/dd9/l11e 0 2024-06-03T01:21:02.979 INFO:tasks.workunit.client.0.smithi032.stdout:3/383: fsync de/d29/d2e/f37 0 2024-06-03T01:21:02.979 INFO:tasks.workunit.client.0.smithi032.stdout:3/384: read - de/d29/d2e/d31/d3c/f54 zero size 2024-06-03T01:21:02.982 INFO:tasks.workunit.client.0.smithi032.stdout:8/335: mknod dc/d14/c70 0 2024-06-03T01:21:02.985 INFO:tasks.workunit.client.0.smithi032.stdout:0/364: link d1/da/d10/f2a d1/da/f74 0 2024-06-03T01:21:02.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/365: write d1/da/d10/f2d [2256323,80898] 0 2024-06-03T01:21:02.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/366: fdatasync d1/f21 0 2024-06-03T01:21:02.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/367: chown d1/da/c11 2124535 1 2024-06-03T01:21:02.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/361: dread d18/f1c [0,4194304] 0 2024-06-03T01:21:02.986 INFO:tasks.workunit.client.0.smithi032.stdout:7/368: getdents dc/dd 0 2024-06-03T01:21:02.989 INFO:tasks.workunit.client.0.smithi032.stdout:2/405: rmdir d5/da/d1a/d69 39 2024-06-03T01:21:02.989 INFO:tasks.workunit.client.0.smithi032.stdout:2/406: chown d5/da/d1a/d2a/d49/d7e/f75 9401 1 2024-06-03T01:21:02.992 INFO:tasks.workunit.client.0.smithi032.stdout:5/437: creat da/dd/df/d10/d16/d18/d35/d70/f86 x:0 0 0 2024-06-03T01:21:02.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/438: fdatasync da/dd/df/d10/d16/d25/f66 0 2024-06-03T01:21:02.993 INFO:tasks.workunit.client.1.smithi044.stdout:5/908: dwrite d2/d11/d12/f25 [0,4194304] 0 2024-06-03T01:21:02.996 INFO:tasks.workunit.client.0.smithi032.stdout:5/439: read da/dd/f5e [386603,2702] 0 2024-06-03T01:21:02.996 INFO:tasks.workunit.client.0.smithi032.stdout:5/440: chown da/dd/df/d10/d16/d18/d1a/d30/d51/d7a 1 1 2024-06-03T01:21:03.001 INFO:tasks.workunit.client.1.smithi044.stdout:8/946: rename d5/d55/d56/d5c/d68/fcf to d5/d55/d74/d87/f13b 0 2024-06-03T01:21:03.001 INFO:tasks.workunit.client.1.smithi044.stdout:8/947: chown d5/d16/d81/de3/d98/fa7 8491 1 2024-06-03T01:21:03.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/441: dread da/dd/df/d10/d16/d18/f1d [0,4194304] 0 2024-06-03T01:21:03.004 INFO:tasks.workunit.client.0.smithi032.stdout:4/382: unlink d4/db/da/d1d/l22 0 2024-06-03T01:21:03.006 INFO:tasks.workunit.client.1.smithi044.stdout:7/929: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d122 0 2024-06-03T01:21:03.006 INFO:tasks.workunit.client.1.smithi044.stdout:7/930: stat d0/d3/d8/d95/l61 0 2024-06-03T01:21:03.019 INFO:tasks.workunit.client.0.smithi032.stdout:3/385: rename de/f57 to de/d63/f76 0 2024-06-03T01:21:03.191 INFO:tasks.workunit.client.0.smithi032.stdout:8/336: creat dc/d25/d28/d51/f71 x:0 0 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:8/337: write dc/d25/d28/d32/f50 [4271860,91058] 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:1/362: symlink d18/d27/d2b/l6d 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:7/369: mknod dc/d12/d2a/d2f/d63/c6e 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:7/370: write dc/d12/d22/d42/f51 [366217,46328] 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:2/407: symlink d5/da/d1a/d69/l8a 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:2/408: dread - d5/f35 zero size 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:3/386: symlink de/d29/d2e/d31/d3c/l77 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:3/387: stat de/d29/d2e/d31/c46 0 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:3/388: dread - de/d29/d2e/d31/d23/f4a zero size 2024-06-03T01:21:03.192 INFO:tasks.workunit.client.0.smithi032.stdout:3/389: getdents de/d29/d2e/d6d 0 2024-06-03T01:21:03.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/442: dread da/dd/df/f46 [0,4194304] 0 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:5/443: dread da/dd/df/d10/f4c [0,4194304] 0 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:4/383: rename d4/db/da/d1d/l67 to d4/db/da/d2d/l8d 0 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:0/368: dwrite d1/d46/f6b [0,4194304] 0 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:0/369: dread - d1/d46/f5f zero size 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:8/338: link dc/d25/d28/c68 dc/c72 0 2024-06-03T01:21:03.199 INFO:tasks.workunit.client.0.smithi032.stdout:2/409: creat d5/d38/d46/d5c/f8b x:0 0 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:3/390: mkdir de/d29/d2e/d31/d78 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:5/444: creat da/dd/df/d10/d16/d18/d35/d44/f87 x:0 0 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/384: mkdir d4/db/da/d5d/d8e 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:8/339: rename dc/de/d1e/d43/d4e/d59/f69 to dc/d25/d28/d51/d5d/d5e/f73 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:8/340: dread dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:2/410: symlink d5/da/d1a/d2a/d36/l8c 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:5/445: getdents da/dd/df/d10/d16/d25 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:8/341: mkdir dc/d14/d31/d3b/d74 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:2/411: truncate d5/da/f29 2736137 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:2/412: write d5/da/f39 [1576108,11191] 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:5/446: getdents da/dd/df/d10/d16/d25/d74 0 2024-06-03T01:21:03.200 INFO:tasks.workunit.client.0.smithi032.stdout:8/342: creat dc/d25/d28/d58/f75 x:0 0 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:2/413: truncate d5/f1c 5522970 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:0/370: dwrite d1/f9 [0,4194304] 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/447: rmdir da/dd/df/d10/d16/d18/d1a/d30/d58 39 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:7/371: dwrite dc/d12/d1b/f68 [0,4194304] 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:8/343: mkdir dc/d14/d31/d3b/d74/d76 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:4/385: dwrite d4/db/da/d1d/f23 [4194304,4194304] 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:4/386: chown d4/db/da/d37/f84 570 1 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:2/414: write d5/da/d1a/d2a/d36/f60 [2389188,73631] 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/448: creat da/dd/df/d10/d16/d25/d74/f88 x:0 0 0 2024-06-03T01:21:03.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/449: dread - da/dd/df/d10/d16/d18/d35/d44/f87 zero size 2024-06-03T01:21:03.202 INFO:tasks.workunit.client.0.smithi032.stdout:5/450: fdatasync da/dd/df/f5c 0 2024-06-03T01:21:03.202 INFO:tasks.workunit.client.0.smithi032.stdout:5/451: truncate da/dd/df/d10/d16/d18/d1a/f77 515829 0 2024-06-03T01:21:03.202 INFO:tasks.workunit.client.0.smithi032.stdout:5/452: dread da/dd/f68 [0,4194304] 0 2024-06-03T01:21:03.202 INFO:tasks.workunit.client.0.smithi032.stdout:5/453: readlink da/dd/df/d10/d16/d18/d35/l3c 0 2024-06-03T01:21:03.202 INFO:tasks.workunit.client.0.smithi032.stdout:7/372: chown l9 0 1 2024-06-03T01:21:03.202 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:02 smithi032 ceph-mon[29360]: pgmap v6: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:21:03.202 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:02 smithi032 ceph-mon[29360]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:03.202 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:02 smithi032 ceph-mon[29360]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:03.214 INFO:tasks.workunit.client.0.smithi032.stdout:8/344: truncate dc/d25/d28/d3c/f3f 4193811 0 2024-06-03T01:21:03.214 INFO:tasks.workunit.client.0.smithi032.stdout:8/345: creat dc/d25/d28/d3c/f77 x:0 0 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:0/371: rename d1/da/d28/c37 to d1/c75 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:2/415: rmdir d5/d38/d46/d53 39 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:5/454: symlink da/l89 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:7/373: truncate dc/d12/d22/d42/f51 216428 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:3/391: dwrite de/f15 [0,4194304] 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:3/392: write de/d29/d2e/d31/d23/f59 [828827,112368] 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:4/387: symlink d4/db/da/d2d/d27/d53/l8f 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:5/455: getdents da/dd/df/d10/d16/d18/d35 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:7/374: unlink l9 0 2024-06-03T01:21:03.215 INFO:tasks.workunit.client.0.smithi032.stdout:3/393: dread de/d29/d2e/f37 [0,4194304] 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:3/394: dread - de/d29/d2e/d31/d23/f5a zero size 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/375: mknod dc/dd/c6f 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/376: dread - dc/d12/d24/d33/f5c zero size 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:3/395: mkdir de/d29/d2e/d31/d23/d53/d79 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:3/396: write de/d29/f4b [923209,85754] 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/377: mknod dc/c70 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/378: truncate dc/d12/d24/f6c 919768 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:0/372: write d1/d46/f6b [4089358,126867] 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/379: mknod dc/d12/d22/d42/c71 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:0/373: creat d1/da/d28/d3b/f76 x:0 0 0 2024-06-03T01:21:03.216 INFO:tasks.workunit.client.0.smithi032.stdout:3/397: dread de/d29/d2e/d31/f18 [0,4194304] 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:7/380: creat dc/d12/d6a/f72 x:0 0 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:7/381: chown dc/d12/d22/c3e 2506 1 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/374: mknod d1/da/d28/d3b/d5d/c77 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:2/416: dwrite d5/da/f2f [0,4194304] 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:2/417: creat d5/f8d x:0 0 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/375: rmdir d1/da/d10/d29 39 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:5/456: dwrite da/dd/f29 [0,4194304] 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:2/418: truncate d5/da/f29 49064 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:8/346: dwrite dc/d25/d28/d51/d5d/d5e/f73 [0,4194304] 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/376: link d1/da/d28/d3b/d68/c71 d1/da/d28/d3b/c78 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/377: read d1/da/d28/f33 [868795,46783] 0 2024-06-03T01:21:03.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/378: creat d1/da/d10/d53/f79 x:0 0 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:0/379: creat d1/da/d10/f7a x:0 0 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:3/398: dread de/d29/d2e/d31/d23/f59 [0,4194304] 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:5/457: rename da/dd/df/d10/d16/d18/d1a/d30/d51/f52 to da/dd/df/d10/d16/f8a 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:5/458: fsync da/dd/df/d10/d16/d18/f27 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:4/388: rmdir d4/db 39 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:0/380: getdents d1/da/d28/d3b/d50 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/347: dwrite dc/d25/d28/d3c/f77 [0,4194304] 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/348: readlink dc/d25/d28/d33/l63 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:3/399: creat de/d63/f7a x:0 0 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:3/400: truncate de/d29/d2e/d31/d23/d53/f6c 686445 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:5/459: rename da/dd/f5e to da/dd/df/d10/d16/d25/f8b 0 2024-06-03T01:21:03.218 INFO:tasks.workunit.client.0.smithi032.stdout:4/389: mkdir d4/db/da/d37/d90 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/390: write d4/db/da/f3d [575282,93997] 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/391: dread - d4/db/da/d1d/d6b/f7c zero size 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:2/419: dwrite d5/d38/f3b [4194304,4194304] 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:0/381: rmdir d1/da/d28/d6a 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/349: mkdir dc/d25/d28/d33/d78 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/350: creat dc/d25/d28/d58/f79 x:0 0 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/351: fsync dc/d25/d28/d51/f71 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/352: write f9 [4883237,60205] 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:3/401: mknod de/d29/d2e/c7b 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:5/460: symlink da/dd/df/d10/d16/d25/l8c 0 2024-06-03T01:21:03.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/392: rmdir d4/db/da 39 2024-06-03T01:21:03.221 INFO:tasks.workunit.client.1.smithi044.stdout:9/887: dwrite d6/dd/d51/d16/d5f/d8b/f107 [0,4194304] 0 2024-06-03T01:21:03.221 INFO:tasks.workunit.client.1.smithi044.stdout:4/972: unlink d17/d1c/d75/d7a/db8/d59/fda 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/909: mkdir d2/d11/d1e/d21/d24/d93/d11e/d130 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:7/931: unlink d0/d3/f7 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/910: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f11b [0,4194304] 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:8/948: dwrite d5/d20/d33/fb1 [0,4194304] 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:9/888: link d6/dd/d51/d16/dee/cf9 d6/dd/d51/c11f 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:9/889: chown d6/dd/f108 17796834 1 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/808: sync 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/809: readlink db/d15/l44 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:4/973: link d17/d1c/d70/l7f d17/d1c/de0/d148/d141/l149 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/810: dread db/d1b/f99 [0,4194304] 0 2024-06-03T01:21:03.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/911: link d2/d11/d12/f1b d2/d11/d12/d2b/d4a/d57/d106/d12c/f131 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:5/912: fdatasync d2/d11/d1e/d21/d24/d93/d9d/d28/f2c 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:9/890: dread d6/dd/d51/d16/d5f/f80 [0,4194304] 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:9/891: read - d6/dd/d51/d16/d27/d5e/d71/f78 zero size 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:9/892: chown d6/dd/d51/d16/d5f/d8b/ffc 478502639 1 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:8/949: symlink d5/d55/d74/db4/l13c 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:8/950: write d5/d16/d81/de3/d98/f134 [725127,41870] 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:8/951: chown d5/d55/d74/d82/db0/c11e 654139 1 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:8/952: read - d5/d20/df5/f120 zero size 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:5/913: write d2/d11/d1e/d21/d24/d9e/dbb/ff7 [694372,88584] 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/811: dread db/d15/d49/d6c/f8c [0,4194304] 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:4/974: creat d17/d1c/d75/d7a/db8/d28/d2b/f14a x:0 0 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:4/975: dread - d17/d1c/d75/d7a/db8/db9/f143 zero size 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/812: write db/d1b/f99 [3775610,2391] 0 2024-06-03T01:21:03.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/813: dread - db/d1b/d2a/d2f/dbe/ded/dfa/ffc zero size 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:8/953: dwrite d5/d16/d81/f9c [0,4194304] 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:9/893: creat d6/dd/d51/d16/d27/d5e/d110/f120 x:0 0 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:5/914: readlink d2/l5 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:9/894: dread d6/f92 [0,4194304] 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/976: symlink d17/d1c/d70/dfb/dff/l14b 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/977: chown d17/d1c/d9c/ff3 453204 1 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/978: chown d17/d1c/d75/d7a/db8/db9/f143 1 1 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:7/932: rename d0/d3/d8/da0 to d0/d3/d8/d95/d84/d123 0 2024-06-03T01:21:03.224 INFO:tasks.workunit.client.1.smithi044.stdout:2/814: mkdir db/d1b/d2a/d67/de4/d100 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:5/915: mknod d2/d11/d1e/d21/d24/d93/d11e/c132 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:4/979: creat d17/d1c/d70/f14c x:0 0 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:8/954: rename d5/d20/d33/f139 to d5/d20/d47/d64/dd0/f13d 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:8/955: stat d5/d55/d74/db4/cda 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:8/956: write d5/d20/d47/d64/fa0 [3681242,117520] 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/933: creat d0/d3/d8/d95/d63/de6/f124 x:0 0 0 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/934: chown d0/d16/d23/f78 6127403 1 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/935: chown d0/d3 0 1 2024-06-03T01:21:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:4/980: getdents d17/d1c/de0 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:8/957: mknod d5/d16/d1d/d32/c13e 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:2/815: getdents db/d1b/d5d/da7/dc9 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/916: dread d2/d11/d1e/d21/d24/d93/d9d/f22 [0,4194304] 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/981: mknod d17/d63/d10c/c14d 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/982: chown d17/d1c/d75/d7a/db3/lba 7 1 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/983: chown d17/d1c/d70/d7d/d135 31003451 1 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/984: fdatasync d17/d1c/fb7 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/985: fsync d17/d1c/d75/fe6 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/986: readlink d17/d1c/d75/d7a/db8/d28/l36 0 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:2/816: chown db/f1f 0 1 2024-06-03T01:21:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/987: link d17/d1c/d75/d7a/db8/d6a/fd3 d17/d1c/d75/d7a/db8/d59/f14e 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/936: write d0/d3/d8/f1b [207219,26249] 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/937: truncate d0/d3/f3f 3335620 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:4/988: dread d17/d1c/d75/d7a/db8/d59/f7b [0,4194304] 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:8/958: dread d5/d20/d33/f63 [0,4194304] 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/938: rename d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/ffd to d0/d3/ddf/f125 0 2024-06-03T01:21:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/939: creat d0/d3/ddf/f126 x:0 0 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:7/940: mkdir d0/d3/d8/d95/d127 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:5/917: dread d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/f11b [0,4194304] 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:9/895: dwrite d6/dd/d51/f13 [0,4194304] 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:5/918: mkdir d2/d11c/d133 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:5/919: fdatasync d2/d11/d1e/d21/d24/d93/d9d/d28/f2c 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:8/959: write d5/d16/f6e [1945011,119607] 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:8/960: dread - d5/d55/d74/f78 zero size 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:5/920: symlink d2/d11/df1/l134 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:5/921: dread - d2/d11/d1e/d21/d24/faa zero size 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:9/896: dread d6/dd/d51/fe [0,4194304] 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:7/941: rmdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d104 0 2024-06-03T01:21:03.228 INFO:tasks.workunit.client.1.smithi044.stdout:8/961: creat d5/d20/f13f x:0 0 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/922: fdatasync d2/d11/d12/d2b/d4a/d57/fc7 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:2/817: dwrite db/d15/d49/f5c [4194304,4194304] 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:7/942: rename d0/d3/d8/d95/d63/d7f/d9c/d6e/faa to d0/d3/d8/f128 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:9/897: dread d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [0,4194304] 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/962: creat d5/d55/d56/d5c/dee/f140 x:0 0 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/923: creat d2/d11/d1e/d21/f135 x:0 0 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:7/943: mkdir d0/d3/d8/d95/d110/d129 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:4/989: dwrite d17/d1c/d75/d7a/db8/d28/f87 [0,4194304] 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:4/990: readlink d17/d1c/d75/d7a/db8/d6a/ld5 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/963: truncate d5/d55/d56/d5c/d68/de5/dea/f7f 784462 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:3/839: sync 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/964: mknod d5/d55/d56/d5c/d68/de5/dea/c141 0 2024-06-03T01:21:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/965: write d5/d55/d56/d5c/d68/de5/dea/de2/f108 [1349903,121187] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/966: fsync d5/d55/f12d 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:3/840: unlink df/d13/d1b/d4b/f4f 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:2/818: dwrite db/d15/d49/f5c [0,4194304] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:3/841: mkdir df/d13/d1b/d56/d5c/de3/d11a 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/924: dwrite d2/d11/d12/d2b/d4a/d57/fc7 [4194304,4194304] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/925: read d2/d11/d1e/f95 [3573822,65131] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/926: chown d2/d11/d3e/f59 418665955 1 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/967: dread d5/d55/f86 [0,4194304] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/968: read - d5/d55/d56/feb zero size 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:2/819: dread db/d15/d49/d6c/d72/dcd/fd5 [0,4194304] 0 2024-06-03T01:21:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:8/969: truncate d5/d16/f3c 3692615 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:8/970: chown d5/d55/d74 239 1 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:8/971: truncate d5/d55/db5/f137 1019120 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/927: getdents d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:7/944: dwrite d0/d3/d8/d32/f6c [0,4194304] 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:3/842: dread df/f65 [0,4194304] 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/928: dread d2/d11/d1e/d21/d24/d93/d9d/f49 [0,4194304] 0 2024-06-03T01:21:03.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/929: write d2/d11/d12/f29 [1612743,34664] 0 2024-06-03T01:21:03.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:02 smithi044 ceph-mon[36956]: pgmap v6: 65 pgs: 65 active+clean; 1.9 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail 2024-06-03T01:21:03.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:02 smithi044 ceph-mon[36956]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:03.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:02 smithi044 ceph-mon[36956]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:03.232 INFO:tasks.workunit.client.1.smithi044.stdout:8/972: truncate d5/d16/d1d/fa5 2826548 0 2024-06-03T01:21:03.232 INFO:tasks.workunit.client.1.smithi044.stdout:8/973: chown d5/d16/d81/de3/d58/c5a 5725129 1 2024-06-03T01:21:03.233 INFO:tasks.workunit.client.1.smithi044.stdout:3/843: creat df/d1e/d2a/d9b/dc2/f11b x:0 0 0 2024-06-03T01:21:03.233 INFO:tasks.workunit.client.1.smithi044.stdout:5/930: rename d2/d11/d1e/d21/fe6 to d2/d11/df1/f136 0 2024-06-03T01:21:03.234 INFO:tasks.workunit.client.0.smithi032.stdout:2/420: symlink d5/d38/l8e 0 2024-06-03T01:21:03.234 INFO:tasks.workunit.client.0.smithi032.stdout:3/402: mknod de/d29/d2e/d31/d23/d53/c7c 0 2024-06-03T01:21:03.234 INFO:tasks.workunit.client.0.smithi032.stdout:5/461: rename da/dd/df/d10/d16/d18/d1a/d30/d51/d7a/d7d to da/dd/df/d10/d8d 0 2024-06-03T01:21:03.236 INFO:tasks.workunit.client.0.smithi032.stdout:0/382: dwrite d1/f2b [0,4194304] 0 2024-06-03T01:21:03.236 INFO:tasks.workunit.client.0.smithi032.stdout:0/383: fsync d1/da/d10/f6f 0 2024-06-03T01:21:03.236 INFO:tasks.workunit.client.1.smithi044.stdout:8/974: mkdir d5/d20/db3/d142 0 2024-06-03T01:21:03.238 INFO:tasks.workunit.client.0.smithi032.stdout:4/393: mkdir d4/db/da/d37/d90/d91 0 2024-06-03T01:21:03.238 INFO:tasks.workunit.client.0.smithi032.stdout:4/394: readlink d4/db/da/d37/l88 0 2024-06-03T01:21:03.239 INFO:tasks.workunit.client.1.smithi044.stdout:2/820: dwrite db/d15/d49/d82/fac [0,4194304] 0 2024-06-03T01:21:03.239 INFO:tasks.workunit.client.1.smithi044.stdout:2/821: creat db/d1b/d2a/d2f/dbe/f101 x:0 0 0 2024-06-03T01:21:03.239 INFO:tasks.workunit.client.1.smithi044.stdout:3/844: creat df/d13/d1b/d56/d5c/d98/df4/f11c x:0 0 0 2024-06-03T01:21:03.239 INFO:tasks.workunit.client.1.smithi044.stdout:7/945: dwrite d0/f1d [0,4194304] 0 2024-06-03T01:21:03.240 INFO:tasks.workunit.client.0.smithi032.stdout:8/353: dwrite dc/d25/d28/d33/f48 [0,4194304] 0 2024-06-03T01:21:03.240 INFO:tasks.workunit.client.0.smithi032.stdout:2/421: mkdir d5/da/d1a/d2a/d49/d8f 0 2024-06-03T01:21:03.240 INFO:tasks.workunit.client.0.smithi032.stdout:2/422: truncate d5/f35 746993 0 2024-06-03T01:21:03.241 INFO:tasks.workunit.client.1.smithi044.stdout:5/931: symlink d2/d11/d1e/d21/d24/d9e/db1/l137 0 2024-06-03T01:21:03.241 INFO:tasks.workunit.client.1.smithi044.stdout:5/932: truncate d2/d11/d12/f39 1925984 0 2024-06-03T01:21:03.243 INFO:tasks.workunit.client.1.smithi044.stdout:8/975: mknod d5/d55/d74/db4/c143 0 2024-06-03T01:21:03.245 INFO:tasks.workunit.client.0.smithi032.stdout:5/462: truncate da/f17 1212628 0 2024-06-03T01:21:03.246 INFO:tasks.workunit.client.0.smithi032.stdout:5/463: chown da/dd/df/d10/d16/d18/d35/d70/f86 0 1 2024-06-03T01:21:03.246 INFO:tasks.workunit.client.0.smithi032.stdout:5/464: fsync da/dd/df/d10/d16/d18/d35/f81 0 2024-06-03T01:21:03.249 INFO:tasks.workunit.client.0.smithi032.stdout:0/384: mknod d1/da/d10/d29/d6c/c7b 0 2024-06-03T01:21:03.249 INFO:tasks.workunit.client.0.smithi032.stdout:0/385: stat d1/da/d10/f2a 0 2024-06-03T01:21:03.251 INFO:tasks.workunit.client.0.smithi032.stdout:4/395: read d4/db/da/d2d/d27/f2b [3218791,86310] 0 2024-06-03T01:21:03.251 INFO:tasks.workunit.client.0.smithi032.stdout:5/465: write da/dd/df/d10/f4d [56012,78595] 0 2024-06-03T01:21:03.251 INFO:tasks.workunit.client.0.smithi032.stdout:4/396: chown d4/db/da/d1d/f75 7 1 2024-06-03T01:21:03.253 INFO:tasks.workunit.client.1.smithi044.stdout:3/845: creat df/d1e/d8a/f11d x:0 0 0 2024-06-03T01:21:03.255 INFO:tasks.workunit.client.0.smithi032.stdout:8/354: mknod dc/d25/d28/d51/d5d/d5e/c7a 0 2024-06-03T01:21:03.256 INFO:tasks.workunit.client.1.smithi044.stdout:5/933: creat d2/d11/d1e/dda/f138 x:0 0 0 2024-06-03T01:21:03.258 INFO:tasks.workunit.client.0.smithi032.stdout:0/386: mknod d1/da/d28/d3b/d5d/c7c 0 2024-06-03T01:21:03.258 INFO:tasks.workunit.client.0.smithi032.stdout:0/387: fsync d1/da/f1a 0 2024-06-03T01:21:03.258 INFO:tasks.workunit.client.1.smithi044.stdout:2/822: truncate db/f73 1745629 0 2024-06-03T01:21:03.259 INFO:tasks.workunit.client.1.smithi044.stdout:2/823: write db/fee [1307789,12664] 0 2024-06-03T01:21:03.260 INFO:tasks.workunit.client.1.smithi044.stdout:3/846: symlink df/d13/d39/d3d/d50/l11e 0 2024-06-03T01:21:03.260 INFO:tasks.workunit.client.1.smithi044.stdout:3/847: chown df/d1e/d2a/d70/c101 1632 1 2024-06-03T01:21:03.261 INFO:tasks.workunit.client.0.smithi032.stdout:4/397: truncate d4/db/da/d2d/d27/f2b 545888 0 2024-06-03T01:21:03.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/398: creat d4/d7f/f92 x:0 0 0 2024-06-03T01:21:03.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/399: write d4/db/da/f2f [4411496,1901] 0 2024-06-03T01:21:03.262 INFO:tasks.workunit.client.0.smithi032.stdout:4/400: dread - d4/db/f51 zero size 2024-06-03T01:21:03.264 INFO:tasks.workunit.client.0.smithi032.stdout:3/403: rmdir de/d29/d2e/d6d 0 2024-06-03T01:21:03.266 INFO:tasks.workunit.client.0.smithi032.stdout:0/388: symlink d1/da/d10/d29/l7d 0 2024-06-03T01:21:03.266 INFO:tasks.workunit.client.0.smithi032.stdout:0/389: chown d1/da/d10/d29/c49 83511 1 2024-06-03T01:21:03.268 INFO:tasks.workunit.client.1.smithi044.stdout:5/934: rename d2/d11c/fe5 to d2/d11/d1e/d21/d24/d93/f139 0 2024-06-03T01:21:03.268 INFO:tasks.workunit.client.0.smithi032.stdout:2/423: dwrite d5/da/d1a/d2a/d49/d7e/f75 [0,4194304] 0 2024-06-03T01:21:03.269 INFO:tasks.workunit.client.1.smithi044.stdout:2/824: symlink db/d1b/d2a/d67/d53/d9b/l102 0 2024-06-03T01:21:03.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/848: mknod df/d1e/d3a/c11f 0 2024-06-03T01:21:03.271 INFO:tasks.workunit.client.0.smithi032.stdout:4/401: rmdir d4 39 2024-06-03T01:21:03.271 INFO:tasks.workunit.client.0.smithi032.stdout:3/404: rename de/d29/d2e/d31/d3c/l4f to de/d29/d2e/d31/d3c/l7d 0 2024-06-03T01:21:03.271 INFO:tasks.workunit.client.0.smithi032.stdout:3/405: write de/d29/d2e/d31/f21 [4935385,28436] 0 2024-06-03T01:21:03.272 INFO:tasks.workunit.client.0.smithi032.stdout:3/406: dread - de/d29/d2e/d31/f1e zero size 2024-06-03T01:21:03.272 INFO:tasks.workunit.client.1.smithi044.stdout:7/946: dwrite d0/d3/ddf/f125 [0,4194304] 0 2024-06-03T01:21:03.273 INFO:tasks.workunit.client.1.smithi044.stdout:5/935: rename d2/d11/d3e/cae to d2/d11/d1e/c13a 0 2024-06-03T01:21:03.274 INFO:tasks.workunit.client.0.smithi032.stdout:2/424: mkdir d5/d38/d46/d53/d90 0 2024-06-03T01:21:03.275 INFO:tasks.workunit.client.1.smithi044.stdout:2/825: creat db/d15/d49/d7b/f103 x:0 0 0 2024-06-03T01:21:03.276 INFO:tasks.workunit.client.1.smithi044.stdout:3/849: mknod df/d13/d1b/d56/d5c/c120 0 2024-06-03T01:21:03.277 INFO:tasks.workunit.client.0.smithi032.stdout:4/402: symlink d4/db/da/d2d/l93 0 2024-06-03T01:21:03.279 INFO:tasks.workunit.client.0.smithi032.stdout:9/483: sync 2024-06-03T01:21:03.282 INFO:tasks.workunit.client.0.smithi032.stdout:3/407: mkdir de/d29/d2e/d31/d3c/d50/d5e/d7e 0 2024-06-03T01:21:03.283 INFO:tasks.workunit.client.1.smithi044.stdout:5/936: write d2/d11/df1/f11f [37782,909] 0 2024-06-03T01:21:03.283 INFO:tasks.workunit.client.0.smithi032.stdout:9/484: creat d0/d11/d9c/fa3 x:0 0 0 2024-06-03T01:21:03.284 INFO:tasks.workunit.client.0.smithi032.stdout:3/408: getdents de/d29/d2e/d31/d3c/d50 0 2024-06-03T01:21:03.284 INFO:tasks.workunit.client.0.smithi032.stdout:3/409: creat de/d29/f7f x:0 0 0 2024-06-03T01:21:03.284 INFO:tasks.workunit.client.0.smithi032.stdout:3/410: fdatasync de/d29/d2e/d31/d24/f25 0 2024-06-03T01:21:03.285 INFO:tasks.workunit.client.0.smithi032.stdout:0/390: write d1/f21 [4088317,82217] 0 2024-06-03T01:21:03.285 INFO:tasks.workunit.client.0.smithi032.stdout:8/355: dwrite dc/de/d1e/f2e [0,4194304] 0 2024-06-03T01:21:03.286 INFO:tasks.workunit.client.0.smithi032.stdout:3/411: creat de/d29/d2e/d31/d78/f80 x:0 0 0 2024-06-03T01:21:03.290 INFO:tasks.workunit.client.0.smithi032.stdout:8/356: truncate dc/de/d1e/f61 2418167 0 2024-06-03T01:21:03.290 INFO:tasks.workunit.client.1.smithi044.stdout:7/947: truncate d0/d16/f54 578698 0 2024-06-03T01:21:03.292 INFO:tasks.workunit.client.0.smithi032.stdout:8/357: creat dc/d14/d31/f7b x:0 0 0 2024-06-03T01:21:03.292 INFO:tasks.workunit.client.0.smithi032.stdout:8/358: write dc/d25/d28/d3c/f5a [119549,61710] 0 2024-06-03T01:21:03.293 INFO:tasks.workunit.client.1.smithi044.stdout:2/826: unlink db/d15/d49/d6c/d72/dcd/ce0 0 2024-06-03T01:21:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.297+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a18103d60 con 0x7f9a00039240 2024-06-03T01:21:03.297 INFO:tasks.workunit.client.0.smithi032.stdout:3/412: unlink de/l33 0 2024-06-03T01:21:03.298 INFO:tasks.workunit.client.1.smithi044.stdout:8/976: dwrite d5/d16/d81/de3/d58/d89/fb9 [0,4194304] 0 2024-06-03T01:21:03.299 INFO:tasks.workunit.client.0.smithi032.stdout:5/466: dwrite da/dd/df/d10/d16/d18/f50 [0,4194304] 0 2024-06-03T01:21:03.300 INFO:tasks.workunit.client.1.smithi044.stdout:3/850: mknod df/d13/d1b/d56/d5c/de3/de6/c121 0 2024-06-03T01:21:03.300 INFO:tasks.workunit.client.1.smithi044.stdout:5/937: truncate d2/d11/d1e/d21/d24/d93/d9d/d28/d66/f8e 579281 0 2024-06-03T01:21:03.300 INFO:tasks.workunit.client.1.smithi044.stdout:7/948: getdents d0/d3/d8/d95/d63/d89/d8b 0 2024-06-03T01:21:03.300 INFO:tasks.workunit.client.1.smithi044.stdout:3/851: chown df/d13/d1b/d4b 4758 1 2024-06-03T01:21:03.301 INFO:tasks.workunit.client.0.smithi032.stdout:0/391: dread d1/fc [0,4194304] 0 2024-06-03T01:21:03.301 INFO:tasks.workunit.client.0.smithi032.stdout:0/392: creat d1/da/d10/d53/f7e x:0 0 0 2024-06-03T01:21:03.302 INFO:tasks.workunit.client.0.smithi032.stdout:2/425: dwrite d5/f58 [4194304,4194304] 0 2024-06-03T01:21:03.310 INFO:tasks.workunit.client.0.smithi032.stdout:0/393: link d1/da/d10/c2c d1/da/d10/d29/d6c/c7f 0 2024-06-03T01:21:03.311 INFO:tasks.workunit.client.1.smithi044.stdout:8/977: truncate d5/fe4 840104 0 2024-06-03T01:21:03.312 INFO:tasks.workunit.client.0.smithi032.stdout:8/359: dread dc/d25/d28/d3c/f5a [0,4194304] 0 2024-06-03T01:21:03.315 INFO:tasks.workunit.client.0.smithi032.stdout:2/426: mknod d5/da/d1a/d69/d83/c91 0 2024-06-03T01:21:03.319 INFO:tasks.workunit.client.1.smithi044.stdout:5/938: getdents d2/d11/d1e/d3b/da6/d116 0 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.320+0000 7f9a14ff9700 1 -- 172.21.15.32:0/385645276 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9a18103d60 con 0x7f9a00039240 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 2m ago 8m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (8m) 2m ago 8m 7490k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (7m) 22s ago 7m 7520k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:21:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (8m) 2m ago 8m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (7m) 22s ago 7m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 2m ago 7m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (2m) 2m ago 2m 16.2M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (2m) 2m ago 2m 14.8M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (2m) 22s ago 2m 12.5M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (2m) 22s ago 2m 15.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:9283,8765,8443 running (9m) 2m ago 9m 489M - 18.2.1 5be31c24972a 5db22291b59c 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (27s) 22s ago 7m 52.6M - 19.0.0-2946-g3d89accf b045ac545f82 57506042450c 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (9m) 2m ago 9m 44.6M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (7m) 22s ago 7m 36.5M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 2m ago 8m 14.2M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 22s ago 7m 16.2M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:21:03.321 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) 2m ago 6m 46.8M 3948M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 2m ago 5m 46.7M 3948M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (5m) 2m ago 5m 49.5M 3948M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (5m) 22s ago 5m 117M 4972M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (4m) 22s ago 4m 112M 4972M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (4m) 22s ago 4m 98.2M 4972M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:21:03.322 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 2m ago 7m 57.9M - 2.43.0 a07b618ecd1d 55ba7950dc8e 2024-06-03T01:21:03.322 INFO:tasks.workunit.client.1.smithi044.stdout:7/949: mkdir d0/d12a 0 2024-06-03T01:21:03.323 INFO:tasks.workunit.client.0.smithi032.stdout:9/485: dwrite d0/d11/d1d/d38/f61 [0,4194304] 0 2024-06-03T01:21:03.324 INFO:tasks.workunit.client.1.smithi044.stdout:2/827: dwrite db/d1b/d27/f32 [0,4194304] 0 2024-06-03T01:21:03.324 INFO:tasks.workunit.client.1.smithi044.stdout:2/828: write db/fee [1860302,71441] 0 2024-06-03T01:21:03.324 INFO:tasks.workunit.client.1.smithi044.stdout:2/829: truncate db/d15/f70 2350946 0 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 msgr2=0x7f9a0003b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 0x7f9a0003b700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009350 tx=0x7f9a0800b540 comp rx=0 tx=0).stop 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 msgr2=0x7f9a181a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b480 tx=0x7f9a0c00b840 comp rx=0 tx=0).stop 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 shutdown_connections 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f9a00039240 0x7f9a0003b700 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a180737f0 0x7f9a181a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 --2- 172.21.15.32:0/385645276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a18074dc0 0x7f9a181a5aa0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b480 tx=0x7f9a0c00b840 comp rx=0 tx=0).stop 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.324+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 >> 172.21.15.32:0/385645276 conn(0x7f9a180fc230 msgr2=0x7f9a18102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.325+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 shutdown_connections 2024-06-03T01:21:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.325+0000 7f9a1df75700 1 -- 172.21.15.32:0/385645276 wait complete. 2024-06-03T01:21:03.326 INFO:tasks.workunit.client.1.smithi044.stdout:8/978: link d5/d16/f7c d5/d55/d74/d82/f144 0 2024-06-03T01:21:03.328 INFO:tasks.workunit.client.0.smithi032.stdout:2/427: dread d5/da/f39 [0,4194304] 0 2024-06-03T01:21:03.328 INFO:tasks.workunit.client.1.smithi044.stdout:3/852: symlink df/d13/l122 0 2024-06-03T01:21:03.328 INFO:tasks.workunit.client.0.smithi032.stdout:4/403: dwrite d4/db/da/d1d/d6b/f7c [0,4194304] 0 2024-06-03T01:21:03.331 INFO:tasks.workunit.client.0.smithi032.stdout:0/394: dwrite d1/d46/f48 [0,4194304] 0 2024-06-03T01:21:03.331 INFO:tasks.workunit.client.0.smithi032.stdout:0/395: truncate d1/da/f16 2141163 0 2024-06-03T01:21:03.332 INFO:tasks.workunit.client.1.smithi044.stdout:5/939: truncate d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/ff2 2217485 0 2024-06-03T01:21:03.332 INFO:tasks.workunit.client.1.smithi044.stdout:5/940: readlink d2/d11/df1/l134 0 2024-06-03T01:21:03.336 INFO:tasks.workunit.client.0.smithi032.stdout:9/486: truncate d0/d11/d1d/d38/f3e 2808595 0 2024-06-03T01:21:03.337 INFO:tasks.workunit.client.1.smithi044.stdout:2/830: creat db/d1b/d27/d79/f104 x:0 0 0 2024-06-03T01:21:03.337 INFO:tasks.workunit.client.1.smithi044.stdout:2/831: creat db/d1b/d2a/d2f/dbe/ded/dfa/f105 x:0 0 0 2024-06-03T01:21:03.338 INFO:tasks.workunit.client.1.smithi044.stdout:4/991: sync 2024-06-03T01:21:03.338 INFO:tasks.workunit.client.1.smithi044.stdout:9/898: sync 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:9/899: write d6/dd/d51/d16/d5f/d8b/d8d/dac/fcf [732540,107529] 0 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:9/900: chown d6/d38/d11a/dd9/lf6 197 1 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:9/901: chown d6/dd/d51/d16/d5f/lf8 648384 1 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:9/902: readlink d6/dd/d51/d16/d5f/d8b/l100 0 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:9/903: write d6/dd/f3d [4527550,1078] 0 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.1.smithi044.stdout:8/979: creat d5/d20/d47/d64/f145 x:0 0 0 2024-06-03T01:21:03.339 INFO:tasks.workunit.client.0.smithi032.stdout:2/428: write d5/da/d1a/f6f [945624,18976] 0 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/429: truncate d5/da/d1a/d2a/d49/d7e/f64 582921 0 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:3/853: symlink df/d13/d39/d3d/l123 0 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:3/854: write df/d1e/d2a/d70/fa3 [1342065,45296] 0 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/941: rmdir d2/d11/df1 39 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/942: stat d2/d11/d1e/d21/d24/d93/d9d/d28/d66/c8b 0 2024-06-03T01:21:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/943: write d2/d11/d1e/d21/d24/f69 [1543517,21395] 0 2024-06-03T01:21:03.341 INFO:tasks.workunit.client.1.smithi044.stdout:5/944: readlink d2/d11/d1e/lc9 0 2024-06-03T01:21:03.341 INFO:tasks.workunit.client.0.smithi032.stdout:4/404: mknod d4/d57/d6c/c94 0 2024-06-03T01:21:03.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/396: mkdir d1/da/d10/d80 0 2024-06-03T01:21:03.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/397: readlink d1/l7 0 2024-06-03T01:21:03.343 INFO:tasks.workunit.client.0.smithi032.stdout:4/405: creat d4/db/da/d37/d90/f95 x:0 0 0 2024-06-03T01:21:03.343 INFO:tasks.workunit.client.0.smithi032.stdout:4/406: readlink d4/db/da/d37/l47 0 2024-06-03T01:21:03.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/430: dread d5/ff [0,4194304] 0 2024-06-03T01:21:03.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/431: dread - d5/da/f74 zero size 2024-06-03T01:21:03.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/432: chown d5/da/d1a/d2a/d36 237 1 2024-06-03T01:21:03.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/433: read d5/ff [2292124,76606] 0 2024-06-03T01:21:03.346 INFO:tasks.workunit.client.1.smithi044.stdout:4/992: unlink d17/d1c/d75/d7a/db8/d8d/fc4 0 2024-06-03T01:21:03.347 INFO:tasks.workunit.client.1.smithi044.stdout:9/904: creat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/f121 x:0 0 0 2024-06-03T01:21:03.350 INFO:tasks.workunit.client.1.smithi044.stdout:3/855: link df/d1e/d2a/d70/dbc/ce7 df/d13/d1b/d4b/c124 0 2024-06-03T01:21:03.352 INFO:tasks.workunit.client.1.smithi044.stdout:4/993: mknod d17/d1c/d75/d7a/db8/d128/c14f 0 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.1.smithi044.stdout:9/905: dread d6/dd/f108 [0,4194304] 0 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/398: mkdir d1/da/d10/d29/d6c/d81 0 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:4/407: rmdir d4/db/da/d2d 39 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/399: chown d1/da/d10/l22 46000958 1 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/400: stat d1/da/d28/f56 0 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/401: truncate d1/da/d10/f4f 265413 0 2024-06-03T01:21:03.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/402: write d1/da/d28/f33 [1549588,13947] 0 2024-06-03T01:21:03.354 INFO:tasks.workunit.client.0.smithi032.stdout:8/360: dwrite dc/f13 [0,4194304] 0 2024-06-03T01:21:03.354 INFO:tasks.workunit.client.0.smithi032.stdout:5/467: dwrite da/dd/df/d10/d16/d18/d1a/d30/f5f [0,4194304] 0 2024-06-03T01:21:03.355 INFO:tasks.workunit.client.0.smithi032.stdout:5/468: write da/dd/df/d10/d16/d25/f8b [1960891,70597] 0 2024-06-03T01:21:03.356 INFO:tasks.workunit.client.1.smithi044.stdout:3/856: dread df/d13/d1b/d4b/d69/f76 [0,4194304] 0 2024-06-03T01:21:03.357 INFO:tasks.workunit.client.1.smithi044.stdout:8/980: write d5/d20/d33/f4b [421992,32703] 0 2024-06-03T01:21:03.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/403: rmdir d1/da/d10/d29/d6c/d81 0 2024-06-03T01:21:03.358 INFO:tasks.workunit.client.1.smithi044.stdout:7/950: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/f9b [0,4194304] 0 2024-06-03T01:21:03.360 INFO:tasks.workunit.client.0.smithi032.stdout:2/434: dread d5/da/d1a/d2a/d49/f7c [0,4194304] 0 2024-06-03T01:21:03.362 INFO:tasks.workunit.client.0.smithi032.stdout:4/408: dread d4/d57/d6c/f6e [0,4194304] 0 2024-06-03T01:21:03.362 INFO:tasks.workunit.client.0.smithi032.stdout:4/409: chown d4/db/da/d2d/f2a 0 1 2024-06-03T01:21:03.362 INFO:tasks.workunit.client.0.smithi032.stdout:5/469: truncate da/dd/f68 717345 0 2024-06-03T01:21:03.363 INFO:tasks.workunit.client.1.smithi044.stdout:2/832: dread db/d1b/f99 [0,4194304] 0 2024-06-03T01:21:03.363 INFO:tasks.workunit.client.1.smithi044.stdout:2/833: chown db/dd 86 1 2024-06-03T01:21:03.364 INFO:tasks.workunit.client.1.smithi044.stdout:8/981: creat d5/d55/d56/d5c/d68/de5/dea/d79/d88/f146 x:0 0 0 2024-06-03T01:21:03.365 INFO:tasks.workunit.client.0.smithi032.stdout:0/404: mkdir d1/da/d28/d3b/d68/d82 0 2024-06-03T01:21:03.367 INFO:tasks.workunit.client.1.smithi044.stdout:4/994: write d17/d1c/d75/d7a/db8/fa5 [2711921,9213] 0 2024-06-03T01:21:03.367 INFO:tasks.workunit.client.1.smithi044.stdout:4/995: creat d17/d1c/d7e/f150 x:0 0 0 2024-06-03T01:21:03.368 INFO:tasks.workunit.client.0.smithi032.stdout:2/435: mknod d5/d38/c92 0 2024-06-03T01:21:03.368 INFO:tasks.workunit.client.0.smithi032.stdout:2/436: fsync d5/f58 0 2024-06-03T01:21:03.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/982: mkdir d5/d55/d74/d82/db0/d147 0 2024-06-03T01:21:03.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/983: truncate d5/d55/db5/f137 1672234 0 2024-06-03T01:21:03.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/984: fdatasync d5/d20/d47/f49 0 2024-06-03T01:21:03.371 INFO:tasks.workunit.client.0.smithi032.stdout:4/410: link d4/d7f/f8b d4/db/da/f96 0 2024-06-03T01:21:03.372 INFO:tasks.workunit.client.0.smithi032.stdout:4/411: chown d4/d57/d6c/f73 7124377 1 2024-06-03T01:21:03.372 INFO:tasks.workunit.client.1.smithi044.stdout:9/906: dread d6/dd/d51/d16/d27/d5e/db7/dd6/fdc [0,4194304] 0 2024-06-03T01:21:03.376 INFO:tasks.workunit.client.1.smithi044.stdout:7/951: dread d0/f74 [0,4194304] 0 2024-06-03T01:21:03.376 INFO:tasks.workunit.client.1.smithi044.stdout:7/952: chown d0/d3/d8/d95/d63/d7f/d9c/d6e/d76/d96/d5c/fbd 27 1 2024-06-03T01:21:03.378 INFO:tasks.workunit.client.1.smithi044.stdout:2/834: dread db/d15/f70 [0,4194304] 0 2024-06-03T01:21:03.380 INFO:tasks.workunit.client.0.smithi032.stdout:5/470: rename da/dd/df/d10/d16/d18/d35/d44/f87 to da/dd/df/d10/d16/d25/d74/f8e 0 2024-06-03T01:21:03.380 INFO:tasks.workunit.client.0.smithi032.stdout:5/471: write da/dd/df/d10/f4d [1294517,59455] 0 2024-06-03T01:21:03.380 INFO:tasks.workunit.client.0.smithi032.stdout:5/472: creat da/dd/df/d10/d16/d18/d35/d44/f8f x:0 0 0 2024-06-03T01:21:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/985: symlink d5/d55/db5/l148 0 2024-06-03T01:21:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/986: fdatasync d5/d16/d81/d9b/f133 0 2024-06-03T01:21:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/987: stat d5/d55/d56/d5c/d68/de5/f104 0 2024-06-03T01:21:03.384 INFO:tasks.workunit.client.1.smithi044.stdout:7/953: mknod d0/d16/d23/c12b 0 2024-06-03T01:21:03.386 INFO:tasks.workunit.client.0.smithi032.stdout:0/405: mkdir d1/da/d28/d3b/d68/d83 0 2024-06-03T01:21:03.390 INFO:tasks.workunit.client.1.smithi044.stdout:5/945: dwrite d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/ffc [0,4194304] 0 2024-06-03T01:21:03.390 INFO:tasks.workunit.client.0.smithi032.stdout:9/487: dwrite d0/d11/f22 [4194304,4194304] 0 2024-06-03T01:21:03.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/473: symlink da/dd/df/d10/d16/l90 0 2024-06-03T01:21:03.397 INFO:tasks.workunit.client.1.smithi044.stdout:2/835: mkdir db/d106 0 2024-06-03T01:21:03.397 INFO:tasks.workunit.client.1.smithi044.stdout:9/907: dread d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68/fc8 [0,4194304] 0 2024-06-03T01:21:03.399 INFO:tasks.workunit.client.0.smithi032.stdout:8/361: dwrite dc/d25/d28/d58/f75 [0,4194304] 0 2024-06-03T01:21:03.400 INFO:tasks.workunit.client.0.smithi032.stdout:0/406: mknod d1/da/d10/d80/c84 0 2024-06-03T01:21:03.400 INFO:tasks.workunit.client.0.smithi032.stdout:0/407: write d1/d46/f55 [605358,104602] 0 2024-06-03T01:21:03.402 INFO:tasks.workunit.client.1.smithi044.stdout:9/908: write d6/dd/d51/d16/d27/d5e/d71/dc1/fc4 [425957,4102] 0 2024-06-03T01:21:03.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/946: dread d2/d11/d12/f39 [0,4194304] 0 2024-06-03T01:21:03.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/947: chown d2/d11c/l3d 20740860 1 2024-06-03T01:21:03.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/948: fsync d2/d11/d1e/d21/d24/d93/d9d/f62 0 2024-06-03T01:21:03.404 INFO:tasks.workunit.client.1.smithi044.stdout:9/909: mknod d6/dd/d51/d16/d27/d5e/de6/c122 0 2024-06-03T01:21:03.408 INFO:tasks.workunit.client.1.smithi044.stdout:5/949: mknod d2/d11/d1e/d3b/da6/d116/c13b 0 2024-06-03T01:21:03.408 INFO:tasks.workunit.client.1.smithi044.stdout:5/950: fsync d2/d11/d12/d2b/d4a/d57/fb4 0 2024-06-03T01:21:03.408 INFO:tasks.workunit.client.1.smithi044.stdout:9/910: getdents d6/dd/d51/d16/d5f/d8b/d8d 0 2024-06-03T01:21:03.409 INFO:tasks.workunit.client.0.smithi032.stdout:8/362: mkdir dc/de/d1e/d43/d7c 0 2024-06-03T01:21:03.410 INFO:tasks.workunit.client.0.smithi032.stdout:2/437: dwrite d5/ff [0,4194304] 0 2024-06-03T01:21:03.411 INFO:tasks.workunit.client.0.smithi032.stdout:0/408: stat d1/da/d28/d3b/d50/l5e 0 2024-06-03T01:21:03.412 INFO:tasks.workunit.client.1.smithi044.stdout:5/951: creat d2/d11/d12/f13c x:0 0 0 2024-06-03T01:21:03.413 INFO:tasks.workunit.client.1.smithi044.stdout:5/952: read - d2/d11/d12/f13c zero size 2024-06-03T01:21:03.415 INFO:tasks.workunit.client.1.smithi044.stdout:5/953: symlink d2/d11/d3e/dce/dd1/l13d 0 2024-06-03T01:21:03.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/911: getdents d6/dd/d51/d16/d27/d5e/d71/dc1 0 2024-06-03T01:21:03.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/912: chown d6/dd/d45 4 1 2024-06-03T01:21:03.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/913: dread - d6/dd/d51/d16/d27/d5e/d71/f78 zero size 2024-06-03T01:21:03.423 INFO:tasks.workunit.client.1.smithi044.stdout:9/914: creat d6/dd/d51/d16/d27/d105/f123 x:0 0 0 2024-06-03T01:21:03.428 INFO:tasks.workunit.client.1.smithi044.stdout:7/954: dwrite d0/d3/d8/d95/d63/d7f/faf [4194304,4194304] 0 2024-06-03T01:21:03.428 INFO:tasks.workunit.client.1.smithi044.stdout:7/955: chown d0/d3/d8/d95/d84/f93 10 1 2024-06-03T01:21:03.428 INFO:tasks.workunit.client.1.smithi044.stdout:7/956: truncate d0/d16/f11c 223206 0 2024-06-03T01:21:03.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/857: dwrite df/f21 [0,4194304] 0 2024-06-03T01:21:03.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/858: write f9 [4516867,6971] 0 2024-06-03T01:21:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/859: chown df/d13/d1b/d59/fd8 3510 1 2024-06-03T01:21:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/860: fsync df/d13/d39/d3d/d43/d71/dd7/f112 0 2024-06-03T01:21:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/861: chown df/d1e/d3a 3151626 1 2024-06-03T01:21:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:8/988: dwrite d5/d20/d47/d64/fa0 [0,4194304] 0 2024-06-03T01:21:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:7/957: rename d0/d16/f22 to d0/d16/f12c 0 2024-06-03T01:21:03.434 INFO:tasks.workunit.client.0.smithi032.stdout:5/474: dwrite f4 [0,4194304] 0 2024-06-03T01:21:03.434 INFO:tasks.workunit.client.1.smithi044.stdout:3/862: mkdir df/d13/d6d/d9a/dad/d125 0 2024-06-03T01:21:03.435 INFO:tasks.workunit.client.1.smithi044.stdout:8/989: unlink d5/d20/d33/le6 0 2024-06-03T01:21:03.435 INFO:tasks.workunit.client.1.smithi044.stdout:7/958: truncate d0/d3/d8/d95/d63/f8f 220197 0 2024-06-03T01:21:03.436 INFO:tasks.workunit.client.1.smithi044.stdout:7/959: fsync d0/d3/d8/d95/d63/d89/da3/d9a/f111 0 2024-06-03T01:21:03.436 INFO:tasks.workunit.client.1.smithi044.stdout:7/960: creat d0/d3/d8/d95/d63/d89/da3/dde/d10a/f12d x:0 0 0 2024-06-03T01:21:03.438 INFO:tasks.workunit.client.0.smithi032.stdout:5/475: creat da/dd/df/d10/d16/d18/d1a/d30/d51/f91 x:0 0 0 2024-06-03T01:21:03.438 INFO:tasks.workunit.client.0.smithi032.stdout:5/476: fsync da/dd/df/d10/d16/d18/d1a/f6d 0 2024-06-03T01:21:03.439 INFO:tasks.workunit.client.0.smithi032.stdout:8/363: dread dc/de/d1e/f2e [0,4194304] 0 2024-06-03T01:21:03.439 INFO:tasks.workunit.client.0.smithi032.stdout:8/364: chown dc/d25/d28/d32/f36 7129168 1 2024-06-03T01:21:03.439 INFO:tasks.workunit.client.0.smithi032.stdout:8/365: creat dc/d25/d28/d32/f7d x:0 0 0 2024-06-03T01:21:03.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/477: creat da/dd/f92 x:0 0 0 2024-06-03T01:21:03.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/478: creat da/dd/df/d10/d16/d25/d74/f93 x:0 0 0 2024-06-03T01:21:03.445 INFO:tasks.workunit.client.1.smithi044.stdout:4/996: dwrite d17/d1c/f118 [0,4194304] 0 2024-06-03T01:21:03.445 INFO:tasks.workunit.client.1.smithi044.stdout:8/990: link d5/d20/d33/f63 d5/d20/df5/f149 0 2024-06-03T01:21:03.445 INFO:tasks.workunit.client.1.smithi044.stdout:8/991: dread - d5/d55/fac zero size 2024-06-03T01:21:03.446 INFO:tasks.workunit.client.0.smithi032.stdout:4/412: dwrite d4/db/da/f31 [0,4194304] 0 2024-06-03T01:21:03.446 INFO:tasks.workunit.client.0.smithi032.stdout:4/413: readlink d4/db/da/d2d/d27/d53/l72 0 2024-06-03T01:21:03.447 INFO:tasks.workunit.client.0.smithi032.stdout:4/414: symlink d4/l97 0 2024-06-03T01:21:03.447 INFO:tasks.workunit.client.0.smithi032.stdout:4/415: dread - d4/db/da/f96 zero size 2024-06-03T01:21:03.448 INFO:tasks.workunit.client.1.smithi044.stdout:3/863: getdents df/d13/d1b/d56/d5c/d98/df4 0 2024-06-03T01:21:03.448 INFO:tasks.workunit.client.1.smithi044.stdout:3/864: write df/fc9 [242504,62611] 0 2024-06-03T01:21:03.448 INFO:tasks.workunit.client.1.smithi044.stdout:3/865: write df/d13/d1b/d56/f67 [239884,83760] 0 2024-06-03T01:21:03.448 INFO:tasks.workunit.client.1.smithi044.stdout:3/866: write df/d1e/d2a/d70/fba [848704,113123] 0 2024-06-03T01:21:03.448 INFO:tasks.workunit.client.0.smithi032.stdout:4/416: mknod d4/db/da/d2d/d27/c98 0 2024-06-03T01:21:03.449 INFO:tasks.workunit.client.1.smithi044.stdout:7/961: read d0/d3/d8/d95/d63/d7f/faf [6599985,78283] 0 2024-06-03T01:21:03.450 INFO:tasks.workunit.client.0.smithi032.stdout:4/417: symlink d4/db/da/d2d/l99 0 2024-06-03T01:21:03.450 INFO:tasks.workunit.client.0.smithi032.stdout:4/418: chown d4/l69 3 1 2024-06-03T01:21:03.456 INFO:tasks.workunit.client.1.smithi044.stdout:4/997: rmdir d17/d1c/d75/d7a/db3 39 2024-06-03T01:21:03.456 INFO:tasks.workunit.client.1.smithi044.stdout:4/998: write d17/d1c/d75/d7a/db8/d35/fce [503431,105972] 0 2024-06-03T01:21:03.457 INFO:tasks.workunit.client.1.smithi044.stdout:4/999: truncate d17/d63/fd0 9245501 0 2024-06-03T01:21:03.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/992: mkdir d5/d16/d81/de3/d14a 0 2024-06-03T01:21:03.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/993: readlink d5/d55/d56/d5c/d68/laf 0 2024-06-03T01:21:03.458 INFO:tasks.workunit.client.0.smithi032.stdout:2/438: dwrite d5/da/d1a/f68 [0,4194304] 0 2024-06-03T01:21:03.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/409: dwrite d1/da/f4d [0,4194304] 0 2024-06-03T01:21:03.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/836: dwrite db/d15/d49/f5c [4194304,4194304] 0 2024-06-03T01:21:03.460 INFO:tasks.workunit.client.0.smithi032.stdout:6/403: sync 2024-06-03T01:21:03.461 INFO:tasks.workunit.client.1.smithi044.stdout:9/915: dwrite d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68/fbd [0,4194304] 0 2024-06-03T01:21:03.464 INFO:tasks.workunit.client.0.smithi032.stdout:2/439: truncate d5/da/d1a/d2a/d49/d7e/f64 578961 0 2024-06-03T01:21:03.466 INFO:tasks.workunit.client.0.smithi032.stdout:8/366: dwrite dc/d25/d28/d58/f79 [0,4194304] 0 2024-06-03T01:21:03.466 INFO:tasks.workunit.client.1.smithi044.stdout:3/867: mknod df/d13/d6d/d9a/dad/c126 0 2024-06-03T01:21:03.467 INFO:tasks.workunit.client.1.smithi044.stdout:7/962: rename d0/d3/d8/d95/d63/de6/c107 to d0/d3/d8/d95/d110/c12e 0 2024-06-03T01:21:03.467 INFO:tasks.workunit.client.1.smithi044.stdout:3/868: fsync df/d13/d1b/f73 0 2024-06-03T01:21:03.468 INFO:tasks.workunit.client.0.smithi032.stdout:0/410: creat d1/da/d28/d3b/d68/f85 x:0 0 0 2024-06-03T01:21:03.468 INFO:tasks.workunit.client.0.smithi032.stdout:0/411: readlink d1/da/d10/d29/l7d 0 2024-06-03T01:21:03.469 INFO:tasks.workunit.client.0.smithi032.stdout:4/419: dwrite d4/db/da/f2c [0,4194304] 0 2024-06-03T01:21:03.469 INFO:tasks.workunit.client.1.smithi044.stdout:8/994: dread d5/d16/d81/de3/d58/d89/fb9 [0,4194304] 0 2024-06-03T01:21:03.470 INFO:tasks.workunit.client.1.smithi044.stdout:8/995: chown d5/d55/d74/d82/db0/c111 210795411 1 2024-06-03T01:21:03.473 INFO:tasks.workunit.client.0.smithi032.stdout:2/440: rename d5/f8d to d5/d38/d46/d5c/d6c/d89/f93 0 2024-06-03T01:21:03.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/869: dread df/d13/d1b/fae [8388608,4194304] 0 2024-06-03T01:21:03.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/870: fdatasync df/d13/d1b/d4b/d69/f95 0 2024-06-03T01:21:03.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/871: stat df/d1e/d2a/d9b/dc2 0 2024-06-03T01:21:03.479 INFO:tasks.workunit.client.0.smithi032.stdout:8/367: creat dc/d25/d28/d33/d78/f7e x:0 0 0 2024-06-03T01:21:03.480 INFO:tasks.workunit.client.0.smithi032.stdout:0/412: symlink d1/da/d28/d3b/l86 0 2024-06-03T01:21:03.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/479: getdents da/dd/df/d10/d16/d25/d74 0 2024-06-03T01:21:03.488 INFO:tasks.workunit.client.0.smithi032.stdout:8/368: creat dc/de/d1e/d43/f7f x:0 0 0 2024-06-03T01:21:03.488 INFO:tasks.workunit.client.0.smithi032.stdout:8/369: chown dc/d25/d28/d33/l41 0 1 2024-06-03T01:21:03.488 INFO:tasks.workunit.client.0.smithi032.stdout:2/441: dread d5/da/f20 [4194304,4194304] 0 2024-06-03T01:21:03.489 INFO:tasks.workunit.client.0.smithi032.stdout:6/404: dwrite d8/d11/f49 [0,4194304] 0 2024-06-03T01:21:03.489 INFO:tasks.workunit.client.1.smithi044.stdout:2/837: write db/f26 [3903972,85695] 0 2024-06-03T01:21:03.490 INFO:tasks.workunit.client.1.smithi044.stdout:3/872: dread df/d13/d39/d3d/d43/d71/f83 [0,4194304] 0 2024-06-03T01:21:03.490 INFO:tasks.workunit.client.1.smithi044.stdout:7/963: symlink d0/d3/da8/l12f 0 2024-06-03T01:21:03.493 INFO:tasks.workunit.client.0.smithi032.stdout:8/370: dread dc/de/d1e/f61 [0,4194304] 0 2024-06-03T01:21:03.493 INFO:tasks.workunit.client.0.smithi032.stdout:0/413: rename d1/da/c26 to d1/da/d28/d3b/d50/c87 0 2024-06-03T01:21:03.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/420: dwrite d4/db/da/f7d [0,4194304] 0 2024-06-03T01:21:03.494 INFO:tasks.workunit.client.1.smithi044.stdout:8/996: rename d5/d16/d1d/f42 to d5/d55/d56/d5c/d68/f14b 0 2024-06-03T01:21:03.494 INFO:tasks.workunit.client.1.smithi044.stdout:8/997: creat d5/d20/d47/d64/dd0/f14c x:0 0 0 2024-06-03T01:21:03.495 INFO:tasks.workunit.client.0.smithi032.stdout:5/480: creat da/dd/df/d10/d16/d18/d1a/d30/f94 x:0 0 0 2024-06-03T01:21:03.495 INFO:tasks.workunit.client.0.smithi032.stdout:5/481: fsync da/dd/df/d10/d16/d25/f56 0 2024-06-03T01:21:03.495 INFO:tasks.workunit.client.1.smithi044.stdout:7/964: write d0/d3/d8/d32/d33/f67 [4342146,65114] 0 2024-06-03T01:21:03.497 INFO:tasks.workunit.client.0.smithi032.stdout:6/405: dread d8/d11/f3e [4194304,4194304] 0 2024-06-03T01:21:03.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/442: unlink d5/f19 0 2024-06-03T01:21:03.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/443: fsync d5/da/d1a/d2a/d36/d3f/f51 0 2024-06-03T01:21:03.504 INFO:tasks.workunit.client.1.smithi044.stdout:7/965: write d0/d3/d8/d95/d63/d7f/f82 [2393926,46881] 0 2024-06-03T01:21:03.504 INFO:tasks.workunit.client.0.smithi032.stdout:0/414: stat d1/da/d28/c3f 0 2024-06-03T01:21:03.505 INFO:tasks.workunit.client.1.smithi044.stdout:2/838: link db/d15/d49/d6c/d72/fd1 db/d15/d49/d6c/d72/d7a/f107 0 2024-06-03T01:21:03.505 INFO:tasks.workunit.client.0.smithi032.stdout:4/421: mkdir d4/db/da/d2d/d27/d8c/d9a 0 2024-06-03T01:21:03.507 INFO:tasks.workunit.client.0.smithi032.stdout:4/422: dread d4/db/da/f1b [0,4194304] 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/482: rmdir da/dd/df/d10/d8d 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/483: fdatasync da/dd/df/d10/d16/d18/d1a/d30/f94 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/484: creat da/dd/df/d10/d16/d18/d35/d70/f95 x:0 0 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/485: write da/dd/df/d10/d16/d25/f56 [685404,59632] 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/486: write da/dd/df/d10/d16/d18/d35/f5d [11956,106870] 0 2024-06-03T01:21:03.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/487: write da/f73 [994989,105195] 0 2024-06-03T01:21:03.512 INFO:tasks.workunit.client.1.smithi044.stdout:8/998: mkdir d5/d20/d33/dfd/d14d 0 2024-06-03T01:21:03.513 INFO:tasks.workunit.client.0.smithi032.stdout:6/406: truncate d8/d11/f17 850645 0 2024-06-03T01:21:03.516 INFO:tasks.workunit.client.0.smithi032.stdout:4/423: write d4/db/da/d1d/f23 [7797292,81100] 0 2024-06-03T01:21:03.516 INFO:tasks.workunit.client.0.smithi032.stdout:4/424: chown d4/db/da/d37/l45 51 1 2024-06-03T01:21:03.516 INFO:tasks.workunit.client.0.smithi032.stdout:4/425: dread - d4/d57/d6c/f73 zero size 2024-06-03T01:21:03.516 INFO:tasks.workunit.client.0.smithi032.stdout:4/426: write d4/db/da/d1d/f4c [931055,109977] 0 2024-06-03T01:21:03.517 INFO:tasks.workunit.client.0.smithi032.stdout:4/427: readlink d4/db/da/d37/l88 0 2024-06-03T01:21:03.517 INFO:tasks.workunit.client.0.smithi032.stdout:4/428: dread - d4/db/f2e zero size 2024-06-03T01:21:03.518 INFO:tasks.workunit.client.0.smithi032.stdout:2/444: link d5/da/f29 d5/da/d1a/d2a/d36/f94 0 2024-06-03T01:21:03.519 INFO:tasks.workunit.client.1.smithi044.stdout:7/966: symlink d0/d3/d8/d95/d63/d7f/d9c/d6e/d122/l130 0 2024-06-03T01:21:03.519 INFO:tasks.workunit.client.1.smithi044.stdout:7/967: write d0/d3/d8/d95/d63/f11d [5744295,98203] 0 2024-06-03T01:21:03.520 INFO:tasks.workunit.client.1.smithi044.stdout:2/839: mknod db/d1b/d2a/d2f/dbe/c108 0 2024-06-03T01:21:03.520 INFO:tasks.workunit.client.0.smithi032.stdout:0/415: link d1/da/d28/d3b/c78 d1/da/d28/c88 0 2024-06-03T01:21:03.521 INFO:tasks.workunit.client.1.smithi044.stdout:3/873: getdents df/d13/d1b/d4b 0 2024-06-03T01:21:03.521 INFO:tasks.workunit.client.1.smithi044.stdout:8/999: write d5/d16/d81/de3/d58/f84 [2599482,71199] 0 2024-06-03T01:21:03.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/874: truncate df/fc9 1106577 0 2024-06-03T01:21:03.526 INFO:tasks.workunit.client.0.smithi032.stdout:4/429: write d4/db/da/d1d/f75 [1509124,97748] 0 2024-06-03T01:21:03.527 INFO:tasks.workunit.client.1.smithi044.stdout:2/840: fdatasync db/f19 0 2024-06-03T01:21:03.527 INFO:tasks.workunit.client.0.smithi032.stdout:5/488: symlink da/dd/df/d10/d16/l96 0 2024-06-03T01:21:03.527 INFO:tasks.workunit.client.0.smithi032.stdout:5/489: write da/dd/df/d10/d16/d18/d35/d44/f4f [488433,84777] 0 2024-06-03T01:21:03.529 INFO:tasks.workunit.client.0.smithi032.stdout:2/445: read d5/da/d1a/f4b [1444058,115407] 0 2024-06-03T01:21:03.529 INFO:tasks.workunit.client.1.smithi044.stdout:5/954: sync 2024-06-03T01:21:03.530 INFO:tasks.workunit.client.1.smithi044.stdout:3/875: write df/d1e/d2a/f36 [1402271,124652] 0 2024-06-03T01:21:03.532 INFO:tasks.workunit.client.0.smithi032.stdout:8/371: rename dc/de/d1e/d43/d4e to dc/d14/d31/d3b/d80 0 2024-06-03T01:21:03.532 INFO:tasks.workunit.client.0.smithi032.stdout:8/372: fsync dc/de/f11 0 2024-06-03T01:21:03.533 INFO:tasks.workunit.client.1.smithi044.stdout:7/968: dread d0/d3/f3f [0,4194304] 0 2024-06-03T01:21:03.533 INFO:tasks.workunit.client.1.smithi044.stdout:7/969: chown d0/d3/d8/d95/f53 130528 1 2024-06-03T01:21:03.533 INFO:tasks.workunit.client.1.smithi044.stdout:2/841: creat db/d15/d49/d6c/d72/d7a/d4e/f109 x:0 0 0 2024-06-03T01:21:03.533 INFO:tasks.workunit.client.0.smithi032.stdout:4/430: symlink d4/l9b 0 2024-06-03T01:21:03.535 INFO:tasks.workunit.client.0.smithi032.stdout:5/490: creat da/dd/df/f97 x:0 0 0 2024-06-03T01:21:03.536 INFO:tasks.workunit.client.1.smithi044.stdout:5/955: unlink d2/d11/d12/d2b/d4a/ce3 0 2024-06-03T01:21:03.537 INFO:tasks.workunit.client.1.smithi044.stdout:5/956: stat d2/d11/d1e/d3b/da6/f118 0 2024-06-03T01:21:03.538 INFO:tasks.workunit.client.1.smithi044.stdout:9/916: sync 2024-06-03T01:21:03.539 INFO:tasks.workunit.client.1.smithi044.stdout:3/876: symlink df/d13/d6d/d9a/dad/d125/l127 0 2024-06-03T01:21:03.539 INFO:tasks.workunit.client.1.smithi044.stdout:3/877: fdatasync df/d1e/d3a/d84/f10e 0 2024-06-03T01:21:03.540 INFO:tasks.workunit.client.0.smithi032.stdout:6/407: rename d8/la to d8/d11/d20/d3b/d66/l7b 0 2024-06-03T01:21:03.541 INFO:tasks.workunit.client.1.smithi044.stdout:7/970: creat d0/d3/da8/de5/f131 x:0 0 0 2024-06-03T01:21:03.543 INFO:tasks.workunit.client.1.smithi044.stdout:2/842: rename db/d1b/d27/da5/cde to db/d15/d49/c10a 0 2024-06-03T01:21:03.543 INFO:tasks.workunit.client.0.smithi032.stdout:4/431: write d4/db/da/d2d/f55 [924425,92336] 0 2024-06-03T01:21:03.543 INFO:tasks.workunit.client.0.smithi032.stdout:8/373: link dc/de/l21 dc/d14/d31/d3b/d80/l81 0 2024-06-03T01:21:03.545 INFO:tasks.workunit.client.0.smithi032.stdout:5/491: creat da/dd/df/d10/d16/d18/d35/f98 x:0 0 0 2024-06-03T01:21:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.546+0000 7f50309ff700 1 -- 172.21.15.32:0/3731414768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 msgr2=0x7f50240a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:03.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.546+0000 7f50309ff700 1 --2- 172.21.15.32:0/3731414768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f50240a5fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f502c050080 tx=0x7f502c069a10 comp rx=0 tx=0).stop 2024-06-03T01:21:03.550 INFO:tasks.workunit.client.1.smithi044.stdout:9/917: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dcd/d124 0 2024-06-03T01:21:03.550 INFO:tasks.workunit.client.1.smithi044.stdout:3/878: truncate df/d13/d1b/d4b/f54 1713068 0 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.547+0000 7f50309ff700 1 -- 172.21.15.32:0/3731414768 shutdown_connections 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.547+0000 7f50309ff700 1 --2- 172.21.15.32:0/3731414768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f50240a5fe0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.547+0000 7f50309ff700 1 --2- 172.21.15.32:0/3731414768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 0x7f50240a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.547+0000 7f50309ff700 1 -- 172.21.15.32:0/3731414768 >> 172.21.15.32:0/3731414768 conn(0x7f502409fee0 msgr2=0x7f50240a2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.548+0000 7f50309ff700 1 -- 172.21.15.32:0/3731414768 shutdown_connections 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.548+0000 7f50309ff700 1 -- 172.21.15.32:0/3731414768 wait complete. 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.549+0000 7f50309ff700 1 Processor -- start 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.550+0000 7f50309ff700 1 -- start start 2024-06-03T01:21:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.550+0000 7f50309ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 0x7f5024142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.550+0000 7f50309ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.550+0000 7f50309ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50241435d0 con 0x7f50240a5b60 2024-06-03T01:21:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.550+0000 7f50309ff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024143710 con 0x7f50240a4a20 2024-06-03T01:21:03.553 INFO:tasks.workunit.client.1.smithi044.stdout:7/971: mkdir d0/d3/d8/d95/d63/d7f/d9c/d6e/d122/d132 0 2024-06-03T01:21:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.553+0000 7f5029d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.553+0000 7f5029d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49120/0 (socket says 172.21.15.32:49120) 2024-06-03T01:21:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.553+0000 7f5029d9b700 1 -- 172.21.15.32:0/2843681611 learned_addr learned my addr 172.21.15.32:0/2843681611 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:03.555 INFO:tasks.workunit.client.1.smithi044.stdout:5/957: getdents d2/d11/d1e/d3b 0 2024-06-03T01:21:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.553+0000 7f502a59c700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 0x7f5024142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.558+0000 7f5029d9b700 1 -- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 msgr2=0x7f5024142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.558+0000 7f5029d9b700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 0x7f5024142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.558+0000 7f5029d9b700 1 -- 172.21.15.32:0/2843681611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c067050 con 0x7f50240a5b60 2024-06-03T01:21:03.559 INFO:tasks.workunit.client.1.smithi044.stdout:9/918: creat d6/dd/d51/d16/d27/d5e/db7/dfd/f125 x:0 0 0 2024-06-03T01:21:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.560+0000 7f5029d9b700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f502c050430 tx=0x7f502c066e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:03.561 INFO:tasks.workunit.client.1.smithi044.stdout:5/958: rename d2/d11/d3e/dce/ld9 to d2/d11/d1e/dda/l13e 0 2024-06-03T01:21:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.562+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f502c078030 con 0x7f50240a5b60 2024-06-03T01:21:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.562+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f502c07f430 con 0x7f50240a5b60 2024-06-03T01:21:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.562+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f502c07d5f0 con 0x7f50240a5b60 2024-06-03T01:21:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.563+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5024148160 con 0x7f50240a5b60 2024-06-03T01:21:03.564 INFO:tasks.workunit.client.1.smithi044.stdout:9/919: symlink d6/dd/d51/db/l126 0 2024-06-03T01:21:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.563+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50241486a0 con 0x7f50240a5b60 2024-06-03T01:21:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.565+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50240b5d30 con 0x7f50240a5b60 2024-06-03T01:21:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.565+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) v1 ==== 46426+0+0 (secure 0 0 0) 0x7f502c073a90 con 0x7f50240a5b60 2024-06-03T01:21:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.566+0000 7f501b7fe700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 0x7f501403b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.566+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f502c0b1670 con 0x7f50240a5b60 2024-06-03T01:21:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.566+0000 7f502a59c700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 0x7f501403b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:03.567 INFO:tasks.workunit.client.1.smithi044.stdout:9/920: rmdir d6/dd/d51/d16/d27/d5e/d71/de0 39 2024-06-03T01:21:03.567 INFO:tasks.workunit.client.1.smithi044.stdout:9/921: chown d6/dd/d51/d16/d27/d5e/d71/dc1/fc4 264895086 1 2024-06-03T01:21:03.567 INFO:tasks.workunit.client.1.smithi044.stdout:9/922: creat d6/dd/d51/d16/d116/f127 x:0 0 0 2024-06-03T01:21:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.567+0000 7f502a59c700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 0x7f501403b8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5020006fd0 tx=0x7f5020009190 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:03.568 INFO:tasks.workunit.client.1.smithi044.stdout:5/959: dread d2/d11/d12/dad/f10e [0,4194304] 0 2024-06-03T01:21:03.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/923: mknod d6/dd/d51/d16/d27/d5e/db7/de3/c128 0 2024-06-03T01:21:03.570 INFO:tasks.workunit.client.1.smithi044.stdout:9/924: chown d6/dd/d51/d46/l7f 64 1 2024-06-03T01:21:03.570 INFO:tasks.workunit.client.0.smithi032.stdout:6/408: dwrite d8/d55/f7a [0,4194304] 0 2024-06-03T01:21:03.570 INFO:tasks.workunit.client.1.smithi044.stdout:9/925: stat d6/dd/d51/db/l25 0 2024-06-03T01:21:03.571 INFO:tasks.workunit.client.1.smithi044.stdout:9/926: write d6/dd/d51/d16/d5f/d8b/f10e [928787,102912] 0 2024-06-03T01:21:03.572 INFO:tasks.workunit.client.1.smithi044.stdout:9/927: mknod d6/d38/c129 0 2024-06-03T01:21:03.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/446: rmdir d5 39 2024-06-03T01:21:03.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/447: readlink d5/da/l1f 0 2024-06-03T01:21:03.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/448: fsync d5/d38/f3b 0 2024-06-03T01:21:03.574 INFO:tasks.workunit.client.1.smithi044.stdout:9/928: creat d6/dd/d51/d16/d27/d5e/db7/de1/f12a x:0 0 0 2024-06-03T01:21:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:03.576+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f502c066640 con 0x7f50240a5b60 2024-06-03T01:21:03.576 INFO:tasks.workunit.client.0.smithi032.stdout:0/416: rename d1/da/d28/c4c to d1/da/d10/d29/c89 0 2024-06-03T01:21:03.578 INFO:tasks.workunit.client.0.smithi032.stdout:5/492: getdents da/dd/df/d10/d16/d18/d1a/d30 0 2024-06-03T01:21:03.578 INFO:tasks.workunit.client.0.smithi032.stdout:4/432: dwrite d4/db/f68 [0,4194304] 0 2024-06-03T01:21:03.579 INFO:tasks.workunit.client.1.smithi044.stdout:7/972: dwrite d0/d3/d8/d32/f90 [0,4194304] 0 2024-06-03T01:21:03.579 INFO:tasks.workunit.client.1.smithi044.stdout:2/843: dwrite db/d15/d49/d6c/d72/d8f/fda [0,4194304] 0 2024-06-03T01:21:03.579 INFO:tasks.workunit.client.1.smithi044.stdout:9/929: getdents d6/dd/d51/d16/d27/d5e/db7 0 2024-06-03T01:21:03.580 INFO:tasks.workunit.client.1.smithi044.stdout:9/930: write d6/dd/d51/d16/f1b [596626,6395] 0 2024-06-03T01:21:03.580 INFO:tasks.workunit.client.0.smithi032.stdout:6/409: dread d8/d11/d20/d2b/d39/d19/f46 [0,4194304] 0 2024-06-03T01:21:03.580 INFO:tasks.workunit.client.0.smithi032.stdout:6/410: readlink d8/d11/d20/d2b/l59 0 2024-06-03T01:21:03.580 INFO:tasks.workunit.client.0.smithi032.stdout:6/411: fsync d8/d11/f34 0 2024-06-03T01:21:03.581 INFO:tasks.workunit.client.0.smithi032.stdout:6/412: fsync d8/d11/d20/d2b/f4a 0 2024-06-03T01:21:03.581 INFO:tasks.workunit.client.0.smithi032.stdout:0/417: rename d1/da/d10/d53/f79 to d1/da/d28/d3b/d50/f8a 0 2024-06-03T01:21:03.583 INFO:tasks.workunit.client.1.smithi044.stdout:5/960: dread d2/d11/d1e/d21/d24/d9e/d107/f5f [0,4194304] 0 2024-06-03T01:21:03.583 INFO:tasks.workunit.client.1.smithi044.stdout:5/961: read - d2/d11/d1e/d21/d24/d93/f139 zero size 2024-06-03T01:21:03.583 INFO:tasks.workunit.client.0.smithi032.stdout:4/433: link d4/db/da/f82 d4/d57/f9c 0 2024-06-03T01:21:03.583 INFO:tasks.workunit.client.0.smithi032.stdout:4/434: write d4/db/da/d1d/f7e [54823,24701] 0 2024-06-03T01:21:03.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/844: chown db/l14 551219 1 2024-06-03T01:21:03.585 INFO:tasks.workunit.client.1.smithi044.stdout:7/973: symlink d0/d3/da8/de5/l133 0 2024-06-03T01:21:03.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/413: link d8/le d8/d11/d20/d31/l7c 0 2024-06-03T01:21:03.623 INFO:tasks.workunit.client.0.smithi032.stdout:0/418: creat d1/da/d28/f8b x:0 0 0 2024-06-03T01:21:03.624 INFO:tasks.workunit.client.0.smithi032.stdout:0/419: chown d1/da/f1a 2705803 1 2024-06-03T01:21:03.624 INFO:tasks.workunit.client.0.smithi032.stdout:0/420: chown d1/lf 923 1 2024-06-03T01:21:03.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/374: rename dc/de/d1e/d43/f7f to dc/d25/d28/d33/d78/f82 0 2024-06-03T01:21:03.628 INFO:tasks.workunit.client.0.smithi032.stdout:2/449: dwrite d5/ff [0,4194304] 0 2024-06-03T01:21:03.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/435: mknod d4/db/da/c9d 0 2024-06-03T01:21:03.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/436: readlink d4/db/da/d5d/l7a 0 2024-06-03T01:21:03.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/437: read d4/db/da/d2d/d27/f2b [530458,83110] 0 2024-06-03T01:21:03.630 INFO:tasks.workunit.client.0.smithi032.stdout:6/414: getdents d8/d11/d20/d21 0 2024-06-03T01:21:03.632 INFO:tasks.workunit.client.1.smithi044.stdout:9/931: dwrite d6/dd/d51/f13 [0,4194304] 0 2024-06-03T01:21:03.632 INFO:tasks.workunit.client.1.smithi044.stdout:9/932: creat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d67/f12b x:0 0 0 2024-06-03T01:21:03.633 INFO:tasks.workunit.client.1.smithi044.stdout:5/962: creat d2/d11/d1e/d21/d24/d93/d11e/f13f x:0 0 0 2024-06-03T01:21:03.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/421: symlink d1/d46/d3a/l8c 0 2024-06-03T01:21:03.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/974: mknod d0/d3/da8/c134 0 2024-06-03T01:21:03.635 INFO:tasks.workunit.client.1.smithi044.stdout:7/975: readlink d0/d3/d8/d32/l5d 0 2024-06-03T01:21:03.636 INFO:tasks.workunit.client.0.smithi032.stdout:5/493: rename da/dd/f29 to da/dd/df/d10/f99 0 2024-06-03T01:21:03.643 INFO:tasks.workunit.client.1.smithi044.stdout:5/963: dread d2/d11/d12/f2d [0,4194304] 0 2024-06-03T01:21:03.644 INFO:tasks.workunit.client.1.smithi044.stdout:2/845: truncate db/d15/d49/d6c/d72/d8f/fda 1490433 0 2024-06-03T01:21:03.644 INFO:tasks.workunit.client.1.smithi044.stdout:2/846: fdatasync db/d1b/f36 0 2024-06-03T01:21:03.645 INFO:tasks.workunit.client.0.smithi032.stdout:4/438: link d4/db/da/d37/l47 d4/db/da/d2d/d27/l9e 0 2024-06-03T01:21:03.646 INFO:tasks.workunit.client.0.smithi032.stdout:6/415: chown d8/d55/c3a 3 1 2024-06-03T01:21:03.646 INFO:tasks.workunit.client.0.smithi032.stdout:6/416: truncate d8/d11/f34 959020 0 2024-06-03T01:21:03.647 INFO:tasks.workunit.client.0.smithi032.stdout:0/422: mkdir d1/da/d28/d3b/d5d/d8d 0 2024-06-03T01:21:03.647 INFO:tasks.workunit.client.0.smithi032.stdout:0/423: dread - d1/da/d28/f8b zero size 2024-06-03T01:21:03.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/494: mknod da/dd/df/d10/d16/d18/d1a/d30/d51/c9a 0 2024-06-03T01:21:03.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/495: fdatasync da/dd/df/d10/d16/d18/d1a/d30/d51/f91 0 2024-06-03T01:21:03.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/496: chown da/dd/df/d10/f42 250185 1 2024-06-03T01:21:03.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/497: write da/dd/df/d10/f39 [197895,109226] 0 2024-06-03T01:21:03.648 INFO:tasks.workunit.client.1.smithi044.stdout:7/976: write d0/d3/d8/d95/d63/d89/d8b/fab [2481038,62864] 0 2024-06-03T01:21:03.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/933: rename d6/d38 to d6/dd/d51/d16/d27/d5e/da0/d12c 0 2024-06-03T01:21:03.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/934: fdatasync d6/dd/d51/d16/d27/d5e/da0/d12c/ff4 0 2024-06-03T01:21:03.653 INFO:tasks.workunit.client.0.smithi032.stdout:2/450: dwrite d5/da/d1a/d2a/d49/d7e/f73 [0,4194304] 0 2024-06-03T01:21:03.653 INFO:tasks.workunit.client.0.smithi032.stdout:4/439: dread d4/db/da/d1d/f23 [4194304,4194304] 0 2024-06-03T01:21:03.653 INFO:tasks.workunit.client.1.smithi044.stdout:5/964: rename d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/d117/f122 to d2/d11/d3e/dce/f140 0 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/498: mknod da/dd/df/d10/d16/d18/d1a/d30/d58/c9b 0 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/499: write da/dd/df/d10/d16/d25/d74/f88 [39708,22833] 0 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/500: chown da/dd/df/d10/d16/d18/d1a/d30/f5f 0 1 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/501: chown da/dd/df/d10/d16/d18/d35/d70/f95 1536663 1 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/502: getdents da/dd/df/d10/d16/d18/d1a/d30 0 2024-06-03T01:21:03.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/503: chown da/l1e 2340 1 2024-06-03T01:21:03.656 INFO:tasks.workunit.client.1.smithi044.stdout:9/935: mknod d6/dd/d51/d16/dee/c12d 0 2024-06-03T01:21:03.657 INFO:tasks.workunit.client.1.smithi044.stdout:9/936: creat d6/dd/d51/d16/d27/d5e/d71/d89/f12e x:0 0 0 2024-06-03T01:21:03.659 INFO:tasks.workunit.client.1.smithi044.stdout:7/977: link d0/d3/d8/d95/d63/l87 d0/d3/d8/l135 0 2024-06-03T01:21:03.660 INFO:tasks.workunit.client.1.smithi044.stdout:5/965: mkdir d2/d11/d1e/d21/d24/d93/d9d/d28/dfe/d141 0 2024-06-03T01:21:03.662 INFO:tasks.workunit.client.1.smithi044.stdout:2/847: rename db/d1b/d2a/lc7 to db/d1b/d2a/d67/d53/d9b/l10b 0 2024-06-03T01:21:03.662 INFO:tasks.workunit.client.0.smithi032.stdout:4/440: creat d4/db/da/d2d/f9f x:0 0 0 2024-06-03T01:21:03.663 INFO:tasks.workunit.client.0.smithi032.stdout:4/441: write d4/db/da/d1d/f21 [1923212,37124] 0 2024-06-03T01:21:03.664 INFO:tasks.workunit.client.1.smithi044.stdout:3/879: sync 2024-06-03T01:21:03.665 INFO:tasks.workunit.client.1.smithi044.stdout:9/937: truncate d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/fc5 1250518 0 2024-06-03T01:21:03.666 INFO:tasks.workunit.client.0.smithi032.stdout:5/504: unlink da/dd/df/d10/d16/d18/d1a/f6d 0 2024-06-03T01:21:03.666 INFO:tasks.workunit.client.0.smithi032.stdout:2/451: write d5/ff [3930015,100470] 0 2024-06-03T01:21:03.670 INFO:tasks.workunit.client.0.smithi032.stdout:0/424: dwrite d1/f43 [0,4194304] 0 2024-06-03T01:21:03.671 INFO:tasks.workunit.client.0.smithi032.stdout:6/417: dwrite d8/d11/f35 [4194304,4194304] 0 2024-06-03T01:21:03.672 INFO:tasks.workunit.client.0.smithi032.stdout:4/442: creat d4/db/fa0 x:0 0 0 2024-06-03T01:21:03.673 INFO:tasks.workunit.client.1.smithi044.stdout:7/978: rmdir d0/d3/d8/d95/d110/d129 0 2024-06-03T01:21:03.673 INFO:tasks.workunit.client.1.smithi044.stdout:7/979: readlink d0/d3/l52 0 2024-06-03T01:21:03.674 INFO:tasks.workunit.client.1.smithi044.stdout:5/966: creat d2/d11/d12/d2b/d108/f142 x:0 0 0 2024-06-03T01:21:03.675 INFO:tasks.workunit.client.1.smithi044.stdout:5/967: write d2/d11/d1e/d3b/da6/fff [1035782,87110] 0 2024-06-03T01:21:03.675 INFO:tasks.workunit.client.1.smithi044.stdout:5/968: fsync d2/d11/d12/f37 0 2024-06-03T01:21:03.675 INFO:tasks.workunit.client.0.smithi032.stdout:2/452: symlink d5/da/d1a/d2a/d49/d8f/l95 0 2024-06-03T01:21:03.676 INFO:tasks.workunit.client.1.smithi044.stdout:9/938: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d103/d12f 0 2024-06-03T01:21:03.676 INFO:tasks.workunit.client.1.smithi044.stdout:7/980: rmdir d0/d3/da8/de5/d106 39 2024-06-03T01:21:03.676 INFO:tasks.workunit.client.1.smithi044.stdout:9/939: write d6/dd/d51/db/f32 [7561414,83292] 0 2024-06-03T01:21:03.677 INFO:tasks.workunit.client.1.smithi044.stdout:2/848: write db/d15/d49/d6c/fa9 [2942351,23201] 0 2024-06-03T01:21:03.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/880: getdents df/d13/d39/d3d/d6a/d9c 0 2024-06-03T01:21:03.681 INFO:tasks.workunit.client.1.smithi044.stdout:5/969: creat d2/d11/d1e/d21/d24/d9e/db1/f143 x:0 0 0 2024-06-03T01:21:03.683 INFO:tasks.workunit.client.0.smithi032.stdout:6/418: creat d8/d11/d20/d36/f7d x:0 0 0 2024-06-03T01:21:03.683 INFO:tasks.workunit.client.0.smithi032.stdout:6/419: readlink d8/d5f/l72 0 2024-06-03T01:21:03.684 INFO:tasks.workunit.client.1.smithi044.stdout:7/981: creat d0/d3/d30/f136 x:0 0 0 2024-06-03T01:21:03.684 INFO:tasks.workunit.client.1.smithi044.stdout:5/970: dread d2/d11/d1e/d3b/da6/fd6 [0,4194304] 0 2024-06-03T01:21:03.684 INFO:tasks.workunit.client.1.smithi044.stdout:5/971: chown d2/d11/d1e/d21/d24/d93/d9d/c86 16 1 2024-06-03T01:21:03.688 INFO:tasks.workunit.client.0.smithi032.stdout:4/443: rename d4/db/da/d2d/d27/d50 to d4/db/da/d37/d90/da1 0 2024-06-03T01:21:03.689 INFO:tasks.workunit.client.1.smithi044.stdout:9/940: link d6/dd/d51/d16/d27/d5e/db7/de3/c128 d6/dd/d51/d16/d27/d5e/da0/d12c/d11a/dd9/c130 0 2024-06-03T01:21:03.689 INFO:tasks.workunit.client.1.smithi044.stdout:2/849: write db/d1b/d2a/d67/d53/d9b/f9e [3297545,114573] 0 2024-06-03T01:21:03.689 INFO:tasks.workunit.client.1.smithi044.stdout:2/850: chown db/d1b/d3f/f41 177471 1 2024-06-03T01:21:03.690 INFO:tasks.workunit.client.0.smithi032.stdout:2/453: creat d5/da/d1a/d69/d83/f96 x:0 0 0 2024-06-03T01:21:03.690 INFO:tasks.workunit.client.1.smithi044.stdout:9/941: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/l131 0 2024-06-03T01:21:03.691 INFO:tasks.workunit.client.1.smithi044.stdout:3/881: dread df/d13/d39/d3d/d50/f6f [0,4194304] 0 2024-06-03T01:21:03.692 INFO:tasks.workunit.client.0.smithi032.stdout:2/454: rename d5/da/d1a/d2a/d49/f7c to d5/d38/f97 0 2024-06-03T01:21:03.693 INFO:tasks.workunit.client.0.smithi032.stdout:2/455: creat d5/d38/d46/d53/d57/d6d/f98 x:0 0 0 2024-06-03T01:21:03.694 INFO:tasks.workunit.client.0.smithi032.stdout:1/363: sync 2024-06-03T01:21:03.694 INFO:tasks.workunit.client.0.smithi032.stdout:1/364: chown d18/d3f/d5f/d6b 1398789 1 2024-06-03T01:21:03.694 INFO:tasks.workunit.client.0.smithi032.stdout:7/382: sync 2024-06-03T01:21:03.696 INFO:tasks.workunit.client.1.smithi044.stdout:2/851: link db/d15/d49/d6c/d72/d7a/f107 db/d1b/d2a/d52/d54/db6/f10c 0 2024-06-03T01:21:03.699 INFO:tasks.workunit.client.0.smithi032.stdout:2/456: link d5/da/d1a/f3e d5/da/d1a/d2a/d49/d7e/d86/f99 0 2024-06-03T01:21:03.699 INFO:tasks.workunit.client.0.smithi032.stdout:2/457: stat d5/d38/d46/d5c/d6c/d80 0 2024-06-03T01:21:03.699 INFO:tasks.workunit.client.1.smithi044.stdout:9/942: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/l132 0 2024-06-03T01:21:03.699 INFO:tasks.workunit.client.0.smithi032.stdout:6/420: dread d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:03.700 INFO:tasks.workunit.client.0.smithi032.stdout:5/505: dwrite da/dd/df/d10/d16/d18/d35/d44/f4f [0,4194304] 0 2024-06-03T01:21:03.700 INFO:tasks.workunit.client.1.smithi044.stdout:9/943: write d6/dd/d51/d46/fa6 [155717,116255] 0 2024-06-03T01:21:03.700 INFO:tasks.workunit.client.1.smithi044.stdout:3/882: creat df/d1e/f128 x:0 0 0 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/852: link db/d15/cd8 db/d15/d49/d6c/d72/dce/c10d 0 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/853: chown db/d15/d49/d6c/d72/d7a/d4e 102635138 1 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/854: stat db/d15/d49/d6c/d72/d8f 0 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/855: write db/d15/d49/d6c/d72/fd1 [1158211,46870] 0 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.0.smithi032.stdout:4/444: write d4/db/da/f7d [3316628,89366] 0 2024-06-03T01:21:03.703 INFO:tasks.workunit.client.0.smithi032.stdout:1/365: getdents d18/d3f 0 2024-06-03T01:21:03.704 INFO:tasks.workunit.client.0.smithi032.stdout:1/366: fsync d18/d2d/d5e/f64 0 2024-06-03T01:21:03.704 INFO:tasks.workunit.client.0.smithi032.stdout:1/367: fsync fa 0 2024-06-03T01:21:03.704 INFO:tasks.workunit.client.1.smithi044.stdout:9/944: rmdir d6/dd/d51/d16/d27/d5e 39 2024-06-03T01:21:03.706 INFO:tasks.workunit.client.1.smithi044.stdout:3/883: symlink df/d13/d39/d3d/d50/l129 0 2024-06-03T01:21:03.706 INFO:tasks.workunit.client.0.smithi032.stdout:0/425: dwrite d1/da/d10/d29/f73 [0,4194304] 0 2024-06-03T01:21:03.711 INFO:tasks.workunit.client.1.smithi044.stdout:2/856: mkdir db/d1b/d5d/da7/dc9/d10e 0 2024-06-03T01:21:03.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/884: mknod df/d13/d1b/d56/d5c/de3/de6/c12a 0 2024-06-03T01:21:03.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/885: stat df/d13/d1b/d4b/d69/f95 0 2024-06-03T01:21:03.716 INFO:tasks.workunit.client.0.smithi032.stdout:5/506: mknod da/dd/df/d10/d16/d18/d35/c9c 0 2024-06-03T01:21:03.718 INFO:tasks.workunit.client.0.smithi032.stdout:6/421: mkdir d8/d11/d20/d2b/d39/d7e 0 2024-06-03T01:21:03.718 INFO:tasks.workunit.client.1.smithi044.stdout:2/857: rename db/d1b/d2a/d2f/dbe/fe2 to db/d15/d49/d6c/df3/f10f 0 2024-06-03T01:21:03.719 INFO:tasks.workunit.client.1.smithi044.stdout:5/972: getdents d2/d11/d1e/d21/d24/d93/d9d/d28/dfe 0 2024-06-03T01:21:03.719 INFO:tasks.workunit.client.0.smithi032.stdout:4/445: write d4/db/da/d2d/d27/d53/f66 [326189,16825] 0 2024-06-03T01:21:03.720 INFO:tasks.workunit.client.1.smithi044.stdout:3/886: mkdir df/d13/d1b/d56/d5c/d98/d12b 0 2024-06-03T01:21:03.721 INFO:tasks.workunit.client.0.smithi032.stdout:1/368: mkdir d18/d29/d3d/d47/d6e 0 2024-06-03T01:21:03.724 INFO:tasks.workunit.client.0.smithi032.stdout:5/507: dread da/f73 [0,4194304] 0 2024-06-03T01:21:03.724 INFO:tasks.workunit.client.0.smithi032.stdout:5/508: stat da/dd/df/d10/f99 0 2024-06-03T01:21:03.724 INFO:tasks.workunit.client.1.smithi044.stdout:5/973: unlink d2/d11/d1e/d21/d24/d93/d9d/d28/d66/fa2 0 2024-06-03T01:21:03.727 INFO:tasks.workunit.client.1.smithi044.stdout:3/887: rename df/d1e/d2a/d70 to df/d1e/d78/dda/d12c 0 2024-06-03T01:21:03.729 INFO:tasks.workunit.client.0.smithi032.stdout:0/426: symlink d1/d46/d3a/l8e 0 2024-06-03T01:21:03.729 INFO:tasks.workunit.client.0.smithi032.stdout:0/427: creat d1/da/f8f x:0 0 0 2024-06-03T01:21:03.730 INFO:tasks.workunit.client.1.smithi044.stdout:9/945: dwrite d6/dd/d45/f56 [4194304,4194304] 0 2024-06-03T01:21:03.730 INFO:tasks.workunit.client.1.smithi044.stdout:9/946: chown d6/dd/d51/d16/d27/d5e/db7/de1/ff2 1 1 2024-06-03T01:21:03.730 INFO:tasks.workunit.client.1.smithi044.stdout:7/982: dwrite d0/d3/d8/d95/d63/f6b [0,4194304] 0 2024-06-03T01:21:03.731 INFO:tasks.workunit.client.0.smithi032.stdout:7/383: dwrite dc/d12/d2a/f43 [0,4194304] 0 2024-06-03T01:21:03.731 INFO:tasks.workunit.client.0.smithi032.stdout:7/384: stat dc/c70 0 2024-06-03T01:21:03.731 INFO:tasks.workunit.client.0.smithi032.stdout:2/458: dwrite d5/f35 [0,4194304] 0 2024-06-03T01:21:03.731 INFO:tasks.workunit.client.0.smithi032.stdout:4/446: chown d4/db/da/d2d/d27/d53/c61 120622 1 2024-06-03T01:21:03.733 INFO:tasks.workunit.client.0.smithi032.stdout:2/459: write d5/da/d1a/f4b [701115,123778] 0 2024-06-03T01:21:03.733 INFO:tasks.workunit.client.0.smithi032.stdout:2/460: truncate d5/da/d1a/d2a/d36/d3f/f51 1605595 0 2024-06-03T01:21:03.734 INFO:tasks.workunit.client.0.smithi032.stdout:1/369: rmdir d18/d32/d39 39 2024-06-03T01:21:03.735 INFO:tasks.workunit.client.0.smithi032.stdout:5/509: truncate da/dd/df/d10/d16/d18/d1a/d30/f5a 3007176 0 2024-06-03T01:21:03.739 INFO:tasks.workunit.client.0.smithi032.stdout:0/428: symlink d1/da/d28/l90 0 2024-06-03T01:21:03.739 INFO:tasks.workunit.client.1.smithi044.stdout:3/888: mkdir df/d12d 0 2024-06-03T01:21:03.741 INFO:tasks.workunit.client.1.smithi044.stdout:2/858: dwrite db/f73 [0,4194304] 0 2024-06-03T01:21:03.744 INFO:tasks.workunit.client.1.smithi044.stdout:9/947: mknod d6/dd/d51/d16/d27/d5e/da0/d12c/d11a/c133 0 2024-06-03T01:21:03.745 INFO:tasks.workunit.client.1.smithi044.stdout:5/974: rename d2/d11/d12/d2b/d4a/d57/l90 to d2/d11/d1e/d3b/da6/l144 0 2024-06-03T01:21:03.745 INFO:tasks.workunit.client.1.smithi044.stdout:5/975: dread - d2/d11/d1e/d21/d24/d93/f139 zero size 2024-06-03T01:21:03.745 INFO:tasks.workunit.client.0.smithi032.stdout:4/447: mkdir d4/db/da/d2d/d27/d8c/d9a/da2 0 2024-06-03T01:21:03.745 INFO:tasks.workunit.client.0.smithi032.stdout:2/461: creat d5/d38/d46/d53/f9a x:0 0 0 2024-06-03T01:21:03.749 INFO:tasks.workunit.client.0.smithi032.stdout:7/385: dread dc/f15 [0,4194304] 0 2024-06-03T01:21:03.750 INFO:tasks.workunit.client.1.smithi044.stdout:2/859: unlink db/dd/l16 0 2024-06-03T01:21:03.754 INFO:tasks.workunit.client.1.smithi044.stdout:9/948: truncate d6/dd/d51/d16/f34 2910686 0 2024-06-03T01:21:03.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/370: write d18/d32/f4b [1440724,97865] 0 2024-06-03T01:21:03.755 INFO:tasks.workunit.client.0.smithi032.stdout:4/448: unlink d4/d5c/f86 0 2024-06-03T01:21:03.755 INFO:tasks.workunit.client.0.smithi032.stdout:4/449: fdatasync d4/db/da/d37/f70 0 2024-06-03T01:21:03.755 INFO:tasks.workunit.client.0.smithi032.stdout:4/450: fdatasync d4/db/da/d1d/d6b/f7c 0 2024-06-03T01:21:03.756 INFO:tasks.workunit.client.1.smithi044.stdout:2/860: chown c8 0 1 2024-06-03T01:21:03.756 INFO:tasks.workunit.client.1.smithi044.stdout:2/861: fdatasync db/d15/d49/d6c/d72/dce/ffb 0 2024-06-03T01:21:03.758 INFO:tasks.workunit.client.0.smithi032.stdout:7/386: link dc/d12/d24/d33/l49 dc/d12/d2a/d2f/l73 0 2024-06-03T01:21:03.759 INFO:tasks.workunit.client.0.smithi032.stdout:7/387: chown dc/f2d 523 1 2024-06-03T01:21:03.759 INFO:tasks.workunit.client.1.smithi044.stdout:2/862: symlink db/d15/l110 0 2024-06-03T01:21:03.760 INFO:tasks.workunit.client.0.smithi032.stdout:6/422: rename d8/d11/d20/d3b/d66/d70 to d8/d11/d20/d7f 0 2024-06-03T01:21:03.762 INFO:tasks.workunit.client.0.smithi032.stdout:0/429: write d1/da/d10/d29/f73 [3063190,92526] 0 2024-06-03T01:21:03.763 INFO:tasks.workunit.client.1.smithi044.stdout:2/863: mkdir db/d1b/d2a/d67/de4/d111 0 2024-06-03T01:21:03.763 INFO:tasks.workunit.client.1.smithi044.stdout:2/864: write db/d1b/d2a/d52/fb4 [5986013,60635] 0 2024-06-03T01:21:03.763 INFO:tasks.workunit.client.1.smithi044.stdout:2/865: creat db/d1b/d27/d79/d91/f112 x:0 0 0 2024-06-03T01:21:03.766 INFO:tasks.workunit.client.0.smithi032.stdout:4/451: creat d4/db/da/d5d/d8e/fa3 x:0 0 0 2024-06-03T01:21:03.769 INFO:tasks.workunit.client.0.smithi032.stdout:7/388: creat dc/dd/f74 x:0 0 0 2024-06-03T01:21:03.769 INFO:tasks.workunit.client.0.smithi032.stdout:7/389: dread - dc/d12/d24/d33/f66 zero size 2024-06-03T01:21:03.769 INFO:tasks.workunit.client.0.smithi032.stdout:7/390: fsync dc/f30 0 2024-06-03T01:21:03.771 INFO:tasks.workunit.client.0.smithi032.stdout:2/462: rename d5/da/d1a/d2a/d49/d7e/f64 to d5/d38/d46/d53/d90/f9b 0 2024-06-03T01:21:03.773 INFO:tasks.workunit.client.1.smithi044.stdout:3/889: rmdir df/d13/d1b/d56/d5c 39 2024-06-03T01:21:03.777 INFO:tasks.workunit.client.0.smithi032.stdout:0/430: unlink d1/d46/f6b 0 2024-06-03T01:21:03.777 INFO:tasks.workunit.client.0.smithi032.stdout:0/431: fsync d1/f64 0 2024-06-03T01:21:03.778 INFO:tasks.workunit.client.1.smithi044.stdout:2/866: mkdir db/d15/d49/d82/d113 0 2024-06-03T01:21:03.779 INFO:tasks.workunit.client.1.smithi044.stdout:9/949: dwrite d6/dd/d51/db/f32 [8388608,4194304] 0 2024-06-03T01:21:03.780 INFO:tasks.workunit.client.1.smithi044.stdout:5/976: rmdir d2/d11/d12/d2b/d4a 39 2024-06-03T01:21:03.781 INFO:tasks.workunit.client.0.smithi032.stdout:4/452: symlink d4/db/da/d2d/d27/d8c/d9a/la4 0 2024-06-03T01:21:03.787 INFO:tasks.workunit.client.1.smithi044.stdout:3/890: mknod df/d12d/c12e 0 2024-06-03T01:21:03.788 INFO:tasks.workunit.client.0.smithi032.stdout:0/432: dread d1/f35 [4194304,4194304] 0 2024-06-03T01:21:03.789 INFO:tasks.workunit.client.0.smithi032.stdout:0/433: truncate d1/da/f4d 288387 0 2024-06-03T01:21:03.792 INFO:tasks.workunit.client.1.smithi044.stdout:2/867: link db/d1b/f22 db/d1b/d2a/d67/de4/f114 0 2024-06-03T01:21:03.795 INFO:tasks.workunit.client.1.smithi044.stdout:9/950: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/l134 0 2024-06-03T01:21:03.795 INFO:tasks.workunit.client.0.smithi032.stdout:7/391: dread dc/d12/d1b/f1d [4194304,4194304] 0 2024-06-03T01:21:03.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/453: write d4/db/da/d1d/d6b/f7c [1474355,104751] 0 2024-06-03T01:21:03.796 INFO:tasks.workunit.client.1.smithi044.stdout:5/977: fdatasync d2/f97 0 2024-06-03T01:21:03.796 INFO:tasks.workunit.client.1.smithi044.stdout:5/978: read - d2/d11/d12/d2b/d4a/d57/d106/f112 zero size 2024-06-03T01:21:03.798 INFO:tasks.workunit.client.0.smithi032.stdout:1/371: dwrite d18/d32/f4b [0,4194304] 0 2024-06-03T01:21:03.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/372: chown d18/d27/f41 18123887 1 2024-06-03T01:21:03.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/373: stat d18/d32/c40 0 2024-06-03T01:21:03.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/374: fsync d18/d2d/f69 0 2024-06-03T01:21:03.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/375: chown d18/d32/c42 0 1 2024-06-03T01:21:03.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/376: chown d18/d29/d3d 357857 1 2024-06-03T01:21:03.805 INFO:tasks.workunit.client.0.smithi032.stdout:4/454: rmdir d4/db/da/d1d/d6b 39 2024-06-03T01:21:03.805 INFO:tasks.workunit.client.0.smithi032.stdout:4/455: fdatasync d4/db/da/d1d/f4c 0 2024-06-03T01:21:03.806 INFO:tasks.workunit.client.1.smithi044.stdout:3/891: mkdir df/d13/d39/d3d/d43/d71/d12f 0 2024-06-03T01:21:03.808 INFO:tasks.workunit.client.0.smithi032.stdout:7/392: dread dc/d12/d2a/f40 [0,4194304] 0 2024-06-03T01:21:03.808 INFO:tasks.workunit.client.1.smithi044.stdout:2/868: rename db/d1b/d2a/d2f/l58 to db/d15/d49/d82/db5/l115 0 2024-06-03T01:21:03.808 INFO:tasks.workunit.client.0.smithi032.stdout:0/434: dwrite d1/f9 [4194304,4194304] 0 2024-06-03T01:21:03.808 INFO:tasks.workunit.client.0.smithi032.stdout:0/435: write d1/f64 [390331,114673] 0 2024-06-03T01:21:03.809 INFO:tasks.workunit.client.0.smithi032.stdout:0/436: write d1/da/d28/d3b/f76 [899832,52133] 0 2024-06-03T01:21:03.811 INFO:tasks.workunit.client.0.smithi032.stdout:7/393: rename dc/d12/d2a/c45 to dc/d12/d2a/d2f/d63/c75 0 2024-06-03T01:21:03.812 INFO:tasks.workunit.client.0.smithi032.stdout:7/394: write dc/dd/f74 [852555,44480] 0 2024-06-03T01:21:03.812 INFO:tasks.workunit.client.1.smithi044.stdout:3/892: mkdir df/d13/d39/d3d/d50/d130 0 2024-06-03T01:21:03.813 INFO:tasks.workunit.client.0.smithi032.stdout:0/437: creat d1/da/d28/d3b/d5d/d8d/f91 x:0 0 0 2024-06-03T01:21:03.815 INFO:tasks.workunit.client.0.smithi032.stdout:7/395: mkdir dc/d12/d2a/d2f/d76 0 2024-06-03T01:21:03.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/438: dread d1/d46/f55 [0,4194304] 0 2024-06-03T01:21:03.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/439: read - d1/da/d28/d3b/d5d/f67 zero size 2024-06-03T01:21:03.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/440: write d1/da/d10/f7a [172328,4870] 0 2024-06-03T01:21:03.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/441: readlink d1/da/d10/d29/l36 0 2024-06-03T01:21:03.819 INFO:tasks.workunit.client.0.smithi032.stdout:0/442: link d1/l38 d1/da/d28/d3b/d5d/d8d/l92 0 2024-06-03T01:21:03.820 INFO:tasks.workunit.client.1.smithi044.stdout:9/951: rename d6/dd/d51/l1d to d6/dd/d51/d16/d27/d5e/d71/de0/l135 0 2024-06-03T01:21:03.821 INFO:tasks.workunit.client.1.smithi044.stdout:9/952: write d6/dd/d51/d16/d27/d5e/db7/de1/ff2 [772318,20726] 0 2024-06-03T01:21:03.821 INFO:tasks.workunit.client.0.smithi032.stdout:0/443: mknod d1/da/d10/d80/c93 0 2024-06-03T01:21:03.822 INFO:tasks.workunit.client.1.smithi044.stdout:7/983: sync 2024-06-03T01:21:03.822 INFO:tasks.workunit.client.1.smithi044.stdout:7/984: fdatasync d0/d3/d8/d95/d63/d7f/faf 0 2024-06-03T01:21:03.822 INFO:tasks.workunit.client.0.smithi032.stdout:1/377: dwrite f17 [0,4194304] 0 2024-06-03T01:21:03.824 INFO:tasks.workunit.client.0.smithi032.stdout:0/444: rename d1/d46/f55 to d1/da/d28/d3b/d5d/f94 0 2024-06-03T01:21:03.824 INFO:tasks.workunit.client.0.smithi032.stdout:0/445: stat d1/da/d28/d3b/d5d/f67 0 2024-06-03T01:21:03.825 INFO:tasks.workunit.client.1.smithi044.stdout:2/869: link db/d15/d49/d6c/fa3 db/d15/d49/d6c/d72/dce/f116 0 2024-06-03T01:21:03.830 INFO:tasks.workunit.client.0.smithi032.stdout:7/396: read dc/f30 [8386698,94010] 0 2024-06-03T01:21:03.833 INFO:tasks.workunit.client.1.smithi044.stdout:5/979: rename d2/d11/d12/d2b/d4a/fca to d2/d11/d1e/d3b/da6/d116/f145 0 2024-06-03T01:21:03.837 INFO:tasks.workunit.client.1.smithi044.stdout:7/985: dread d0/f1d [0,4194304] 0 2024-06-03T01:21:03.837 INFO:tasks.workunit.client.1.smithi044.stdout:7/986: fsync d0/d3/d8/d95/d63/d7f/d9c/f92 0 2024-06-03T01:21:03.838 INFO:tasks.workunit.client.0.smithi032.stdout:7/397: creat dc/d12/f77 x:0 0 0 2024-06-03T01:21:03.839 INFO:tasks.workunit.client.1.smithi044.stdout:9/953: rmdir d6/dd/d51/d16/d5f/d8b/d8d/dac 39 2024-06-03T01:21:03.840 INFO:tasks.workunit.client.0.smithi032.stdout:7/398: mkdir dc/d12/d78 0 2024-06-03T01:21:03.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/870: creat db/d1b/d2a/d52/df0/f117 x:0 0 0 2024-06-03T01:21:03.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/871: chown db/d1b/d5d/da7/dc9 3 1 2024-06-03T01:21:03.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/872: fdatasync db/d1b/d2a/d52/f8b 0 2024-06-03T01:21:03.842 INFO:tasks.workunit.client.1.smithi044.stdout:2/873: dread db/d15/d49/d82/db5/ff4 [0,4194304] 0 2024-06-03T01:21:03.844 INFO:tasks.workunit.client.0.smithi032.stdout:3/413: sync 2024-06-03T01:21:03.845 INFO:tasks.workunit.client.0.smithi032.stdout:7/399: dread dc/f15 [0,4194304] 0 2024-06-03T01:21:03.845 INFO:tasks.workunit.client.1.smithi044.stdout:5/980: creat d2/d11/d1e/d3b/da6/d116/f146 x:0 0 0 2024-06-03T01:21:03.845 INFO:tasks.workunit.client.1.smithi044.stdout:5/981: truncate d2/d11/d1e/d21/d24/faa 133615 0 2024-06-03T01:21:03.847 INFO:tasks.workunit.client.0.smithi032.stdout:3/414: rename de/d29/d2e/d31/d3c/d50/d56/l72 to de/d29/d2e/d31/d3c/l81 0 2024-06-03T01:21:03.848 INFO:tasks.workunit.client.1.smithi044.stdout:7/987: creat d0/d3/da8/f137 x:0 0 0 2024-06-03T01:21:03.848 INFO:tasks.workunit.client.1.smithi044.stdout:7/988: creat d0/d3/da8/de5/f138 x:0 0 0 2024-06-03T01:21:03.848 INFO:tasks.workunit.client.0.smithi032.stdout:7/400: symlink dc/d12/d2a/d2f/d76/l79 0 2024-06-03T01:21:03.850 INFO:tasks.workunit.client.0.smithi032.stdout:7/401: mknod dc/c7a 0 2024-06-03T01:21:03.850 INFO:tasks.workunit.client.0.smithi032.stdout:0/446: dwrite d1/da/d10/f2d [0,4194304] 0 2024-06-03T01:21:03.850 INFO:tasks.workunit.client.0.smithi032.stdout:0/447: dread - d1/da/d28/d3b/d5d/d8d/f91 zero size 2024-06-03T01:21:03.851 INFO:tasks.workunit.client.1.smithi044.stdout:9/954: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/dae/l136 0 2024-06-03T01:21:03.853 INFO:tasks.workunit.client.1.smithi044.stdout:7/989: getdents d0/d3/d30 0 2024-06-03T01:21:03.854 INFO:tasks.workunit.client.1.smithi044.stdout:2/874: dread db/dd/fe [0,4194304] 0 2024-06-03T01:21:03.854 INFO:tasks.workunit.client.1.smithi044.stdout:2/875: dread - db/d1b/d2a/d52/fd9 zero size 2024-06-03T01:21:03.855 INFO:tasks.workunit.client.1.smithi044.stdout:7/990: mkdir d0/d3/d8/d95/d63/d89/d8b/d139 0 2024-06-03T01:21:03.855 INFO:tasks.workunit.client.1.smithi044.stdout:7/991: stat d0/d3/d8/d32/f4a 0 2024-06-03T01:21:03.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/402: rename dc/dd to dc/d12/d24/d7b 0 2024-06-03T01:21:03.856 INFO:tasks.workunit.client.1.smithi044.stdout:2/876: mkdir db/d1b/d118 0 2024-06-03T01:21:03.857 INFO:tasks.workunit.client.1.smithi044.stdout:2/877: mknod db/d15/d49/d7b/c119 0 2024-06-03T01:21:03.857 INFO:tasks.workunit.client.1.smithi044.stdout:2/878: write db/d1b/d3f/f41 [1970462,3515] 0 2024-06-03T01:21:03.857 INFO:tasks.workunit.client.1.smithi044.stdout:2/879: chown db/d15/d49/lc1 226 1 2024-06-03T01:21:03.857 INFO:tasks.workunit.client.0.smithi032.stdout:3/415: dread de/d29/d2e/d31/d3c/d50/d56/f67 [0,4194304] 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/448: write d1/f18 [4592635,82818] 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/449: write d1/f6 [3482,32347] 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/450: truncate d1/da/d28/f33 2979932 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/451: write d1/da/d10/f5b [77473,54831] 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/452: truncate d1/da/f1a 1685799 0 2024-06-03T01:21:03.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/453: chown d1/da/d10/d29/l36 467 1 2024-06-03T01:21:03.859 INFO:tasks.workunit.client.0.smithi032.stdout:3/416: mknod de/d29/d2e/d31/d3c/d50/d56/c82 0 2024-06-03T01:21:03.860 INFO:tasks.workunit.client.0.smithi032.stdout:0/454: creat d1/da/d28/d3b/d68/d83/f95 x:0 0 0 2024-06-03T01:21:03.861 INFO:tasks.workunit.client.0.smithi032.stdout:3/417: truncate de/d29/f65 7726 0 2024-06-03T01:21:03.861 INFO:tasks.workunit.client.0.smithi032.stdout:3/418: write de/d29/f3f [238790,96341] 0 2024-06-03T01:21:03.861 INFO:tasks.workunit.client.0.smithi032.stdout:3/419: stat de/d29/d2e/d31/d3c/d50/d56 0 2024-06-03T01:21:03.863 INFO:tasks.workunit.client.0.smithi032.stdout:3/420: link de/d29/d2e/d31/d24/l6a de/d29/d2e/d31/d3c/d50/d5e/l83 0 2024-06-03T01:21:03.863 INFO:tasks.workunit.client.0.smithi032.stdout:3/421: fdatasync de/d29/d2e/d31/d24/f39 0 2024-06-03T01:21:03.870 INFO:tasks.workunit.client.1.smithi044.stdout:7/992: read d0/d3/f47 [3229925,26417] 0 2024-06-03T01:21:03.870 INFO:tasks.workunit.client.1.smithi044.stdout:7/993: stat d0/d3/d8/d32/f6c 0 2024-06-03T01:21:03.871 INFO:tasks.workunit.client.0.smithi032.stdout:7/403: dread dc/d12/f13 [0,4194304] 0 2024-06-03T01:21:03.871 INFO:tasks.workunit.client.0.smithi032.stdout:7/404: chown dc/d12/d24/d33/f4b 1135599975 1 2024-06-03T01:21:03.871 INFO:tasks.workunit.client.1.smithi044.stdout:7/994: rename d0/d3/d30/c6a to d0/d3/d8/d95/c13a 0 2024-06-03T01:21:03.874 INFO:tasks.workunit.client.1.smithi044.stdout:7/995: dread d0/d3/d8/d95/d63/d89/da3/dde/d10a/f119 [0,4194304] 0 2024-06-03T01:21:03.875 INFO:tasks.workunit.client.0.smithi032.stdout:7/405: rename dc/d12/d22/c23 to dc/d12/d78/c7c 0 2024-06-03T01:21:03.875 INFO:tasks.workunit.client.1.smithi044.stdout:7/996: truncate d0/d3/ddf/f125 2195710 0 2024-06-03T01:21:03.875 INFO:tasks.workunit.client.1.smithi044.stdout:7/997: stat d0/d3/d8/d95/d63/d89/d8b/lcc 0 2024-06-03T01:21:03.876 INFO:tasks.workunit.client.0.smithi032.stdout:7/406: creat dc/d12/d2a/d2f/d76/f7d x:0 0 0 2024-06-03T01:21:03.876 INFO:tasks.workunit.client.0.smithi032.stdout:7/407: readlink l0 0 2024-06-03T01:21:03.877 INFO:tasks.workunit.client.0.smithi032.stdout:7/408: symlink dc/d12/d2a/d2f/d63/l7e 0 2024-06-03T01:21:03.879 INFO:tasks.workunit.client.0.smithi032.stdout:7/409: unlink dc/d12/d24/d7b/lf 0 2024-06-03T01:21:03.880 INFO:tasks.workunit.client.0.smithi032.stdout:3/422: dwrite de/d29/d2e/d31/d24/f73 [0,4194304] 0 2024-06-03T01:21:03.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/410: symlink dc/d12/d78/l7f 0 2024-06-03T01:21:03.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/411: fsync dc/d12/d24/d33/f4b 0 2024-06-03T01:21:03.883 INFO:tasks.workunit.client.0.smithi032.stdout:3/423: creat de/d29/d2e/d31/d23/d53/d69/f84 x:0 0 0 2024-06-03T01:21:03.891 INFO:tasks.workunit.client.0.smithi032.stdout:0/455: fdatasync d1/f6 0 2024-06-03T01:21:03.891 INFO:tasks.workunit.client.0.smithi032.stdout:7/412: write dc/d12/d1b/f68 [1173471,34799] 0 2024-06-03T01:21:03.892 INFO:tasks.workunit.client.1.smithi044.stdout:5/982: dwrite d2/d11/d12/dad/fbf [4194304,4194304] 0 2024-06-03T01:21:03.892 INFO:tasks.workunit.client.1.smithi044.stdout:5/983: readlink d2/d11/d12/d2b/d4a/d57/l11d 0 2024-06-03T01:21:03.895 INFO:tasks.workunit.client.1.smithi044.stdout:2/880: dwrite db/d1b/d5d/f62 [4194304,4194304] 0 2024-06-03T01:21:03.895 INFO:tasks.workunit.client.1.smithi044.stdout:2/881: creat db/d15/d49/d6c/d72/dce/f11a x:0 0 0 2024-06-03T01:21:03.896 INFO:tasks.workunit.client.0.smithi032.stdout:7/413: unlink dc/d12/d2a/d2f/f5d 0 2024-06-03T01:21:03.896 INFO:tasks.workunit.client.1.smithi044.stdout:5/984: getdents d2/d11/df1 0 2024-06-03T01:21:03.897 INFO:tasks.workunit.client.1.smithi044.stdout:7/998: dwrite d0/d3/d8/d95/d63/d7f/d9c/d6e/dc0/dee/f113 [4194304,4194304] 0 2024-06-03T01:21:03.897 INFO:tasks.workunit.client.0.smithi032.stdout:7/414: rmdir dc 39 2024-06-03T01:21:03.897 INFO:tasks.workunit.client.0.smithi032.stdout:7/415: creat dc/d12/d2a/d2f/d76/f80 x:0 0 0 2024-06-03T01:21:03.898 INFO:tasks.workunit.client.0.smithi032.stdout:3/424: write de/d29/f2b [3178611,9036] 0 2024-06-03T01:21:03.899 INFO:tasks.workunit.client.0.smithi032.stdout:3/425: chown de/d29/d2e/l6f 107578766 1 2024-06-03T01:21:03.900 INFO:tasks.workunit.client.1.smithi044.stdout:7/999: getdents d0/d3/d8/d32/d33/dd1 0 2024-06-03T01:21:03.905 INFO:tasks.workunit.client.1.smithi044.stdout:2/882: dread db/d1b/d2a/d52/fb4 [0,4194304] 0 2024-06-03T01:21:03.905 INFO:tasks.workunit.client.0.smithi032.stdout:7/416: mkdir dc/d12/d2a/d81 0 2024-06-03T01:21:03.913 INFO:tasks.workunit.client.0.smithi032.stdout:0/456: dwrite d1/da/f4d [0,4194304] 0 2024-06-03T01:21:03.916 INFO:tasks.workunit.client.1.smithi044.stdout:2/883: write db/d1b/d2a/d52/f8b [525968,112866] 0 2024-06-03T01:21:03.916 INFO:tasks.workunit.client.1.smithi044.stdout:2/884: fsync db/d15/d49/d6c/d72/d7a/f107 0 2024-06-03T01:21:03.919 INFO:tasks.workunit.client.1.smithi044.stdout:2/885: getdents db/d15/d49/d6c/d72/d7a/d4e/d7e 0 2024-06-03T01:21:03.920 INFO:tasks.workunit.client.1.smithi044.stdout:2/886: mkdir db/d15/d49/d7b/df8/d11b 0 2024-06-03T01:21:03.921 INFO:tasks.workunit.client.0.smithi032.stdout:3/426: dread f8 [4194304,4194304] 0 2024-06-03T01:21:03.924 INFO:tasks.workunit.client.0.smithi032.stdout:7/417: dwrite dc/d12/f13 [4194304,4194304] 0 2024-06-03T01:21:03.927 INFO:tasks.workunit.client.0.smithi032.stdout:7/418: mkdir dc/d12/d22/d42/d61/d82 0 2024-06-03T01:21:03.930 INFO:tasks.workunit.client.0.smithi032.stdout:7/419: chown dc/d12/d1b/c46 3 1 2024-06-03T01:21:03.930 INFO:tasks.workunit.client.0.smithi032.stdout:7/420: truncate dc/d12/f6b 1124911 0 2024-06-03T01:21:03.936 INFO:tasks.workunit.client.0.smithi032.stdout:7/421: rename dc/d12/d1b/f68 to dc/d12/f83 0 2024-06-03T01:21:03.937 INFO:tasks.workunit.client.1.smithi044.stdout:2/887: dwrite db/d1b/d27/f50 [0,4194304] 0 2024-06-03T01:21:03.937 INFO:tasks.workunit.client.1.smithi044.stdout:2/888: stat db/d15/d49/d6c/d72/dcd/cdd 0 2024-06-03T01:21:03.941 INFO:tasks.workunit.client.1.smithi044.stdout:2/889: truncate db/d1b/d5d/f62 2802991 0 2024-06-03T01:21:03.942 INFO:tasks.workunit.client.1.smithi044.stdout:2/890: unlink db/dd/l38 0 2024-06-03T01:21:03.945 INFO:tasks.workunit.client.1.smithi044.stdout:2/891: creat db/d15/d49/d6c/d72/d7a/d4e/d7e/f11c x:0 0 0 2024-06-03T01:21:03.946 INFO:tasks.workunit.client.1.smithi044.stdout:2/892: link db/d15/l40 db/d1b/d5d/da7/dc9/l11d 0 2024-06-03T01:21:03.962 INFO:tasks.workunit.client.1.smithi044.stdout:3/893: sync 2024-06-03T01:21:03.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/893: dwrite db/d15/d49/d6c/d72/d7a/d4e/f109 [0,4194304] 0 2024-06-03T01:21:03.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/894: truncate db/d15/d49/d6c/d72/dcd/fd5 1965596 0 2024-06-03T01:21:03.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/895: dread - db/d1b/d2a/d2f/dbe/f101 zero size 2024-06-03T01:21:03.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/422: dwrite dc/d12/d22/d42/f51 [0,4194304] 0 2024-06-03T01:21:03.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/423: chown dc/d12/c48 0 1 2024-06-03T01:21:03.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/424: fdatasync dc/d12/d24/f6c 0 2024-06-03T01:21:03.965 INFO:tasks.workunit.client.1.smithi044.stdout:3/894: link df/d13/d6d/d9a/db7/f10c df/d1e/d3a/d84/f131 0 2024-06-03T01:21:03.966 INFO:tasks.workunit.client.1.smithi044.stdout:2/896: mknod db/d15/d49/d82/dc8/c11e 0 2024-06-03T01:21:03.966 INFO:tasks.workunit.client.1.smithi044.stdout:2/897: chown db/d1b/d27 45144 1 2024-06-03T01:21:03.967 INFO:tasks.workunit.client.1.smithi044.stdout:2/898: fdatasync db/dd/fe 0 2024-06-03T01:21:03.967 INFO:tasks.workunit.client.1.smithi044.stdout:3/895: mknod df/d13/de5/c132 0 2024-06-03T01:21:03.967 INFO:tasks.workunit.client.1.smithi044.stdout:3/896: truncate df/d1e/f79 1056716 0 2024-06-03T01:21:03.970 INFO:tasks.workunit.client.1.smithi044.stdout:3/897: dread df/d13/d1b/f5e [0,4194304] 0 2024-06-03T01:21:03.973 INFO:tasks.workunit.client.0.smithi032.stdout:7/425: write dc/d12/d24/d33/f4c [969524,113807] 0 2024-06-03T01:21:03.974 INFO:tasks.workunit.client.1.smithi044.stdout:2/899: dread db/d1b/f22 [0,4194304] 0 2024-06-03T01:21:03.975 INFO:tasks.workunit.client.0.smithi032.stdout:7/426: mkdir dc/d12/d24/d7b/d84 0 2024-06-03T01:21:03.977 INFO:tasks.workunit.client.0.smithi032.stdout:7/427: truncate dc/d12/d1b/f21 1413424 0 2024-06-03T01:21:03.977 INFO:tasks.workunit.client.1.smithi044.stdout:2/900: symlink db/d15/d49/d6c/d72/d8f/de5/l11f 0 2024-06-03T01:21:03.979 INFO:tasks.workunit.client.1.smithi044.stdout:3/898: dread df/d13/d6d/f10a [0,4194304] 0 2024-06-03T01:21:03.980 INFO:tasks.workunit.client.1.smithi044.stdout:3/899: dread - df/d13/d1b/d93/fd4 zero size 2024-06-03T01:21:03.980 INFO:tasks.workunit.client.0.smithi032.stdout:7/428: unlink dc/d12/d1b/f1d 0 2024-06-03T01:21:03.982 INFO:tasks.workunit.client.1.smithi044.stdout:2/901: getdents db/d15/d49/d82/dc8 0 2024-06-03T01:21:03.985 INFO:tasks.workunit.client.1.smithi044.stdout:9/955: sync 2024-06-03T01:21:03.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/985: sync 2024-06-03T01:21:03.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/986: dread - d2/d11/d12/d2b/d4a/d57/f113 zero size 2024-06-03T01:21:03.986 INFO:tasks.workunit.client.1.smithi044.stdout:5/987: creat d2/d11/d1e/d3b/da6/d116/f147 x:0 0 0 2024-06-03T01:21:03.987 INFO:tasks.workunit.client.1.smithi044.stdout:9/956: creat d6/dd/d51/d46/f137 x:0 0 0 2024-06-03T01:21:03.987 INFO:tasks.workunit.client.1.smithi044.stdout:9/957: write d6/dd/d51/d16/d5f/d8b/ffc [695787,38753] 0 2024-06-03T01:21:03.987 INFO:tasks.workunit.client.1.smithi044.stdout:9/958: stat d6/dd/d51/d46/f4e 0 2024-06-03T01:21:03.990 INFO:tasks.workunit.client.1.smithi044.stdout:9/959: mkdir d6/dd/d51/d16/d27/d138 0 2024-06-03T01:21:03.990 INFO:tasks.workunit.client.1.smithi044.stdout:9/960: chown d6/f11 904 1 2024-06-03T01:21:03.990 INFO:tasks.workunit.client.1.smithi044.stdout:9/961: write d6/dd/d51/d16/f7b [4943861,77617] 0 2024-06-03T01:21:03.992 INFO:tasks.workunit.client.1.smithi044.stdout:5/988: dread d2/d11/d12/f37 [0,4194304] 0 2024-06-03T01:21:03.992 INFO:tasks.workunit.client.1.smithi044.stdout:9/962: truncate d6/dd/d51/f86 3898154 0 2024-06-03T01:21:03.995 INFO:tasks.workunit.client.1.smithi044.stdout:5/989: link d2/d11/d1e/d21/d24/d93/d9d/d28/f74 d2/d11/d1e/d21/d24/d9e/dbb/f148 0 2024-06-03T01:21:03.995 INFO:tasks.workunit.client.1.smithi044.stdout:5/990: creat d2/d11/d12/f149 x:0 0 0 2024-06-03T01:21:03.996 INFO:tasks.workunit.client.1.smithi044.stdout:9/963: mknod d6/dd/d51/d16/c139 0 2024-06-03T01:21:03.999 INFO:tasks.workunit.client.1.smithi044.stdout:5/991: mkdir d2/d11c/df5/d14a 0 2024-06-03T01:21:04.000 INFO:tasks.workunit.client.1.smithi044.stdout:9/964: mkdir d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/dae/d13a 0 2024-06-03T01:21:04.000 INFO:tasks.workunit.client.1.smithi044.stdout:9/965: chown d6/dd/f17 28194203 1 2024-06-03T01:21:04.000 INFO:tasks.workunit.client.1.smithi044.stdout:9/966: fdatasync d6/dd/d51/d16/d27/f64 0 2024-06-03T01:21:04.001 INFO:tasks.workunit.client.1.smithi044.stdout:5/992: link d2/d11/d12/f13c d2/d11/d12/f14b 0 2024-06-03T01:21:04.003 INFO:tasks.workunit.client.1.smithi044.stdout:9/967: mknod d6/dd/d51/db/dd5/c13b 0 2024-06-03T01:21:04.004 INFO:tasks.workunit.client.1.smithi044.stdout:9/968: creat d6/dd/d51/d16/d5f/f13c x:0 0 0 2024-06-03T01:21:04.004 INFO:tasks.workunit.client.1.smithi044.stdout:9/969: symlink d6/dd/d51/d16/d5f/d8b/d8d/dac/l13d 0 2024-06-03T01:21:04.006 INFO:tasks.workunit.client.1.smithi044.stdout:3/900: dwrite df/d13/d1b/d56/d5c/d98/df4/f91 [4194304,4194304] 0 2024-06-03T01:21:04.006 INFO:tasks.workunit.client.1.smithi044.stdout:9/970: unlink d6/dd/d51/d16/d27/d5e/da0/d12c/l85 0 2024-06-03T01:21:04.008 INFO:tasks.workunit.client.1.smithi044.stdout:3/901: unlink df/c12 0 2024-06-03T01:21:04.008 INFO:tasks.workunit.client.1.smithi044.stdout:2/902: dwrite db/d1b/d2a/d52/d54/db6/fc6 [0,4194304] 0 2024-06-03T01:21:04.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/971: mknod d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dcd/d124/c13e 0 2024-06-03T01:21:04.011 INFO:tasks.workunit.client.1.smithi044.stdout:3/902: rename df/d13/d6d/d9a/dad/fd6 to df/d13/d1b/d4b/f133 0 2024-06-03T01:21:04.013 INFO:tasks.workunit.client.1.smithi044.stdout:2/903: rename db/d1b/d2a/d67/de4/d111 to db/d1b/d5d/da7/d120 0 2024-06-03T01:21:04.013 INFO:tasks.workunit.client.1.smithi044.stdout:3/903: creat df/d13/de5/f134 x:0 0 0 2024-06-03T01:21:04.014 INFO:tasks.workunit.client.1.smithi044.stdout:3/904: fsync df/d13/d1b/d4b/fea 0 2024-06-03T01:21:04.014 INFO:tasks.workunit.client.1.smithi044.stdout:5/993: dwrite d2/d11/f100 [0,4194304] 0 2024-06-03T01:21:04.016 INFO:tasks.workunit.client.1.smithi044.stdout:9/972: rename d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/d68 to d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/dae/d13a/d13f 0 2024-06-03T01:21:04.021 INFO:tasks.workunit.client.1.smithi044.stdout:5/994: truncate d2/d11/d1e/f51 5303986 0 2024-06-03T01:21:04.022 INFO:tasks.workunit.client.1.smithi044.stdout:9/973: mknod d6/dd/c140 0 2024-06-03T01:21:04.031 INFO:tasks.workunit.client.1.smithi044.stdout:9/974: write d6/dd/d51/d46/fb0 [2810879,124946] 0 2024-06-03T01:21:04.035 INFO:tasks.workunit.client.1.smithi044.stdout:2/904: rmdir db/d1b 39 2024-06-03T01:21:04.038 INFO:tasks.workunit.client.1.smithi044.stdout:5/995: rename d2/d11/d1e/d21/d24/f69 to d2/d11/d1e/d21/d24/d93/d9d/d28/d66/da3/f14c 0 2024-06-03T01:21:04.038 INFO:tasks.workunit.client.1.smithi044.stdout:5/996: readlink d2/d11/d12/d2b/d4a/l53 0 2024-06-03T01:21:04.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/975: unlink d6/dd/d51/d16/f3f 0 2024-06-03T01:21:04.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/976: fdatasync d6/dd/d51/d16/d116/f127 0 2024-06-03T01:21:04.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/977: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [4304717,69393] 0 2024-06-03T01:21:04.040 INFO:tasks.workunit.client.1.smithi044.stdout:9/978: write d6/dd/f11d [1120228,21833] 0 2024-06-03T01:21:04.041 INFO:tasks.workunit.client.1.smithi044.stdout:2/905: mknod db/d1b/d27/c121 0 2024-06-03T01:21:04.041 INFO:tasks.workunit.client.1.smithi044.stdout:2/906: chown db/d15/d49/d82 269930710 1 2024-06-03T01:21:04.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/997: creat d2/d11/d1e/d21/d24/d93/d9d/d28/d4c/d124/f14d x:0 0 0 2024-06-03T01:21:04.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/998: write d2/d11/d3e/ffb [616800,26537] 0 2024-06-03T01:21:04.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/999: truncate d2/d11/d12/f104 5210883 0 2024-06-03T01:21:04.043 INFO:tasks.workunit.client.1.smithi044.stdout:2/907: getdents db/d15/d49/d7b/df8 0 2024-06-03T01:21:04.043 INFO:tasks.workunit.client.1.smithi044.stdout:2/908: write db/d1b/d3f/f59 [383206,29029] 0 2024-06-03T01:21:04.056 INFO:tasks.workunit.client.1.smithi044.stdout:9/979: dwrite d6/dd/d51/d16/d27/d5e/d71/de0/f10a [4194304,4194304] 0 2024-06-03T01:21:04.061 INFO:tasks.workunit.client.1.smithi044.stdout:9/980: dread d6/dd/d51/d16/d27/d5e/da0/d12c/d11a/f6c [0,4194304] 0 2024-06-03T01:21:04.063 INFO:tasks.workunit.client.1.smithi044.stdout:2/909: dwrite db/d15/d49/d6c/fa9 [0,4194304] 0 2024-06-03T01:21:04.065 INFO:tasks.workunit.client.1.smithi044.stdout:2/910: mknod db/d1b/d5d/c122 0 2024-06-03T01:21:04.067 INFO:tasks.workunit.client.1.smithi044.stdout:2/911: link db/d15/d49/d6c/d72/dcd/cdd db/d1b/d2a/d2f/dbe/c123 0 2024-06-03T01:21:04.067 INFO:tasks.workunit.client.1.smithi044.stdout:2/912: read db/d1b/d3f/f66 [1159763,118810] 0 2024-06-03T01:21:04.067 INFO:tasks.workunit.client.1.smithi044.stdout:2/913: fsync db/d1b/f36 0 2024-06-03T01:21:04.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/429: rename dc/d12/d2a/f40 to dc/d12/d22/f85 0 2024-06-03T01:21:04.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/430: chown dc/d12/d78 0 1 2024-06-03T01:21:04.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/431: read - dc/d12/f77 zero size 2024-06-03T01:21:04.084 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:03 smithi032 ceph-mon[29360]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:04.087 INFO:tasks.workunit.client.1.smithi044.stdout:9/981: dwrite d6/dd/d51/d16/d27/d5e/da0/d12c/d11a/f7e [0,4194304] 0 2024-06-03T01:21:04.087 INFO:tasks.workunit.client.1.smithi044.stdout:9/982: read d6/dd/d51/d16/d27/d5e/fde [360129,58845] 0 2024-06-03T01:21:04.088 INFO:tasks.workunit.client.1.smithi044.stdout:9/983: symlink d6/dd/d51/d16/l141 0 2024-06-03T01:21:04.088 INFO:tasks.workunit.client.1.smithi044.stdout:9/984: fsync d6/dd/fa3 0 2024-06-03T01:21:04.088 INFO:tasks.workunit.client.1.smithi044.stdout:9/985: fsync d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dcd/ff7 0 2024-06-03T01:21:04.088 INFO:tasks.workunit.client.1.smithi044.stdout:9/986: truncate d6/dd/f108 1723658 0 2024-06-03T01:21:04.093 INFO:tasks.workunit.client.0.smithi032.stdout:7/432: dwrite dc/f30 [8388608,4194304] 0 2024-06-03T01:21:04.099 INFO:tasks.workunit.client.0.smithi032.stdout:7/433: mkdir dc/d12/d6a/d86 0 2024-06-03T01:21:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.101+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5024148d30 con 0x7f50240a5b60 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.102+0000 7f501b7fe700 1 -- 172.21.15.32:0/2843681611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f502c0832c0 con 0x7f50240a5b60 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:21:04.103 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:21:04.104 INFO:tasks.workunit.client.1.smithi044.stdout:9/987: dwrite d6/dd/d51/f13 [4194304,4194304] 0 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 msgr2=0x7f501403b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 0x7f501403b8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5020006fd0 tx=0x7f5020009190 comp rx=0 tx=0).stop 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 msgr2=0x7f5024142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f502c050430 tx=0x7f502c066e30 comp rx=0 tx=0).stop 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 shutdown_connections 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f5014039400 0x7f501403b8c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50240a4a20 0x7f5024142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 --2- 172.21.15.32:0/2843681611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50240a5b60 0x7f5024142fb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.104+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 >> 172.21.15.32:0/2843681611 conn(0x7f502409fee0 msgr2=0x7f50240a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:04.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.105+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 shutdown_connections 2024-06-03T01:21:04.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.105+0000 7f50309ff700 1 -- 172.21.15.32:0/2843681611 wait complete. 2024-06-03T01:21:04.107 INFO:tasks.workunit.client.1.smithi044.stdout:9/988: mknod d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dcd/c142 0 2024-06-03T01:21:04.107 INFO:tasks.workunit.client.1.smithi044.stdout:9/989: stat d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/dae/fe4 0 2024-06-03T01:21:04.108 INFO:tasks.workunit.client.1.smithi044.stdout:9/990: stat d6/dd/d45/c60 0 2024-06-03T01:21:04.109 INFO:tasks.workunit.client.0.smithi032.stdout:9/488: sync 2024-06-03T01:21:04.110 INFO:tasks.workunit.client.1.smithi044.stdout:9/991: link d6/l29 d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/d55/l143 0 2024-06-03T01:21:04.111 INFO:tasks.workunit.client.0.smithi032.stdout:9/489: mkdir d0/da4 0 2024-06-03T01:21:04.111 INFO:tasks.workunit.client.0.smithi032.stdout:9/490: chown d0/d23/d25/l6e 3793 1 2024-06-03T01:21:04.111 INFO:tasks.workunit.client.0.smithi032.stdout:9/491: truncate d0/d23/d25/f8e 4428630 0 2024-06-03T01:21:04.112 INFO:tasks.workunit.client.1.smithi044.stdout:9/992: mkdir d6/dd/d51/d16/d27/d5e/db7/de1/d144 0 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/492: unlink d0/d11/d1d/d4a/c4b 0 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/493: getdents d0/da4 0 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/494: fdatasync d0/d11/d1d/d4a/d71/f96 0 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/495: chown d0/d11/d9c/fa3 1 1 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/496: chown d0/c12 125909127 1 2024-06-03T01:21:04.113 INFO:tasks.workunit.client.1.smithi044.stdout:9/993: mkdir d6/dd/d51/d16/d27/d5e/da0/d145 0 2024-06-03T01:21:04.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/497: rename d0/d11/d1d/d4a/f7f to d0/d11/fa5 0 2024-06-03T01:21:04.116 INFO:tasks.workunit.client.1.smithi044.stdout:9/994: mkdir d6/dd/d51/d16/d27/d105/d146 0 2024-06-03T01:21:04.116 INFO:tasks.workunit.client.1.smithi044.stdout:9/995: write d6/dd/d51/fe [939346,16779] 0 2024-06-03T01:21:04.117 INFO:tasks.workunit.client.0.smithi032.stdout:7/434: dwrite dc/f30 [0,4194304] 0 2024-06-03T01:21:04.117 INFO:tasks.workunit.client.1.smithi044.stdout:3/905: sync 2024-06-03T01:21:04.117 INFO:tasks.workunit.client.1.smithi044.stdout:3/906: fsync df/d1e/d78/da4/fbf 0 2024-06-03T01:21:04.118 INFO:tasks.workunit.client.1.smithi044.stdout:3/907: chown df/d13/d39/d3d/d50/lcf 466 1 2024-06-03T01:21:04.119 INFO:tasks.workunit.client.1.smithi044.stdout:3/908: creat df/d13/d1b/d56/d5c/de3/f135 x:0 0 0 2024-06-03T01:21:04.119 INFO:tasks.workunit.client.0.smithi032.stdout:7/435: rename dc/d12/d22/d42/c71 to dc/d12/d22/c87 0 2024-06-03T01:21:04.120 INFO:tasks.workunit.client.0.smithi032.stdout:7/436: mkdir dc/d12/d6a/d86/d88 0 2024-06-03T01:21:04.121 INFO:tasks.workunit.client.0.smithi032.stdout:7/437: mkdir dc/d12/d6a/d86/d89 0 2024-06-03T01:21:04.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/438: mknod dc/d12/d2a/d2f/c8a 0 2024-06-03T01:21:04.123 INFO:tasks.workunit.client.1.smithi044.stdout:2/914: sync 2024-06-03T01:21:04.124 INFO:tasks.workunit.client.0.smithi032.stdout:9/498: dread d0/f6 [0,4194304] 0 2024-06-03T01:21:04.125 INFO:tasks.workunit.client.0.smithi032.stdout:7/439: dread dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:3/909: rmdir df/d1e/d78 39 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:3/910: truncate df/d1e/d3a/d84/f10d 277214 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:3/911: creat df/d13/d6d/d9a/db7/f136 x:0 0 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:3/912: chown df/d13/d1b/d56/d5c 8 1 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:9/996: dread d6/dd/d51/d16/d27/d5e/da0/d12c/d11a/f7e [0,4194304] 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:2/915: write db/d1b/d3f/f59 [272568,47399] 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:9/997: write d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/fbf [4171202,118342] 0 2024-06-03T01:21:04.127 INFO:tasks.workunit.client.1.smithi044.stdout:2/916: write db/d15/d49/d6c/d72/dce/f116 [1214836,106403] 0 2024-06-03T01:21:04.129 INFO:tasks.workunit.client.1.smithi044.stdout:2/917: truncate db/fa2 757518 0 2024-06-03T01:21:04.131 INFO:tasks.workunit.client.1.smithi044.stdout:2/918: mkdir db/d1b/d5d/da7/d124 0 2024-06-03T01:21:04.131 INFO:tasks.workunit.client.1.smithi044.stdout:2/919: fdatasync db/d15/d49/d6c/df3/f10f 0 2024-06-03T01:21:04.132 INFO:tasks.workunit.client.1.smithi044.stdout:9/998: dread d6/dd/f36 [0,4194304] 0 2024-06-03T01:21:04.133 INFO:tasks.workunit.client.1.smithi044.stdout:9/999: getdents d6/dd/d51/d16/d5f/d8b/d8d/dac/de9/dad/dfe 0 2024-06-03T01:21:04.135 INFO:tasks.workunit.client.0.smithi032.stdout:7/440: dread dc/d12/d2a/d2f/d63/d69/f5e [0,4194304] 0 2024-06-03T01:21:04.136 INFO:tasks.workunit.client.1.smithi044.stdout:2/920: creat db/d1b/d5d/da7/dc9/f125 x:0 0 0 2024-06-03T01:21:04.136 INFO:tasks.workunit.client.1.smithi044.stdout:2/921: stat db/d15/d49/d6c/d72/dce/ffb 0 2024-06-03T01:21:04.136 INFO:tasks.workunit.client.1.smithi044.stdout:2/922: creat db/d15/d49/d6c/d72/d7a/d4e/f126 x:0 0 0 2024-06-03T01:21:04.136 INFO:tasks.workunit.client.0.smithi032.stdout:9/499: rename d0/d11/d1d/d4a to d0/d11/d1d/da6 0 2024-06-03T01:21:04.137 INFO:tasks.workunit.client.0.smithi032.stdout:7/441: unlink dc/d12/d24/l41 0 2024-06-03T01:21:04.138 INFO:tasks.workunit.client.1.smithi044.stdout:3/913: write df/d1e/f79 [735659,111475] 0 2024-06-03T01:21:04.138 INFO:tasks.workunit.client.0.smithi032.stdout:9/500: stat d0/d11/d79/l98 0 2024-06-03T01:21:04.139 INFO:tasks.workunit.client.1.smithi044.stdout:3/914: creat df/d13/d1b/d56/d5c/de3/de6/f137 x:0 0 0 2024-06-03T01:21:04.140 INFO:tasks.workunit.client.0.smithi032.stdout:7/442: mkdir dc/d12/d6a/d86/d88/d8b 0 2024-06-03T01:21:04.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/915: dwrite df/f10b [0,4194304] 0 2024-06-03T01:21:04.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/916: dread - df/d1e/d3a/d84/f10e zero size 2024-06-03T01:21:04.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/917: truncate df/d13/d1b/d56/d5c/de3/f135 785258 0 2024-06-03T01:21:04.167 INFO:tasks.workunit.client.1.smithi044.stdout:3/918: write df/d1e/d3a/d84/f131 [509435,19137] 0 2024-06-03T01:21:04.168 INFO:tasks.workunit.client.1.smithi044.stdout:3/919: symlink df/d13/d1b/d56/l138 0 2024-06-03T01:21:04.168 INFO:tasks.workunit.client.1.smithi044.stdout:3/920: stat df/d13/d1b/d4b/d69/f76 0 2024-06-03T01:21:04.169 INFO:tasks.workunit.client.1.smithi044.stdout:2/923: dwrite db/d15/f70 [0,4194304] 0 2024-06-03T01:21:04.170 INFO:tasks.workunit.client.0.smithi032.stdout:9/501: dwrite d0/d23/d25/f62 [0,4194304] 0 2024-06-03T01:21:04.170 INFO:tasks.workunit.client.0.smithi032.stdout:9/502: write d0/d11/f48 [2461208,43402] 0 2024-06-03T01:21:04.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/443: dwrite dc/d12/d1b/f5b [0,4194304] 0 2024-06-03T01:21:04.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/444: stat dc/d12/d24/d7b/f74 0 2024-06-03T01:21:04.171 INFO:tasks.workunit.client.1.smithi044.stdout:3/921: link df/d13/d1b/d4b/c82 df/d13/d39/d3d/d50/c139 0 2024-06-03T01:21:04.173 INFO:tasks.workunit.client.1.smithi044.stdout:2/924: creat db/d1b/d2a/d67/d53/f127 x:0 0 0 2024-06-03T01:21:04.174 INFO:tasks.workunit.client.0.smithi032.stdout:9/503: creat d0/d11/d76/fa7 x:0 0 0 2024-06-03T01:21:04.174 INFO:tasks.workunit.client.0.smithi032.stdout:9/504: write d0/f57 [1844840,13612] 0 2024-06-03T01:21:04.175 INFO:tasks.workunit.client.0.smithi032.stdout:9/505: symlink d0/d11/d79/la8 0 2024-06-03T01:21:04.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/445: dread dc/d12/d2a/d2f/d63/d69/f5e [0,4194304] 0 2024-06-03T01:21:04.176 INFO:tasks.workunit.client.1.smithi044.stdout:3/922: creat df/d13/d39/d3d/d6a/d9c/f13a x:0 0 0 2024-06-03T01:21:04.177 INFO:tasks.workunit.client.1.smithi044.stdout:2/925: link db/d15/d49/d82/dc8/c11e db/dd/c128 0 2024-06-03T01:21:04.178 INFO:tasks.workunit.client.1.smithi044.stdout:2/926: chown db/d15/d49/d82/dc8/cf7 1 1 2024-06-03T01:21:04.182 INFO:tasks.workunit.client.1.smithi044.stdout:3/923: symlink df/d1e/d78/dda/d12c/dbc/l13b 0 2024-06-03T01:21:04.182 INFO:tasks.workunit.client.1.smithi044.stdout:2/927: symlink db/d1b/d2a/d67/d53/l129 0 2024-06-03T01:21:04.185 INFO:tasks.workunit.client.1.smithi044.stdout:3/924: rename df/cb0 to df/d1e/d3a/d84/d90/c13c 0 2024-06-03T01:21:04.195 INFO:tasks.workunit.client.1.smithi044.stdout:3/925: write df/f19 [259036,114690] 0 2024-06-03T01:21:04.207 INFO:tasks.workunit.client.1.smithi044.stdout:2/928: dwrite db/d1b/d2a/d67/f7c [0,4194304] 0 2024-06-03T01:21:04.208 INFO:tasks.workunit.client.0.smithi032.stdout:9/506: dwrite d0/d11/d1d/d38/f6d [0,4194304] 0 2024-06-03T01:21:04.208 INFO:tasks.workunit.client.0.smithi032.stdout:7/446: dwrite dc/d12/d24/d7b/f20 [0,4194304] 0 2024-06-03T01:21:04.208 INFO:tasks.workunit.client.0.smithi032.stdout:7/447: stat dc/d12/d24/d7b/f74 0 2024-06-03T01:21:04.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/929: getdents db/d15/d49/d6c/d72/d8f 0 2024-06-03T01:21:04.210 INFO:tasks.workunit.client.1.smithi044.stdout:2/930: readlink db/d1b/d2a/d67/ld2 0 2024-06-03T01:21:04.211 INFO:tasks.workunit.client.0.smithi032.stdout:7/448: write dc/d12/d1b/f60 [107403,101315] 0 2024-06-03T01:21:04.211 INFO:tasks.workunit.client.0.smithi032.stdout:7/449: chown dc/d12/f83 6609322 1 2024-06-03T01:21:04.218 INFO:tasks.workunit.client.1.smithi044.stdout:3/926: dwrite df/d13/d1b/d56/d5c/d98/df4/f91 [4194304,4194304] 0 2024-06-03T01:21:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:7/450: dread dc/d12/d1b/f60 [0,4194304] 0 2024-06-03T01:21:04.220 INFO:tasks.workunit.client.1.smithi044.stdout:3/927: mkdir df/d1e/d78/da4/d13d 0 2024-06-03T01:21:04.220 INFO:tasks.workunit.client.0.smithi032.stdout:7/451: creat dc/d12/d6a/d86/d88/f8c x:0 0 0 2024-06-03T01:21:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:03 smithi044 ceph-mon[36956]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:04.222 INFO:tasks.workunit.client.1.smithi044.stdout:3/928: symlink df/d13/d1b/d56/d5c/d98/d12b/l13e 0 2024-06-03T01:21:04.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/507: dread d0/d11/f63 [0,4194304] 0 2024-06-03T01:21:04.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/508: readlink d0/d23/d24/d26/l95 0 2024-06-03T01:21:04.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/509: dread - d0/d11/d1d/d38/f97 zero size 2024-06-03T01:21:04.225 INFO:tasks.workunit.client.1.smithi044.stdout:2/931: dwrite db/d1b/d2a/f43 [0,4194304] 0 2024-06-03T01:21:04.225 INFO:tasks.workunit.client.1.smithi044.stdout:2/932: chown db/d15/d49/d6c/d72/dcd 23070931 1 2024-06-03T01:21:04.227 INFO:tasks.workunit.client.0.smithi032.stdout:9/510: mknod d0/d11/d1d/da6/d71/ca9 0 2024-06-03T01:21:04.229 INFO:tasks.workunit.client.0.smithi032.stdout:9/511: creat d0/d23/d24/d26/d41/faa x:0 0 0 2024-06-03T01:21:04.230 INFO:tasks.workunit.client.1.smithi044.stdout:2/933: link db/d1b/d2a/d2f/dbe/f101 db/d1b/d2a/d52/d54/f12a 0 2024-06-03T01:21:04.231 INFO:tasks.workunit.client.0.smithi032.stdout:9/512: symlink d0/da4/lab 0 2024-06-03T01:21:04.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/934: rename db/d15/l20 to db/d1b/d2a/d67/l12b 0 2024-06-03T01:21:04.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/935: fsync db/d15/d49/d6c/d72/d7a/d4e/f126 0 2024-06-03T01:21:04.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/936: truncate db/d1b/d2a/d2f/dbe/ded/dfa/f105 746462 0 2024-06-03T01:21:04.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/937: read db/f26 [3675351,46552] 0 2024-06-03T01:21:04.233 INFO:tasks.workunit.client.1.smithi044.stdout:2/938: creat db/d15/d49/d7b/df8/d11b/f12c x:0 0 0 2024-06-03T01:21:04.233 INFO:tasks.workunit.client.1.smithi044.stdout:3/929: dread df/f58 [0,4194304] 0 2024-06-03T01:21:04.234 INFO:tasks.workunit.client.0.smithi032.stdout:9/513: unlink d0/d11/d1d/d38/f6d 0 2024-06-03T01:21:04.234 INFO:tasks.workunit.client.1.smithi044.stdout:2/939: creat db/d15/d49/d82/dc8/f12d x:0 0 0 2024-06-03T01:21:04.236 INFO:tasks.workunit.client.1.smithi044.stdout:3/930: link df/d13/d39/d3d/d50/l11e df/d13/d1b/d56/d5c/d98/df4/l13f 0 2024-06-03T01:21:04.238 INFO:tasks.workunit.client.1.smithi044.stdout:2/940: rename l4 to db/d15/d49/d6c/d72/d7a/l12e 0 2024-06-03T01:21:04.241 INFO:tasks.workunit.client.0.smithi032.stdout:7/452: dwrite dc/d12/d1b/f34 [4194304,4194304] 0 2024-06-03T01:21:04.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/453: mknod dc/d12/d2a/d2f/d76/c8d 0 2024-06-03T01:21:04.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/454: chown l0 183426782 1 2024-06-03T01:21:04.247 INFO:tasks.workunit.client.1.smithi044.stdout:3/931: read df/d13/f29 [96809,55941] 0 2024-06-03T01:21:04.247 INFO:tasks.workunit.client.0.smithi032.stdout:7/455: mkdir dc/d12/d78/d8e 0 2024-06-03T01:21:04.247 INFO:tasks.workunit.client.0.smithi032.stdout:7/456: write dc/d12/d22/d42/f50 [1017163,118923] 0 2024-06-03T01:21:04.248 INFO:tasks.workunit.client.0.smithi032.stdout:7/457: write dc/d12/d2a/d2f/d76/f80 [369521,125625] 0 2024-06-03T01:21:04.249 INFO:tasks.workunit.client.0.smithi032.stdout:7/458: mknod dc/d12/d2a/d2f/c8f 0 2024-06-03T01:21:04.251 INFO:tasks.workunit.client.0.smithi032.stdout:7/459: symlink dc/d12/d78/d8e/l90 0 2024-06-03T01:21:04.251 INFO:tasks.workunit.client.0.smithi032.stdout:7/460: truncate dc/d12/d22/f52 255052 0 2024-06-03T01:21:04.258 INFO:tasks.workunit.client.0.smithi032.stdout:7/461: dread dc/d12/f83 [0,4194304] 0 2024-06-03T01:21:04.259 INFO:tasks.workunit.client.0.smithi032.stdout:9/514: dwrite d0/d11/d1d/f99 [0,4194304] 0 2024-06-03T01:21:04.272 INFO:tasks.workunit.client.1.smithi044.stdout:3/932: dwrite df/d13/d39/d3d/d43/d71/f83 [4194304,4194304] 0 2024-06-03T01:21:04.272 INFO:tasks.workunit.client.1.smithi044.stdout:2/941: dwrite db/d1b/d2a/d2f/dbe/ded/dfa/f105 [0,4194304] 0 2024-06-03T01:21:04.276 INFO:tasks.workunit.client.1.smithi044.stdout:2/942: symlink db/d1b/d2a/d67/d53/d9b/de6/l12f 0 2024-06-03T01:21:04.277 INFO:tasks.workunit.client.1.smithi044.stdout:2/943: truncate db/d1b/d27/f32 2450782 0 2024-06-03T01:21:04.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/944: link db/d15/d49/d6c/d72/d7a/d4e/laf db/d1b/d2a/d52/d54/db6/l130 0 2024-06-03T01:21:04.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/945: chown db/d1b/d2a/d52/d54/c6a 434208 1 2024-06-03T01:21:04.282 INFO:tasks.workunit.client.1.smithi044.stdout:2/946: rename db/dd/lc2 to db/d15/l131 0 2024-06-03T01:21:04.284 INFO:tasks.workunit.client.1.smithi044.stdout:2/947: rename db/d1b/d5d/cfe to db/d1b/d2a/d67/d53/d9b/c132 0 2024-06-03T01:21:04.285 INFO:tasks.workunit.client.1.smithi044.stdout:2/948: symlink db/d1b/d2a/d52/d54/db6/l133 0 2024-06-03T01:21:04.285 INFO:tasks.workunit.client.1.smithi044.stdout:2/949: fsync db/d15/d49/d6c/d72/dce/f11a 0 2024-06-03T01:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 -- 172.21.15.32:0/2677567963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc0ffba0 msgr2=0x7f75cc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 --2- 172.21.15.32:0/2677567963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc0ffba0 0x7f75cc0fffc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009910 tx=0x7f75bc009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 -- 172.21.15.32:0/2677567963 shutdown_connections 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 --2- 172.21.15.32:0/2677567963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc100da0 0x7f75cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 --2- 172.21.15.32:0/2677567963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc0ffba0 0x7f75cc0fffc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 -- 172.21.15.32:0/2677567963 >> 172.21.15.32:0/2677567963 conn(0x7f75cc0fb120 msgr2=0x7f75cc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 -- 172.21.15.32:0/2677567963 shutdown_connections 2024-06-03T01:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.285+0000 7f75d4039700 1 -- 172.21.15.32:0/2677567963 wait complete. 2024-06-03T01:21:04.287 INFO:tasks.workunit.client.1.smithi044.stdout:2/950: getdents db/d1b/d2a 0 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.286+0000 7f75d4039700 1 Processor -- start 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.286+0000 7f75d4039700 1 -- start start 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d4039700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d4039700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc100da0 0x7f75cc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d4039700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1970a0 con 0x7f75cc100da0 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d4039700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc19db70 con 0x7f75cc0ffba0 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d15d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc100da0 0x7f75cc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:59308/0 (socket says 172.21.15.32:59308) 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 -- 172.21.15.32:0/2822759766 learned_addr learned my addr 172.21.15.32:0/2822759766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 -- 172.21.15.32:0/2822759766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc100da0 msgr2=0x7f75cc19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc100da0 0x7f75cc19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.287+0000 7f75d1dd5700 1 -- 172.21.15.32:0/2822759766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0095f0 con 0x7f75cc0ffba0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75d1dd5700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc01d070 con 0x7f75cc0ffba0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f75bc005dc0 con 0x7f75cc0ffba0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc022470 con 0x7f75cc0ffba0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc19ddc0 con 0x7f75cc0ffba0 2024-06-03T01:21:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.288+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc19e2b0 con 0x7f75cc0ffba0 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.289+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc108700 con 0x7f75cc0ffba0 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.289+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 46426+0+0 (secure 0 0 0) 0x7f75bc00fa70 con 0x7f75cc0ffba0 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.290+0000 7f75c2ffd700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 0x7f75b803b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.290+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f75bc04eaa0 con 0x7f75cc0ffba0 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.290+0000 7f75d15d4700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 0x7f75b803b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.290+0000 7f75d15d4700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 0x7f75b803b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75cc069ef0 tx=0x7f75c8009260 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:04.296 INFO:tasks.workunit.client.1.smithi044.stdout:2/951: dread db/f19 [0,4194304] 0 2024-06-03T01:21:04.296 INFO:tasks.workunit.client.1.smithi044.stdout:2/952: chown db/dd/f11 5331365 1 2024-06-03T01:21:04.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.297+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f75bc016c40 con 0x7f75cc0ffba0 2024-06-03T01:21:04.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/462: dwrite dc/d12/d24/d33/f5c [0,4194304] 0 2024-06-03T01:21:04.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/463: chown dc/d12/d2a/d2f/d76/f7d 13682892 1 2024-06-03T01:21:04.307 INFO:tasks.workunit.client.0.smithi032.stdout:9/515: dwrite d0/d23/d25/f62 [4194304,4194304] 0 2024-06-03T01:21:04.307 INFO:tasks.workunit.client.1.smithi044.stdout:3/933: dwrite df/d13/d39/d3d/f8b [0,4194304] 0 2024-06-03T01:21:04.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/516: rename d0/d23/d25/l2b to d0/d23/d24/d26/d41/lac 0 2024-06-03T01:21:04.322 INFO:tasks.workunit.client.0.smithi032.stdout:9/517: mknod d0/d23/cad 0 2024-06-03T01:21:04.322 INFO:tasks.workunit.client.0.smithi032.stdout:9/518: write d0/d23/d25/f8e [2397523,24019] 0 2024-06-03T01:21:04.338 INFO:tasks.workunit.client.0.smithi032.stdout:7/464: dwrite dc/d12/d22/f29 [0,4194304] 0 2024-06-03T01:21:04.339 INFO:tasks.workunit.client.1.smithi044.stdout:3/934: dwrite df/d13/d39/d3d/fa5 [0,4194304] 0 2024-06-03T01:21:04.339 INFO:tasks.workunit.client.1.smithi044.stdout:3/935: write df/d13/d1b/d56/d5c/d98/ff9 [897774,113295] 0 2024-06-03T01:21:04.343 INFO:tasks.workunit.client.0.smithi032.stdout:7/465: rename dc/d12/d24/c3a to dc/d12/d6a/c91 0 2024-06-03T01:21:04.352 INFO:tasks.workunit.client.0.smithi032.stdout:9/519: rmdir d0/d23/d25 39 2024-06-03T01:21:04.352 INFO:tasks.workunit.client.0.smithi032.stdout:7/466: dread dc/d12/d22/d42/f51 [0,4194304] 0 2024-06-03T01:21:04.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/467: creat dc/d12/d24/d33/f92 x:0 0 0 2024-06-03T01:21:04.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/468: creat dc/d12/f93 x:0 0 0 2024-06-03T01:21:04.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/469: fsync dc/d12/d1b/f21 0 2024-06-03T01:21:04.355 INFO:tasks.workunit.client.0.smithi032.stdout:7/470: creat dc/d12/d6a/d86/d88/f94 x:0 0 0 2024-06-03T01:21:04.359 INFO:tasks.workunit.client.1.smithi044.stdout:3/936: dwrite df/d1e/d78/dda/d12c/fa3 [0,4194304] 0 2024-06-03T01:21:04.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/471: dread dc/f14 [0,4194304] 0 2024-06-03T01:21:04.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/472: fsync dc/d12/d2a/d2f/d63/d69/f5e 0 2024-06-03T01:21:04.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/473: fdatasync dc/f30 0 2024-06-03T01:21:04.366 INFO:tasks.workunit.client.0.smithi032.stdout:7/474: link dc/d12/d22/f52 dc/d12/d22/d42/d61/f95 0 2024-06-03T01:21:04.372 INFO:tasks.workunit.client.0.smithi032.stdout:9/520: dwrite d0/d11/d1d/da6/f91 [0,4194304] 0 2024-06-03T01:21:04.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/521: truncate d0/d23/d24/f2a 984478 0 2024-06-03T01:21:04.374 INFO:tasks.workunit.client.0.smithi032.stdout:7/475: dread dc/f2b [0,4194304] 0 2024-06-03T01:21:04.374 INFO:tasks.workunit.client.0.smithi032.stdout:7/476: readlink dc/d12/d78/d8e/l90 0 2024-06-03T01:21:04.375 INFO:tasks.workunit.client.0.smithi032.stdout:9/522: mknod d0/d11/d1d/d38/d4f/cae 0 2024-06-03T01:21:04.376 INFO:tasks.workunit.client.0.smithi032.stdout:9/523: unlink d0/d23/l75 0 2024-06-03T01:21:04.378 INFO:tasks.workunit.client.0.smithi032.stdout:9/524: truncate d0/d23/d25/f8d 3589286 0 2024-06-03T01:21:04.378 INFO:tasks.workunit.client.0.smithi032.stdout:9/525: stat d0/d23/d25/f4c 0 2024-06-03T01:21:04.380 INFO:tasks.workunit.client.0.smithi032.stdout:9/526: creat d0/da4/faf x:0 0 0 2024-06-03T01:21:04.385 INFO:tasks.workunit.client.0.smithi032.stdout:9/527: rename d0/d23/d24/d26 to d0/d11/d1d/d38/d4f/db0 0 2024-06-03T01:21:04.387 INFO:tasks.workunit.client.0.smithi032.stdout:9/528: mknod d0/d11/d1d/d38/d4f/db0/cb1 0 2024-06-03T01:21:04.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/477: dwrite dc/d12/d22/f52 [0,4194304] 0 2024-06-03T01:21:04.393 INFO:tasks.workunit.client.0.smithi032.stdout:7/478: symlink dc/d12/d2a/d81/l96 0 2024-06-03T01:21:04.393 INFO:tasks.workunit.client.0.smithi032.stdout:7/479: write dc/d12/f93 [809327,92733] 0 2024-06-03T01:21:04.394 INFO:tasks.workunit.client.0.smithi032.stdout:7/480: creat dc/d12/d22/d42/d61/d82/f97 x:0 0 0 2024-06-03T01:21:04.395 INFO:tasks.workunit.client.0.smithi032.stdout:7/481: creat dc/d12/d6a/d86/d88/f98 x:0 0 0 2024-06-03T01:21:04.396 INFO:tasks.workunit.client.0.smithi032.stdout:9/529: read d0/d11/d1d/d38/f61 [920757,5192] 0 2024-06-03T01:21:04.396 INFO:tasks.workunit.client.0.smithi032.stdout:9/530: creat d0/da4/fb2 x:0 0 0 2024-06-03T01:21:04.397 INFO:tasks.workunit.client.0.smithi032.stdout:9/531: write d0/d11/d1d/d38/d4f/db0/f39 [4773079,106470] 0 2024-06-03T01:21:04.400 INFO:tasks.workunit.client.0.smithi032.stdout:7/482: unlink dc/d12/d22/d42/d61/d82/f97 0 2024-06-03T01:21:04.415 INFO:tasks.workunit.client.1.smithi044.stdout:2/953: sync 2024-06-03T01:21:04.417 INFO:tasks.workunit.client.1.smithi044.stdout:2/954: creat db/d15/d49/d82/d113/f134 x:0 0 0 2024-06-03T01:21:04.417 INFO:tasks.workunit.client.1.smithi044.stdout:2/955: fsync db/d1b/d27/f32 0 2024-06-03T01:21:04.417 INFO:tasks.workunit.client.1.smithi044.stdout:2/956: creat db/d15/d49/d82/db5/f135 x:0 0 0 2024-06-03T01:21:04.417 INFO:tasks.workunit.client.1.smithi044.stdout:2/957: dread - db/d15/d49/d82/db5/f135 zero size 2024-06-03T01:21:04.418 INFO:tasks.workunit.client.1.smithi044.stdout:2/958: mkdir db/d1b/d2a/d52/df0/d136 0 2024-06-03T01:21:04.420 INFO:tasks.workunit.client.1.smithi044.stdout:2/959: creat db/d1b/d2a/d52/d54/f137 x:0 0 0 2024-06-03T01:21:04.422 INFO:tasks.workunit.client.0.smithi032.stdout:7/483: dwrite dc/f3d [0,4194304] 0 2024-06-03T01:21:04.422 INFO:tasks.workunit.client.0.smithi032.stdout:9/532: dwrite d0/d23/f3d [0,4194304] 0 2024-06-03T01:21:04.422 INFO:tasks.workunit.client.0.smithi032.stdout:9/533: fdatasync d0/ff 0 2024-06-03T01:21:04.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/484: dread dc/d12/d2a/d2f/f39 [0,4194304] 0 2024-06-03T01:21:04.432 INFO:tasks.workunit.client.0.smithi032.stdout:9/534: read d0/d11/d1d/da6/d71/f96 [2530634,113020] 0 2024-06-03T01:21:04.434 INFO:tasks.workunit.client.0.smithi032.stdout:7/485: symlink dc/d12/l99 0 2024-06-03T01:21:04.434 INFO:tasks.workunit.client.0.smithi032.stdout:9/535: truncate d0/d23/d24/f84 4439877 0 2024-06-03T01:21:04.436 INFO:tasks.workunit.client.0.smithi032.stdout:9/536: truncate d0/d11/d1d/f99 3806884 0 2024-06-03T01:21:04.437 INFO:tasks.workunit.client.0.smithi032.stdout:7/486: rmdir dc/d12/d6a/d86/d88 39 2024-06-03T01:21:04.437 INFO:tasks.workunit.client.0.smithi032.stdout:7/487: chown dc/d12/d1b 230 1 2024-06-03T01:21:04.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/960: dwrite db/d15/d49/d6c/d72/d8f/fda [0,4194304] 0 2024-06-03T01:21:04.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/961: write db/d15/d49/d82/db5/ff4 [707067,1178] 0 2024-06-03T01:21:04.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/962: chown db/d15/d49/d6c 1778079107 1 2024-06-03T01:21:04.438 INFO:tasks.workunit.client.0.smithi032.stdout:9/537: creat d0/d11/d1d/d38/d4f/db0/d41/fb3 x:0 0 0 2024-06-03T01:21:04.439 INFO:tasks.workunit.client.1.smithi044.stdout:2/963: creat db/d1b/d3f/f138 x:0 0 0 2024-06-03T01:21:04.440 INFO:tasks.workunit.client.0.smithi032.stdout:7/488: rename dc/d12/d78/d8e to dc/d12/d2a/d2f/d9a 0 2024-06-03T01:21:04.443 INFO:tasks.workunit.client.0.smithi032.stdout:9/538: unlink d0/d11/d1d/d38/c6c 0 2024-06-03T01:21:04.446 INFO:tasks.workunit.client.0.smithi032.stdout:9/539: rmdir d0/d11/d9c 39 2024-06-03T01:21:04.451 INFO:tasks.workunit.client.0.smithi032.stdout:9/540: symlink d0/d11/d1d/d38/d4f/db0/lb4 0 2024-06-03T01:21:04.451 INFO:tasks.workunit.client.0.smithi032.stdout:7/489: write dc/f30 [7083307,112883] 0 2024-06-03T01:21:04.456 INFO:tasks.workunit.client.1.smithi044.stdout:2/964: dwrite db/d15/d49/d82/d113/f134 [0,4194304] 0 2024-06-03T01:21:04.457 INFO:tasks.workunit.client.0.smithi032.stdout:8/375: sync 2024-06-03T01:21:04.458 INFO:tasks.workunit.client.0.smithi032.stdout:8/376: stat dc/l30 0 2024-06-03T01:21:04.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/490: dread dc/f3d [0,4194304] 0 2024-06-03T01:21:04.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/491: write dc/d12/d22/f85 [1362448,9948] 0 2024-06-03T01:21:04.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/492: fsync dc/d12/d1b/f2c 0 2024-06-03T01:21:04.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/493: mkdir dc/d12/d22/d42/d9b 0 2024-06-03T01:21:04.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/494: dread - dc/f2d zero size 2024-06-03T01:21:04.466 INFO:tasks.workunit.client.0.smithi032.stdout:9/541: dread d0/d11/d1d/da6/d71/f96 [0,4194304] 0 2024-06-03T01:21:04.466 INFO:tasks.workunit.client.0.smithi032.stdout:7/495: mknod dc/d12/d2a/d2f/d9a/c9c 0 2024-06-03T01:21:04.467 INFO:tasks.workunit.client.0.smithi032.stdout:7/496: chown dc/d12/d2a/d2f/d63/c6e 98527010 1 2024-06-03T01:21:04.467 INFO:tasks.workunit.client.0.smithi032.stdout:7/497: dread - dc/d12/d6a/d86/d88/f94 zero size 2024-06-03T01:21:04.467 INFO:tasks.workunit.client.0.smithi032.stdout:7/498: chown dc/d12/d2a/d2f/c8a 2661 1 2024-06-03T01:21:04.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/542: getdents d0/d11/d1d 0 2024-06-03T01:21:04.469 INFO:tasks.workunit.client.0.smithi032.stdout:7/499: truncate dc/f2b 1493825 0 2024-06-03T01:21:04.471 INFO:tasks.workunit.client.0.smithi032.stdout:9/543: getdents d0/d11 0 2024-06-03T01:21:04.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/500: mkdir dc/d12/d24/d7b/d9d 0 2024-06-03T01:21:04.475 INFO:tasks.workunit.client.0.smithi032.stdout:8/377: write dc/d25/f35 [3875585,127300] 0 2024-06-03T01:21:04.478 INFO:tasks.workunit.client.0.smithi032.stdout:7/501: mknod dc/d12/d6a/d86/d88/d8b/c9e 0 2024-06-03T01:21:04.479 INFO:tasks.workunit.client.0.smithi032.stdout:3/427: sync 2024-06-03T01:21:04.480 INFO:tasks.workunit.client.0.smithi032.stdout:1/378: sync 2024-06-03T01:21:04.480 INFO:tasks.workunit.client.0.smithi032.stdout:3/428: readlink de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:04.480 INFO:tasks.workunit.client.0.smithi032.stdout:4/456: sync 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:0/457: sync 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:6/423: sync 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:6/424: creat d8/d11/d20/d3b/f80 x:0 0 0 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:2/463: sync 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:5/510: sync 2024-06-03T01:21:04.481 INFO:tasks.workunit.client.0.smithi032.stdout:5/511: creat da/dd/df/f9d x:0 0 0 2024-06-03T01:21:04.487 INFO:tasks.workunit.client.0.smithi032.stdout:8/378: truncate dc/d25/d28/d3c/f42 38448 0 2024-06-03T01:21:04.487 INFO:tasks.workunit.client.0.smithi032.stdout:8/379: chown dc/d25/d64 19374796 1 2024-06-03T01:21:04.487 INFO:tasks.workunit.client.0.smithi032.stdout:7/502: rename dc/d12/d22/d42/d9b to dc/d12/d22/d9f 0 2024-06-03T01:21:04.495 INFO:tasks.workunit.client.0.smithi032.stdout:3/429: truncate de/f10 1033721 0 2024-06-03T01:21:04.495 INFO:tasks.workunit.client.0.smithi032.stdout:3/430: write de/d29/d2e/d31/f1c [2130494,93634] 0 2024-06-03T01:21:04.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/379: mknod d18/d29/d3d/d47/c6f 0 2024-06-03T01:21:04.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/464: mknod d5/da/d1a/d2a/c9c 0 2024-06-03T01:21:04.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/465: write d5/f3c [1810286,50043] 0 2024-06-03T01:21:04.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/466: stat d5/d38/d46/d5c/d6c/c7f 0 2024-06-03T01:21:04.500 INFO:tasks.workunit.client.0.smithi032.stdout:5/512: mkdir da/dd/df/d10/d16/d18/d1a/d30/d51/d9e 0 2024-06-03T01:21:04.501 INFO:tasks.workunit.client.0.smithi032.stdout:5/513: write da/dd/df/d10/f4c [177646,14364] 0 2024-06-03T01:21:04.504 INFO:tasks.workunit.client.0.smithi032.stdout:7/503: rename dc/d12/d24/d33/f4b to dc/d12/d22/d42/fa0 0 2024-06-03T01:21:04.504 INFO:tasks.workunit.client.0.smithi032.stdout:7/504: creat dc/d12/fa1 x:0 0 0 2024-06-03T01:21:04.505 INFO:tasks.workunit.client.0.smithi032.stdout:9/544: dwrite d0/f1 [4194304,4194304] 0 2024-06-03T01:21:04.506 INFO:tasks.workunit.client.0.smithi032.stdout:5/514: dread da/dd/df/d10/d16/d25/f56 [0,4194304] 0 2024-06-03T01:21:04.506 INFO:tasks.workunit.client.0.smithi032.stdout:5/515: truncate da/dd/df/d10/d16/d18/f69 797654 0 2024-06-03T01:21:04.506 INFO:tasks.workunit.client.0.smithi032.stdout:5/516: write da/dd/df/d10/d16/d25/d74/f93 [566665,117222] 0 2024-06-03T01:21:04.507 INFO:tasks.workunit.client.0.smithi032.stdout:3/431: truncate de/d29/d2e/d31/d24/f73 2350522 0 2024-06-03T01:21:04.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/467: dread d5/da/f2f [4194304,4194304] 0 2024-06-03T01:21:04.515 INFO:tasks.workunit.client.0.smithi032.stdout:7/505: unlink dc/d12/d6a/d86/d88/d8b/c9e 0 2024-06-03T01:21:04.517 INFO:tasks.workunit.client.0.smithi032.stdout:9/545: creat d0/d23/fb5 x:0 0 0 2024-06-03T01:21:04.517 INFO:tasks.workunit.client.0.smithi032.stdout:9/546: chown d0/d11/f22 17977 1 2024-06-03T01:21:04.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/547: write d0/d23/d25/f62 [9144883,32824] 0 2024-06-03T01:21:04.520 INFO:tasks.workunit.client.0.smithi032.stdout:4/457: dwrite d4/db/da/f31 [0,4194304] 0 2024-06-03T01:21:04.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/506: mknod dc/d12/d2a/ca2 0 2024-06-03T01:21:04.521 INFO:tasks.workunit.client.0.smithi032.stdout:3/432: mknod de/d29/d2e/d31/d3c/d50/d5e/c85 0 2024-06-03T01:21:04.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/507: stat dc/d12/d2a/d2f/d9a/c9c 0 2024-06-03T01:21:04.522 INFO:tasks.workunit.client.0.smithi032.stdout:5/517: dread da/dd/df/f46 [4194304,4194304] 0 2024-06-03T01:21:04.524 INFO:tasks.workunit.client.0.smithi032.stdout:6/425: dwrite f4 [0,4194304] 0 2024-06-03T01:21:04.525 INFO:tasks.workunit.client.0.smithi032.stdout:6/426: readlink l7 0 2024-06-03T01:21:04.525 INFO:tasks.workunit.client.0.smithi032.stdout:1/380: dwrite d18/d32/d3c/f61 [0,4194304] 0 2024-06-03T01:21:04.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/548: fdatasync d0/d23/d25/f8e 0 2024-06-03T01:21:04.528 INFO:tasks.workunit.client.0.smithi032.stdout:9/549: readlink d0/d11/l32 0 2024-06-03T01:21:04.528 INFO:tasks.workunit.client.0.smithi032.stdout:8/380: dwrite dc/f13 [0,4194304] 0 2024-06-03T01:21:04.528 INFO:tasks.workunit.client.0.smithi032.stdout:2/468: dread d5/da/f39 [0,4194304] 0 2024-06-03T01:21:04.529 INFO:tasks.workunit.client.0.smithi032.stdout:7/508: link dc/d12/c65 dc/d12/d24/d7b/ca3 0 2024-06-03T01:21:04.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/433: unlink de/d29/d2e/d31/d23/d53/d69/c6b 0 2024-06-03T01:21:04.529 INFO:tasks.workunit.client.0.smithi032.stdout:7/509: fdatasync dc/d12/d1b/f21 0 2024-06-03T01:21:04.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/434: creat de/d29/d2e/d31/d78/f86 x:0 0 0 2024-06-03T01:21:04.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/435: truncate de/d29/d2e/d31/d78/f80 632092 0 2024-06-03T01:21:04.530 INFO:tasks.workunit.client.0.smithi032.stdout:0/458: dwrite d1/da/d28/d3b/d50/f58 [0,4194304] 0 2024-06-03T01:21:04.530 INFO:tasks.workunit.client.0.smithi032.stdout:0/459: chown d1/da/d10/f2d 2077 1 2024-06-03T01:21:04.531 INFO:tasks.workunit.client.0.smithi032.stdout:6/427: dread d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:04.541 INFO:tasks.workunit.client.0.smithi032.stdout:4/458: dwrite d4/db/f1c [0,4194304] 0 2024-06-03T01:21:04.542 INFO:tasks.workunit.client.0.smithi032.stdout:9/550: truncate d0/d11/d1d/f99 1300864 0 2024-06-03T01:21:04.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/469: mknod d5/d38/d46/d53/d90/c9d 0 2024-06-03T01:21:04.545 INFO:tasks.workunit.client.0.smithi032.stdout:6/428: dread d8/d11/d20/d2b/d39/d19/f1d [0,4194304] 0 2024-06-03T01:21:04.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/510: symlink dc/d12/d6a/d86/d89/la4 0 2024-06-03T01:21:04.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/511: chown dc/d12/d22/c87 6785739 1 2024-06-03T01:21:04.553 INFO:tasks.workunit.client.0.smithi032.stdout:3/436: symlink de/d29/l87 0 2024-06-03T01:21:04.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/937: sync 2024-06-03T01:21:04.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/938: write df/d13/d1b/d56/f86 [910986,52903] 0 2024-06-03T01:21:04.557 INFO:tasks.workunit.client.0.smithi032.stdout:0/460: link d1/f6 d1/da/d10/d29/f96 0 2024-06-03T01:21:04.557 INFO:tasks.workunit.client.0.smithi032.stdout:0/461: chown d1/da/f4d 188697563 1 2024-06-03T01:21:04.559 INFO:tasks.workunit.client.0.smithi032.stdout:5/518: dwrite da/dd/df/d10/d16/d25/d74/f93 [0,4194304] 0 2024-06-03T01:21:04.560 INFO:tasks.workunit.client.0.smithi032.stdout:1/381: dwrite d18/d32/f52 [4194304,4194304] 0 2024-06-03T01:21:04.560 INFO:tasks.workunit.client.0.smithi032.stdout:1/382: fsync d18/d27/f41 0 2024-06-03T01:21:04.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/551: mknod d0/d11/d79/cb6 0 2024-06-03T01:21:04.566 INFO:tasks.workunit.client.0.smithi032.stdout:2/470: mknod d5/d38/d46/d5c/c9e 0 2024-06-03T01:21:04.570 INFO:tasks.workunit.client.0.smithi032.stdout:7/512: truncate dc/f14 1924734 0 2024-06-03T01:21:04.570 INFO:tasks.workunit.client.0.smithi032.stdout:7/513: chown dc/d12/d2a/d2f/f39 1 1 2024-06-03T01:21:04.573 INFO:tasks.workunit.client.0.smithi032.stdout:3/437: symlink de/d29/d2e/d31/d24/l88 0 2024-06-03T01:21:04.574 INFO:tasks.workunit.client.0.smithi032.stdout:4/459: dwrite d4/db/da/f7d [0,4194304] 0 2024-06-03T01:21:04.574 INFO:tasks.workunit.client.0.smithi032.stdout:4/460: readlink d4/d78/l81 0 2024-06-03T01:21:04.575 INFO:tasks.workunit.client.0.smithi032.stdout:0/462: symlink d1/da/d10/l97 0 2024-06-03T01:21:04.575 INFO:tasks.workunit.client.0.smithi032.stdout:0/463: write d1/da/d28/d3b/d68/d83/f95 [389440,27326] 0 2024-06-03T01:21:04.575 INFO:tasks.workunit.client.0.smithi032.stdout:0/464: chown d1/da/d10/d29/l4b 0 1 2024-06-03T01:21:04.577 INFO:tasks.workunit.client.1.smithi044.stdout:3/939: dwrite df/d13/d1b/d59/fd8 [0,4194304] 0 2024-06-03T01:21:04.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/383: dwrite fa [0,4194304] 0 2024-06-03T01:21:04.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/384: chown d18/d27/d2b/d5b 27 1 2024-06-03T01:21:04.581 INFO:tasks.workunit.client.1.smithi044.stdout:3/940: truncate df/f19 797665 0 2024-06-03T01:21:04.582 INFO:tasks.workunit.client.0.smithi032.stdout:5/519: unlink da/dd/df/d10/d16/d18/d1a/d30/f5f 0 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.1.smithi044.stdout:3/941: symlink df/d13/d39/d3d/d50/d130/l140 0 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.1.smithi044.stdout:3/942: dread - df/d13/d39/d3d/fa9 zero size 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.1.smithi044.stdout:3/943: fdatasync df/d13/d39/d3d/d43/d71/f94 0 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.0.smithi032.stdout:1/385: dread d18/d29/d3d/d47/f51 [0,4194304] 0 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.0.smithi032.stdout:9/552: link d0/d23/d24/f9d d0/d11/d79/fb7 0 2024-06-03T01:21:04.583 INFO:tasks.workunit.client.0.smithi032.stdout:9/553: write d0/d11/d76/f7c [4880870,73201] 0 2024-06-03T01:21:04.584 INFO:tasks.workunit.client.0.smithi032.stdout:9/554: chown d0/d11/d79/l98 3208 1 2024-06-03T01:21:04.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/965: sync 2024-06-03T01:21:04.584 INFO:tasks.workunit.client.1.smithi044.stdout:3/944: creat df/d13/d1b/d56/d5c/d98/d12b/f141 x:0 0 0 2024-06-03T01:21:04.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/966: mkdir db/d1b/d2a/d52/df0/d136/d139 0 2024-06-03T01:21:04.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/967: dread - db/d1b/d2a/d52/df0/f117 zero size 2024-06-03T01:21:04.589 INFO:tasks.workunit.client.0.smithi032.stdout:2/471: unlink d5/da/d1a/d2a/d36/f94 0 2024-06-03T01:21:04.589 INFO:tasks.workunit.client.0.smithi032.stdout:2/472: readlink d5/d38/d46/d5c/d6c/d80/l85 0 2024-06-03T01:21:04.589 INFO:tasks.workunit.client.0.smithi032.stdout:2/473: write d5/da/d2e/f3d [971446,67127] 0 2024-06-03T01:21:04.589 INFO:tasks.workunit.client.1.smithi044.stdout:3/945: rename df/d13/d1b/d4b/d69/dd1/f118 to df/d1e/d78/da4/d13d/f142 0 2024-06-03T01:21:04.591 INFO:tasks.workunit.client.0.smithi032.stdout:5/520: dread da/dd/f48 [0,4194304] 0 2024-06-03T01:21:04.592 INFO:tasks.workunit.client.0.smithi032.stdout:5/521: chown da/dd/df/d10/d16/d18/d35/d44/f8f 9281 1 2024-06-03T01:21:04.593 INFO:tasks.workunit.client.1.smithi044.stdout:2/968: mkdir db/d15/d49/d7b/df8/d13a 0 2024-06-03T01:21:04.595 INFO:tasks.workunit.client.1.smithi044.stdout:2/969: unlink db/d15/l40 0 2024-06-03T01:21:04.595 INFO:tasks.workunit.client.1.smithi044.stdout:2/970: chown db/d15/d49/c6f 757117559 1 2024-06-03T01:21:04.596 INFO:tasks.workunit.client.0.smithi032.stdout:7/514: unlink dc/d12/d22/c3e 0 2024-06-03T01:21:04.596 INFO:tasks.workunit.client.0.smithi032.stdout:7/515: chown dc/d12/d2a/d2f/c8a 490 1 2024-06-03T01:21:04.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/438: creat de/d29/d2e/d31/d3c/d50/d5e/f89 x:0 0 0 2024-06-03T01:21:04.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/439: stat de/d29/d2e/d31/d23/f6e 0 2024-06-03T01:21:04.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/440: chown de/d29/d2e/d31/d3c/d50/d56/f60 52011596 1 2024-06-03T01:21:04.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/461: mknod d4/d57/d6c/ca5 0 2024-06-03T01:21:04.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/462: stat d4/db/da/d37/f70 0 2024-06-03T01:21:04.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/463: fdatasync d4/db/da/d1d/f23 0 2024-06-03T01:21:04.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/464: fdatasync d4/db/f36 0 2024-06-03T01:21:04.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/465: creat d1/d46/f98 x:0 0 0 2024-06-03T01:21:04.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/466: chown d1/da/d28/d3b/d50 194661109 1 2024-06-03T01:21:04.612 INFO:tasks.workunit.client.0.smithi032.stdout:1/386: mknod d18/d2d/c70 0 2024-06-03T01:21:04.619 INFO:tasks.workunit.client.1.smithi044.stdout:3/946: dwrite df/d13/d1b/fc6 [0,4194304] 0 2024-06-03T01:21:04.619 INFO:tasks.workunit.client.1.smithi044.stdout:3/947: chown df/d12d/c12e 20715092 1 2024-06-03T01:21:04.619 INFO:tasks.workunit.client.1.smithi044.stdout:3/948: read - df/d1e/d2a/d9b/dc2/f11b zero size 2024-06-03T01:21:04.619 INFO:tasks.workunit.client.0.smithi032.stdout:9/555: rename d0/d23/l77 to d0/d11/d1d/d38/lb8 0 2024-06-03T01:21:04.620 INFO:tasks.workunit.client.0.smithi032.stdout:3/441: dwrite de/d29/d2e/d31/d78/f86 [0,4194304] 0 2024-06-03T01:21:04.620 INFO:tasks.workunit.client.1.smithi044.stdout:2/971: dwrite db/d15/d49/f8e [0,4194304] 0 2024-06-03T01:21:04.623 INFO:tasks.workunit.client.0.smithi032.stdout:2/474: stat d5/da/d1a/f41 0 2024-06-03T01:21:04.624 INFO:tasks.workunit.client.1.smithi044.stdout:3/949: rename df/d12d to df/d13/d1b/d4b/d69/d143 0 2024-06-03T01:21:04.627 INFO:tasks.workunit.client.0.smithi032.stdout:7/516: creat dc/d12/d1b/fa5 x:0 0 0 2024-06-03T01:21:04.630 INFO:tasks.workunit.client.0.smithi032.stdout:0/467: dwrite d1/fc [0,4194304] 0 2024-06-03T01:21:04.631 INFO:tasks.workunit.client.1.smithi044.stdout:2/972: write db/d15/d49/d6c/fa9 [825790,95592] 0 2024-06-03T01:21:04.633 INFO:tasks.workunit.client.0.smithi032.stdout:4/465: mknod d4/db/da/d2d/d27/d8c/ca6 0 2024-06-03T01:21:04.633 INFO:tasks.workunit.client.0.smithi032.stdout:9/556: symlink d0/da4/lb9 0 2024-06-03T01:21:04.633 INFO:tasks.workunit.client.0.smithi032.stdout:9/557: chown d0/d11/d1d/f1e 8283 1 2024-06-03T01:21:04.635 INFO:tasks.workunit.client.1.smithi044.stdout:2/973: unlink db/d1b/l21 0 2024-06-03T01:21:04.636 INFO:tasks.workunit.client.0.smithi032.stdout:5/522: rename da/dd/df/l22 to da/dd/df/d10/d16/d18/d1a/d30/d58/l9f 0 2024-06-03T01:21:04.636 INFO:tasks.workunit.client.0.smithi032.stdout:5/523: chown da/dd/df/d10/d16/d18/d1a/d30/d58/l6b 1 1 2024-06-03T01:21:04.637 INFO:tasks.workunit.client.0.smithi032.stdout:3/442: link de/d29/d2e/l6f de/d29/d2e/d31/d23/d53/l8a 0 2024-06-03T01:21:04.647 INFO:tasks.workunit.client.0.smithi032.stdout:7/517: symlink dc/d12/d2a/la6 0 2024-06-03T01:21:04.649 INFO:tasks.workunit.client.0.smithi032.stdout:7/518: write dc/d12/d22/d42/f50 [98994,79978] 0 2024-06-03T01:21:04.649 INFO:tasks.workunit.client.0.smithi032.stdout:7/519: truncate dc/d12/f6b 1925419 0 2024-06-03T01:21:04.649 INFO:tasks.workunit.client.0.smithi032.stdout:7/520: chown dc/d12/d1b/fa5 9 1 2024-06-03T01:21:04.650 INFO:tasks.workunit.client.0.smithi032.stdout:0/468: rename d1/da/d10/d53/l59 to d1/da/d10/d53/l99 0 2024-06-03T01:21:04.650 INFO:tasks.workunit.client.0.smithi032.stdout:0/469: readlink d1/da/d10/l20 0 2024-06-03T01:21:04.652 INFO:tasks.workunit.client.0.smithi032.stdout:1/387: rename d18/d3f/c46 to d18/d2d/d5e/c71 0 2024-06-03T01:21:04.653 INFO:tasks.workunit.client.0.smithi032.stdout:1/388: stat d18/d29/d3d/l4d 0 2024-06-03T01:21:04.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/524: link da/dd/df/d10/d16/d18/f1d da/dd/df/d10/d16/fa0 0 2024-06-03T01:21:04.656 INFO:tasks.workunit.client.0.smithi032.stdout:5/525: fsync da/dd/df/d10/d16/d18/d35/f98 0 2024-06-03T01:21:04.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/526: dread da/dd/df/d10/d16/d18/d1a/d30/d51/f57 [0,4194304] 0 2024-06-03T01:21:04.657 INFO:tasks.workunit.client.0.smithi032.stdout:3/443: creat de/f8b x:0 0 0 2024-06-03T01:21:04.657 INFO:tasks.workunit.client.0.smithi032.stdout:3/444: chown de/d63/f7a 764636 1 2024-06-03T01:21:04.659 INFO:tasks.workunit.client.0.smithi032.stdout:7/521: truncate dc/d12/d24/d7b/f20 2920961 0 2024-06-03T01:21:04.659 INFO:tasks.workunit.client.0.smithi032.stdout:7/522: creat dc/d12/d6a/fa7 x:0 0 0 2024-06-03T01:21:04.660 INFO:tasks.workunit.client.0.smithi032.stdout:1/389: dread d18/d32/f52 [0,4194304] 0 2024-06-03T01:21:04.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/390: write d18/d3f/f53 [81309,70509] 0 2024-06-03T01:21:04.664 INFO:tasks.workunit.client.0.smithi032.stdout:0/470: creat d1/da/d28/d3b/d68/d82/f9a x:0 0 0 2024-06-03T01:21:04.664 INFO:tasks.workunit.client.0.smithi032.stdout:0/471: truncate d1/f64 1311131 0 2024-06-03T01:21:04.668 INFO:tasks.workunit.client.0.smithi032.stdout:9/558: rename d0/d23/d24/f84 to d0/d11/d1d/da6/fba 0 2024-06-03T01:21:04.668 INFO:tasks.workunit.client.0.smithi032.stdout:4/466: dwrite d4/db/da/d1d/f7e [0,4194304] 0 2024-06-03T01:21:04.668 INFO:tasks.workunit.client.0.smithi032.stdout:4/467: stat d4/d57/f9c 0 2024-06-03T01:21:04.668 INFO:tasks.workunit.client.0.smithi032.stdout:4/468: write d4/d5c/f5e [1583519,3640] 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/950: dwrite df/d13/d39/d3d/d50/ffd [0,4194304] 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/951: write df/d1e/d3a/d84/d90/ffa [770931,49741] 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/952: write df/d13/d1b/d56/f67 [2594638,12899] 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/953: read f9 [2048849,80904] 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/954: getdents df/d13/d6d/d9a/db7/df8 0 2024-06-03T01:21:04.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/955: fsync df/d1e/d78/dda/d12c/fd2 0 2024-06-03T01:21:04.670 INFO:tasks.workunit.client.1.smithi044.stdout:2/974: dwrite db/d1b/d2a/d2f/dbe/ded/dfa/f105 [0,4194304] 0 2024-06-03T01:21:04.672 INFO:tasks.workunit.client.0.smithi032.stdout:5/527: creat da/dd/df/d10/d16/fa1 x:0 0 0 2024-06-03T01:21:04.672 INFO:tasks.workunit.client.0.smithi032.stdout:9/559: dread d0/d11/d1d/f54 [0,4194304] 0 2024-06-03T01:21:04.672 INFO:tasks.workunit.client.1.smithi044.stdout:2/975: dread db/d15/d49/d82/db5/ff4 [0,4194304] 0 2024-06-03T01:21:04.673 INFO:tasks.workunit.client.1.smithi044.stdout:2/976: read db/d15/d49/d6c/d72/dce/f116 [34381,101976] 0 2024-06-03T01:21:04.677 INFO:tasks.workunit.client.1.smithi044.stdout:3/956: creat df/d13/d6d/d9a/db7/df8/f144 x:0 0 0 2024-06-03T01:21:04.679 INFO:tasks.workunit.client.0.smithi032.stdout:1/391: creat d18/d32/d39/d60/f72 x:0 0 0 2024-06-03T01:21:04.679 INFO:tasks.workunit.client.0.smithi032.stdout:1/392: truncate d18/d2d/f69 487029 0 2024-06-03T01:21:04.681 INFO:tasks.workunit.client.1.smithi044.stdout:2/977: mkdir db/d1b/d3f/d13b 0 2024-06-03T01:21:04.681 INFO:tasks.workunit.client.0.smithi032.stdout:0/472: mknod d1/da/d10/d29/d6c/c9b 0 2024-06-03T01:21:04.681 INFO:tasks.workunit.client.0.smithi032.stdout:0/473: write d1/da/f74 [186992,113625] 0 2024-06-03T01:21:04.681 INFO:tasks.workunit.client.0.smithi032.stdout:0/474: stat d1/da/d28/d3b/d5d/f94 0 2024-06-03T01:21:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/523: dwrite dc/d12/d24/d33/f5c [0,4194304] 0 2024-06-03T01:21:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/524: chown dc/d12/d22 135654229 1 2024-06-03T01:21:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/525: fdatasync dc/f2b 0 2024-06-03T01:21:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/526: dread - dc/f2d zero size 2024-06-03T01:21:04.687 INFO:tasks.workunit.client.0.smithi032.stdout:4/469: getdents d4/db/da/d2d/d27/d8c 0 2024-06-03T01:21:04.692 INFO:tasks.workunit.client.0.smithi032.stdout:3/445: getdents de/d29/d2e/d31/d78 0 2024-06-03T01:21:04.696 INFO:tasks.workunit.client.0.smithi032.stdout:1/393: rename d18/d27/f41 to d18/d3f/d5f/d6b/f73 0 2024-06-03T01:21:04.698 INFO:tasks.workunit.client.0.smithi032.stdout:1/394: read d18/f19 [5057996,38700] 0 2024-06-03T01:21:04.698 INFO:tasks.workunit.client.0.smithi032.stdout:1/395: fsync fa 0 2024-06-03T01:21:04.699 INFO:tasks.workunit.client.0.smithi032.stdout:4/470: dread d4/db/da/f15 [0,4194304] 0 2024-06-03T01:21:04.699 INFO:tasks.workunit.client.0.smithi032.stdout:0/475: mkdir d1/da/d28/d3b/d5d/d8d/d9c 0 2024-06-03T01:21:04.699 INFO:tasks.workunit.client.0.smithi032.stdout:0/476: chown d1/da/d10/d53/f7e 75198201 1 2024-06-03T01:21:04.709 INFO:tasks.workunit.client.0.smithi032.stdout:7/527: creat dc/d12/d24/d7b/d9d/fa8 x:0 0 0 2024-06-03T01:21:04.709 INFO:tasks.workunit.client.0.smithi032.stdout:7/528: creat dc/d12/d2a/d2f/fa9 x:0 0 0 2024-06-03T01:21:04.711 INFO:tasks.workunit.client.0.smithi032.stdout:9/560: dwrite d0/d11/f86 [0,4194304] 0 2024-06-03T01:21:04.711 INFO:tasks.workunit.client.0.smithi032.stdout:9/561: write d0/d23/d24/f9d [22063,96201] 0 2024-06-03T01:21:04.712 INFO:tasks.workunit.client.0.smithi032.stdout:5/528: dwrite da/dd/df/d10/d16/d18/d35/d70/f86 [0,4194304] 0 2024-06-03T01:21:04.712 INFO:tasks.workunit.client.0.smithi032.stdout:5/529: dread - da/dd/df/d10/d16/d18/d35/d44/f8f zero size 2024-06-03T01:21:04.712 INFO:tasks.workunit.client.0.smithi032.stdout:5/530: chown da/dd/df/d10/d16/d18/d1a/d30/d58/l67 228242 1 2024-06-03T01:21:04.712 INFO:tasks.workunit.client.0.smithi032.stdout:5/531: dread - da/dd/df/d10/d16/d18/d1a/d30/d51/f7b zero size 2024-06-03T01:21:04.713 INFO:tasks.workunit.client.0.smithi032.stdout:3/446: mkdir de/d29/d2e/d31/d23/d53/d69/d8c 0 2024-06-03T01:21:04.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/957: dwrite f9 [0,4194304] 0 2024-06-03T01:21:04.713 INFO:tasks.workunit.client.1.smithi044.stdout:2/978: dwrite db/d15/d49/d6c/d72/dcd/fcf [0,4194304] 0 2024-06-03T01:21:04.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/958: dread df/d13/d1b/f5e [0,4194304] 0 2024-06-03T01:21:04.715 INFO:tasks.workunit.client.1.smithi044.stdout:2/979: symlink db/d1b/dd0/l13c 0 2024-06-03T01:21:04.715 INFO:tasks.workunit.client.1.smithi044.stdout:2/980: write db/d15/d49/d6c/d72/d8f/de5/ff6 [1254,16710] 0 2024-06-03T01:21:04.715 INFO:tasks.workunit.client.1.smithi044.stdout:3/959: rmdir df/d1e/d78/dda/d12c/dbc 39 2024-06-03T01:21:04.719 INFO:tasks.workunit.client.1.smithi044.stdout:2/981: rename db/d1b/d27 to db/d106/d13d 0 2024-06-03T01:21:04.720 INFO:tasks.workunit.client.0.smithi032.stdout:0/477: symlink d1/da/d10/d80/l9d 0 2024-06-03T01:21:04.720 INFO:tasks.workunit.client.0.smithi032.stdout:0/478: creat d1/da/f9e x:0 0 0 2024-06-03T01:21:04.720 INFO:tasks.workunit.client.0.smithi032.stdout:0/479: truncate d1/da/d28/d3b/d68/d83/f95 1220157 0 2024-06-03T01:21:04.722 INFO:tasks.workunit.client.0.smithi032.stdout:4/471: dwrite d4/db/da/d37/f84 [0,4194304] 0 2024-06-03T01:21:04.722 INFO:tasks.workunit.client.0.smithi032.stdout:4/472: stat d4/db/da/d2d/d27/d53/c61 0 2024-06-03T01:21:04.723 INFO:tasks.workunit.client.0.smithi032.stdout:4/473: read d4/db/da/d2d/d27/f60 [1561,87386] 0 2024-06-03T01:21:04.723 INFO:tasks.workunit.client.0.smithi032.stdout:4/474: write d4/db/da/f96 [124954,86594] 0 2024-06-03T01:21:04.723 INFO:tasks.workunit.client.0.smithi032.stdout:4/475: write d4/db/da/d37/d90/f95 [836606,60381] 0 2024-06-03T01:21:04.724 INFO:tasks.workunit.client.1.smithi044.stdout:2/982: creat db/d1b/d2a/d2f/dbe/f13e x:0 0 0 2024-06-03T01:21:04.724 INFO:tasks.workunit.client.1.smithi044.stdout:2/983: write db/d106/d13d/d79/f104 [565695,114178] 0 2024-06-03T01:21:04.724 INFO:tasks.workunit.client.1.smithi044.stdout:2/984: chown db/d1b/d3f/f66 1114155 1 2024-06-03T01:21:04.724 INFO:tasks.workunit.client.1.smithi044.stdout:2/985: chown db/dd/df9 0 1 2024-06-03T01:21:04.726 INFO:tasks.workunit.client.1.smithi044.stdout:2/986: mknod db/d1b/d2a/d52/df0/c13f 0 2024-06-03T01:21:04.726 INFO:tasks.workunit.client.1.smithi044.stdout:2/987: getdents db/d1b/d2a/d2f/d4c/df5 0 2024-06-03T01:21:04.726 INFO:tasks.workunit.client.0.smithi032.stdout:1/396: dwrite d18/f1a [0,4194304] 0 2024-06-03T01:21:04.728 INFO:tasks.workunit.client.1.smithi044.stdout:2/988: mkdir db/dd/df9/d140 0 2024-06-03T01:21:04.730 INFO:tasks.workunit.client.0.smithi032.stdout:9/562: dwrite d0/d11/d1d/da6/fa2 [0,4194304] 0 2024-06-03T01:21:04.733 INFO:tasks.workunit.client.0.smithi032.stdout:4/476: dread d4/db/da/f31 [0,4194304] 0 2024-06-03T01:21:04.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/447: mkdir de/d29/d2e/d31/d8d 0 2024-06-03T01:21:04.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/448: write de/d29/d2e/d31/d3c/f61 [1540420,59892] 0 2024-06-03T01:21:04.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/449: read de/d29/d2e/d31/d3c/f61 [441927,25772] 0 2024-06-03T01:21:04.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/450: truncate de/d29/d2e/f45 3826709 0 2024-06-03T01:21:04.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/451: dread - de/d63/f7a zero size 2024-06-03T01:21:04.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/452: write de/d29/d2e/d31/f21 [5497913,79595] 0 2024-06-03T01:21:04.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/453: truncate de/d29/d2e/d31/d3c/f54 187729 0 2024-06-03T01:21:04.737 INFO:tasks.workunit.client.0.smithi032.stdout:0/480: mkdir d1/da/d10/d80/d9f 0 2024-06-03T01:21:04.740 INFO:tasks.workunit.client.0.smithi032.stdout:7/529: truncate dc/d12/d1b/f36 3627584 0 2024-06-03T01:21:04.744 INFO:tasks.workunit.client.0.smithi032.stdout:1/397: mknod d18/d27/d2b/c74 0 2024-06-03T01:21:04.744 INFO:tasks.workunit.client.0.smithi032.stdout:1/398: fdatasync d18/d32/f52 0 2024-06-03T01:21:04.749 INFO:tasks.workunit.client.1.smithi044.stdout:3/960: dwrite df/d13/d1b/d56/f86 [0,4194304] 0 2024-06-03T01:21:04.750 INFO:tasks.workunit.client.1.smithi044.stdout:3/961: creat df/d13/de5/f145 x:0 0 0 2024-06-03T01:21:04.750 INFO:tasks.workunit.client.1.smithi044.stdout:3/962: readlink df/d1e/d2a/l10f 0 2024-06-03T01:21:04.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.751+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75cc1056f0 con 0x7f75b8039360 2024-06-03T01:21:04.752 INFO:tasks.workunit.client.0.smithi032.stdout:4/477: symlink d4/db/da/d1d/d6b/la7 0 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.752+0000 7f75c2ffd700 1 -- 172.21.15.32:0/2822759766 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f75cc1056f0 con 0x7f75b8039360 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "1/2 daemons upgraded", 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:21:04.753 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:21:04.756 INFO:tasks.workunit.client.1.smithi044.stdout:3/963: link df/d13/d1b/d56/d5c/f5f df/d1e/d78/da4/d13d/f146 0 2024-06-03T01:21:04.756 INFO:tasks.workunit.client.1.smithi044.stdout:3/964: creat df/d1e/d3a/d84/f147 x:0 0 0 2024-06-03T01:21:04.759 INFO:tasks.workunit.client.1.smithi044.stdout:3/965: dread fe [0,4194304] 0 2024-06-03T01:21:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 msgr2=0x7f75b803b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 0x7f75b803b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75cc069ef0 tx=0x7f75c8009260 comp rx=0 tx=0).stop 2024-06-03T01:21:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 msgr2=0x7f75cc069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc00f690 comp rx=0 tx=0).stop 2024-06-03T01:21:04.760 INFO:tasks.workunit.client.0.smithi032.stdout:9/563: dwrite d0/d23/f85 [0,4194304] 0 2024-06-03T01:21:04.760 INFO:tasks.workunit.client.0.smithi032.stdout:9/564: write d0/d11/f22 [8654059,15730] 0 2024-06-03T01:21:04.760 INFO:tasks.workunit.client.0.smithi032.stdout:9/565: stat d0/d11/d1d/d38/d4f/cae 0 2024-06-03T01:21:04.760 INFO:tasks.workunit.client.0.smithi032.stdout:9/566: readlink d0/l7 0 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 shutdown_connections 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f75b8039360 0x7f75b803b820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75cc0ffba0 0x7f75cc069970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 --2- 172.21.15.32:0/2822759766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75cc100da0 0x7f75cc19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.759+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 >> 172.21.15.32:0/2822759766 conn(0x7f75cc0fb120 msgr2=0x7f75cc103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.760+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 shutdown_connections 2024-06-03T01:21:04.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:04.760+0000 7f75d4039700 1 -- 172.21.15.32:0/2822759766 wait complete. 2024-06-03T01:21:04.762 INFO:tasks.workunit.client.1.smithi044.stdout:2/989: dwrite db/d106/d13d/d79/f104 [0,4194304] 0 2024-06-03T01:21:04.765 INFO:tasks.workunit.client.0.smithi032.stdout:9/567: dread d0/d11/d1d/d38/f61 [0,4194304] 0 2024-06-03T01:21:04.766 INFO:tasks.workunit.client.1.smithi044.stdout:2/990: truncate db/d1b/f22 3629231 0 2024-06-03T01:21:04.767 INFO:tasks.workunit.client.1.smithi044.stdout:2/991: write db/d1b/d2a/d67/d53/f93 [2484221,47341] 0 2024-06-03T01:21:04.767 INFO:tasks.workunit.client.1.smithi044.stdout:2/992: chown db/d1b/d2a/d52/d54/db6/fc6 14062 1 2024-06-03T01:21:04.767 INFO:tasks.workunit.client.0.smithi032.stdout:1/399: dwrite d18/d3f/f53 [0,4194304] 0 2024-06-03T01:21:04.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/454: link de/d29/d2e/d31/d78/f86 de/d29/d2e/d31/d23/d53/d69/f8e 0 2024-06-03T01:21:04.768 INFO:tasks.workunit.client.1.smithi044.stdout:3/966: write df/f23 [1870201,99790] 0 2024-06-03T01:21:04.769 INFO:tasks.workunit.client.1.smithi044.stdout:2/993: link db/d106/d13d/f32 db/d1b/d2a/d52/df0/d136/f141 0 2024-06-03T01:21:04.773 INFO:tasks.workunit.client.0.smithi032.stdout:0/481: creat d1/da/d28/d3b/d5d/d8d/d9c/fa0 x:0 0 0 2024-06-03T01:21:04.773 INFO:tasks.workunit.client.0.smithi032.stdout:0/482: write d1/da/d10/f19 [5220561,109342] 0 2024-06-03T01:21:04.773 INFO:tasks.workunit.client.0.smithi032.stdout:0/483: dread - d1/da/d28/d3b/d50/f5c zero size 2024-06-03T01:21:04.773 INFO:tasks.workunit.client.1.smithi044.stdout:2/994: dread - db/d1b/d5d/da7/dc9/fe3 zero size 2024-06-03T01:21:04.775 INFO:tasks.workunit.client.0.smithi032.stdout:7/530: unlink dc/d12/c3b 0 2024-06-03T01:21:04.778 INFO:tasks.workunit.client.0.smithi032.stdout:4/478: rename d4/db/da/l56 to d4/d7f/la8 0 2024-06-03T01:21:04.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/532: write da/dd/df/d10/d16/d18/d35/d70/f86 [4970660,82204] 0 2024-06-03T01:21:04.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/533: stat da/dd/df/f33 0 2024-06-03T01:21:04.785 INFO:tasks.workunit.client.1.smithi044.stdout:2/995: dread db/d1b/d2a/d67/f7c [0,4194304] 0 2024-06-03T01:21:04.786 INFO:tasks.workunit.client.1.smithi044.stdout:2/996: mknod db/d1b/d2a/d52/d54/c142 0 2024-06-03T01:21:04.786 INFO:tasks.workunit.client.0.smithi032.stdout:9/568: creat d0/d11/d1d/fbb x:0 0 0 2024-06-03T01:21:04.786 INFO:tasks.workunit.client.0.smithi032.stdout:9/569: write d0/f15 [854039,58114] 0 2024-06-03T01:21:04.786 INFO:tasks.workunit.client.1.smithi044.stdout:2/997: mkdir db/d1b/d2a/d2f/d143 0 2024-06-03T01:21:04.790 INFO:tasks.workunit.client.1.smithi044.stdout:2/998: chown db/d1b/d5d/da7/dc9/f125 3 1 2024-06-03T01:21:04.791 INFO:tasks.workunit.client.1.smithi044.stdout:2/999: write db/f12 [252878,118802] 0 2024-06-03T01:21:04.794 INFO:tasks.workunit.client.0.smithi032.stdout:9/570: dread d0/f19 [0,4194304] 0 2024-06-03T01:21:04.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/400: mknod d18/d27/d2b/c75 0 2024-06-03T01:21:04.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/401: fdatasync d18/f1c 0 2024-06-03T01:21:04.798 INFO:tasks.workunit.client.0.smithi032.stdout:3/455: rename de/d29/d2e/d31/l2a to de/d29/d2e/d31/l8f 0 2024-06-03T01:21:04.801 INFO:tasks.workunit.client.0.smithi032.stdout:5/534: dwrite da/dd/df/d10/d16/d18/d35/f81 [0,4194304] 0 2024-06-03T01:21:04.802 INFO:tasks.workunit.client.0.smithi032.stdout:0/484: link d1/da/d28/l54 d1/da/d28/d3b/d68/d82/la1 0 2024-06-03T01:21:04.805 INFO:tasks.workunit.client.0.smithi032.stdout:0/485: write d1/da/d28/d3b/d68/d83/f95 [360475,57859] 0 2024-06-03T01:21:04.805 INFO:tasks.workunit.client.0.smithi032.stdout:0/486: write d1/da/d10/f7a [1125578,42608] 0 2024-06-03T01:21:04.805 INFO:tasks.workunit.client.0.smithi032.stdout:0/487: fsync d1/da/d10/d29/f45 0 2024-06-03T01:21:04.806 INFO:tasks.workunit.client.0.smithi032.stdout:7/531: mkdir dc/d12/d24/d33/daa 0 2024-06-03T01:21:04.806 INFO:tasks.workunit.client.0.smithi032.stdout:7/532: chown dc/d12/d2a/d2f/fa9 1 1 2024-06-03T01:21:04.808 INFO:tasks.workunit.client.0.smithi032.stdout:4/479: creat d4/db/da/d1d/fa9 x:0 0 0 2024-06-03T01:21:04.816 INFO:tasks.workunit.client.0.smithi032.stdout:3/456: dwrite de/d29/f4b [0,4194304] 0 2024-06-03T01:21:04.818 INFO:tasks.workunit.client.0.smithi032.stdout:4/480: dread d4/db/da/d37/f84 [0,4194304] 0 2024-06-03T01:21:04.818 INFO:tasks.workunit.client.0.smithi032.stdout:4/481: write d4/d57/d6c/f6e [1638735,7221] 0 2024-06-03T01:21:04.819 INFO:tasks.workunit.client.0.smithi032.stdout:0/488: dwrite d1/da/d10/d53/f60 [0,4194304] 0 2024-06-03T01:21:04.820 INFO:tasks.workunit.client.0.smithi032.stdout:9/571: link d0/d11/d9c/fa3 d0/d11/d1d/fbc 0 2024-06-03T01:21:04.820 INFO:tasks.workunit.client.0.smithi032.stdout:9/572: dread - d0/d11/d1d/d38/d4f/db0/d41/faa zero size 2024-06-03T01:21:04.820 INFO:tasks.workunit.client.0.smithi032.stdout:9/573: write d0/d11/d1d/d38/f97 [716749,73272] 0 2024-06-03T01:21:04.822 INFO:tasks.workunit.client.0.smithi032.stdout:1/402: rename d18/d29/d3d/d47/l38 to d18/d2d/d5e/l76 0 2024-06-03T01:21:04.826 INFO:tasks.workunit.client.0.smithi032.stdout:7/533: link dc/d12/f13 dc/d12/d6a/d86/d89/fab 0 2024-06-03T01:21:04.829 INFO:tasks.workunit.client.0.smithi032.stdout:3/457: truncate de/d29/d2e/d31/d23/f59 924861 0 2024-06-03T01:21:04.831 INFO:tasks.workunit.client.0.smithi032.stdout:4/482: mknod d4/d57/d6c/caa 0 2024-06-03T01:21:04.833 INFO:tasks.workunit.client.0.smithi032.stdout:6/429: sync 2024-06-03T01:21:04.833 INFO:tasks.workunit.client.0.smithi032.stdout:8/381: sync 2024-06-03T01:21:04.833 INFO:tasks.workunit.client.0.smithi032.stdout:8/382: readlink dc/l30 0 2024-06-03T01:21:04.834 INFO:tasks.workunit.client.0.smithi032.stdout:8/383: dread dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:04.835 INFO:tasks.workunit.client.0.smithi032.stdout:8/384: truncate f1 1766448 0 2024-06-03T01:21:04.835 INFO:tasks.workunit.client.0.smithi032.stdout:0/489: symlink d1/da/d28/la2 0 2024-06-03T01:21:04.837 INFO:tasks.workunit.client.0.smithi032.stdout:9/574: symlink d0/d23/d24/lbd 0 2024-06-03T01:21:04.838 INFO:tasks.workunit.client.0.smithi032.stdout:9/575: creat d0/da4/fbe x:0 0 0 2024-06-03T01:21:04.842 INFO:tasks.workunit.client.1.smithi044.stdout:3/967: sync 2024-06-03T01:21:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/968: truncate df/d1e/d78/dda/d12c/fba 1438366 0 2024-06-03T01:21:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/969: dread - df/d1e/d78/dda/d12c/ff0 zero size 2024-06-03T01:21:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/970: truncate df/d13/f25 1236116 0 2024-06-03T01:21:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/971: write df/d13/d1b/d56/f7f [1487354,122277] 0 2024-06-03T01:21:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/972: fsync df/d13/d6d/fed 0 2024-06-03T01:21:04.844 INFO:tasks.workunit.client.0.smithi032.stdout:1/403: dwrite d18/d27/d2b/f54 [0,4194304] 0 2024-06-03T01:21:04.844 INFO:tasks.workunit.client.0.smithi032.stdout:7/534: creat dc/d12/d2a/d81/fac x:0 0 0 2024-06-03T01:21:04.845 INFO:tasks.workunit.client.1.smithi044.stdout:3/973: link df/d13/d39/d3d/l123 df/d13/d1b/d56/d5c/de3/d11a/l148 0 2024-06-03T01:21:04.846 INFO:tasks.workunit.client.0.smithi032.stdout:3/458: mkdir de/d29/d2e/d31/d23/d53/d69/d90 0 2024-06-03T01:21:04.847 INFO:tasks.workunit.client.1.smithi044.stdout:3/974: rmdir df/d1e/d3a/d84/d90 39 2024-06-03T01:21:04.848 INFO:tasks.workunit.client.0.smithi032.stdout:4/483: mkdir d4/db/da/d37/dab 0 2024-06-03T01:21:04.848 INFO:tasks.workunit.client.0.smithi032.stdout:4/484: chown d4/db/da/d2d/c26 11391686 1 2024-06-03T01:21:04.849 INFO:tasks.workunit.client.0.smithi032.stdout:6/430: link d8/d11/d20/d31/l33 d8/d11/d20/d2b/d39/d19/l81 0 2024-06-03T01:21:04.851 INFO:tasks.workunit.client.0.smithi032.stdout:8/385: creat dc/d14/f83 x:0 0 0 2024-06-03T01:21:04.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/475: sync 2024-06-03T01:21:04.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/476: write d5/da/d1a/d2a/d36/f60 [3701652,102638] 0 2024-06-03T01:21:04.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/477: truncate d5/da/f39 4327720 0 2024-06-03T01:21:04.855 INFO:tasks.workunit.client.1.smithi044.stdout:3/975: dread df/d13/d1b/fc6 [0,4194304] 0 2024-06-03T01:21:04.856 INFO:tasks.workunit.client.0.smithi032.stdout:9/576: rename d0/d11/d1d/l67 to d0/d23/lbf 0 2024-06-03T01:21:04.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/404: dread d18/d27/d2b/f54 [0,4194304] 0 2024-06-03T01:21:04.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/405: creat d18/d29/d3d/d47/f77 x:0 0 0 2024-06-03T01:21:04.866 INFO:tasks.workunit.client.0.smithi032.stdout:7/535: creat dc/d12/d22/fad x:0 0 0 2024-06-03T01:21:04.866 INFO:tasks.workunit.client.0.smithi032.stdout:7/536: write dc/d12/d24/d33/f66 [372361,128946] 0 2024-06-03T01:21:04.866 INFO:tasks.workunit.client.0.smithi032.stdout:7/537: stat dc/d12/f77 0 2024-06-03T01:21:04.867 INFO:tasks.workunit.client.1.smithi044.stdout:3/976: dread df/f10b [0,4194304] 0 2024-06-03T01:21:04.867 INFO:tasks.workunit.client.1.smithi044.stdout:3/977: write df/d1e/d3a/d84/fc0 [339361,23165] 0 2024-06-03T01:21:04.872 INFO:tasks.workunit.client.0.smithi032.stdout:4/485: mknod d4/db/da/d37/d90/da1/cac 0 2024-06-03T01:21:04.872 INFO:tasks.workunit.client.0.smithi032.stdout:4/486: truncate d4/db/da/d2d/f9f 619147 0 2024-06-03T01:21:04.872 INFO:tasks.workunit.client.0.smithi032.stdout:4/487: creat d4/d7f/fad x:0 0 0 2024-06-03T01:21:04.873 INFO:tasks.workunit.client.0.smithi032.stdout:6/431: chown d8/d55/l30 784613147 1 2024-06-03T01:21:04.878 INFO:tasks.workunit.client.0.smithi032.stdout:3/459: read de/d29/d2e/f45 [3540415,107368] 0 2024-06-03T01:21:04.880 INFO:tasks.workunit.client.0.smithi032.stdout:2/478: creat d5/da/d1a/d2a/d36/f9f x:0 0 0 2024-06-03T01:21:04.886 INFO:tasks.workunit.client.0.smithi032.stdout:9/577: dwrite d0/fe [8388608,4194304] 0 2024-06-03T01:21:04.887 INFO:tasks.workunit.client.0.smithi032.stdout:3/460: rmdir de/d29/d2e/d31/d3c 39 2024-06-03T01:21:04.887 INFO:tasks.workunit.client.0.smithi032.stdout:7/538: write dc/d12/d24/d33/f5c [2117413,126424] 0 2024-06-03T01:21:04.887 INFO:tasks.workunit.client.0.smithi032.stdout:7/539: chown dc/l53 4 1 2024-06-03T01:21:04.891 INFO:tasks.workunit.client.0.smithi032.stdout:2/479: mknod d5/da/d1a/d2a/d49/d7e/d86/ca0 0 2024-06-03T01:21:04.891 INFO:tasks.workunit.client.0.smithi032.stdout:2/480: dread - d5/d38/d46/f81 zero size 2024-06-03T01:21:04.891 INFO:tasks.workunit.client.0.smithi032.stdout:2/481: chown d5/da/f2f 63732937 1 2024-06-03T01:21:04.894 INFO:tasks.workunit.client.0.smithi032.stdout:8/386: dwrite fa [0,4194304] 0 2024-06-03T01:21:04.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/578: unlink d0/d11/d79/cb6 0 2024-06-03T01:21:04.896 INFO:tasks.workunit.client.0.smithi032.stdout:3/461: symlink de/d29/d2e/d31/d24/l91 0 2024-06-03T01:21:04.896 INFO:tasks.workunit.client.0.smithi032.stdout:1/406: dwrite fa [4194304,4194304] 0 2024-06-03T01:21:04.898 INFO:tasks.workunit.client.0.smithi032.stdout:3/462: mknod de/d29/d2e/d31/d8d/c92 0 2024-06-03T01:21:04.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/407: truncate d18/f1a 3945534 0 2024-06-03T01:21:04.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/408: chown l13 13364703 1 2024-06-03T01:21:04.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/409: chown d18/d32/d39/d60/f72 6 1 2024-06-03T01:21:04.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/410: fdatasync d18/d2d/d5e/f64 0 2024-06-03T01:21:04.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/411: write d18/f34 [602518,23180] 0 2024-06-03T01:21:04.902 INFO:tasks.workunit.client.0.smithi032.stdout:7/540: dread dc/d12/d22/d42/f51 [0,4194304] 0 2024-06-03T01:21:04.902 INFO:tasks.workunit.client.0.smithi032.stdout:7/541: creat dc/d12/d2a/d2f/d76/fae x:0 0 0 2024-06-03T01:21:04.903 INFO:tasks.workunit.client.1.smithi044.stdout:3/978: dwrite df/d1e/d3a/fb6 [0,4194304] 0 2024-06-03T01:21:04.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/482: dread d5/da/d1a/f68 [0,4194304] 0 2024-06-03T01:21:04.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/483: creat d5/da/d1a/d69/d83/fa1 x:0 0 0 2024-06-03T01:21:04.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/484: readlink d5/d38/l8e 0 2024-06-03T01:21:04.905 INFO:tasks.workunit.client.0.smithi032.stdout:3/463: symlink de/d29/d2e/d31/d23/l93 0 2024-06-03T01:21:04.905 INFO:tasks.workunit.client.0.smithi032.stdout:3/464: readlink de/d29/d2e/d31/d3c/l7d 0 2024-06-03T01:21:04.905 INFO:tasks.workunit.client.0.smithi032.stdout:4/488: dwrite d4/db/da/f59 [0,4194304] 0 2024-06-03T01:21:04.907 INFO:tasks.workunit.client.1.smithi044.stdout:3/979: unlink df/d13/d39/d3d/d43/d71/ff3 0 2024-06-03T01:21:04.907 INFO:tasks.workunit.client.1.smithi044.stdout:3/980: chown df/d1e/d8a 235957 1 2024-06-03T01:21:04.907 INFO:tasks.workunit.client.1.smithi044.stdout:3/981: getdents df/d13/d39/d3d/d102 0 2024-06-03T01:21:04.908 INFO:tasks.workunit.client.1.smithi044.stdout:3/982: mkdir df/d13/d39/d3d/d50/d130/d149 0 2024-06-03T01:21:04.910 INFO:tasks.workunit.client.0.smithi032.stdout:6/432: dwrite d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:04.911 INFO:tasks.workunit.client.1.smithi044.stdout:3/983: rename df/d13/d1b/d56/d119 to df/d13/d1b/d56/d5c/d14a 0 2024-06-03T01:21:04.911 INFO:tasks.workunit.client.1.smithi044.stdout:3/984: write df/d1e/d3a/d84/f10e [341564,125484] 0 2024-06-03T01:21:04.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/535: truncate da/f2a 299598 0 2024-06-03T01:21:04.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/536: write da/dd/df/d10/d16/d18/f27 [5074776,101725] 0 2024-06-03T01:21:04.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/537: fdatasync da/f15 0 2024-06-03T01:21:04.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/538: creat da/dd/df/d10/d16/d18/d1a/d30/d51/fa2 x:0 0 0 2024-06-03T01:21:04.913 INFO:tasks.workunit.client.0.smithi032.stdout:5/539: chown da/dd/df/d10/d16/d18/d1a/d30/d51/d7a 12840601 1 2024-06-03T01:21:04.913 INFO:tasks.workunit.client.0.smithi032.stdout:5/540: chown da/dd/df/d10/d16/d18/d35/c9c 555518714 1 2024-06-03T01:21:04.913 INFO:tasks.workunit.client.1.smithi044.stdout:3/985: rmdir df/d13/d6d/d9a/dad/d125 39 2024-06-03T01:21:04.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/986: link df/d13/d6d/d9a/c9d df/d13/d1b/d56/d5c/d98/df4/c14b 0 2024-06-03T01:21:04.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/987: write df/f65 [4338269,75453] 0 2024-06-03T01:21:04.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/988: write df/f47 [4602599,99984] 0 2024-06-03T01:21:04.916 INFO:tasks.workunit.client.1.smithi044.stdout:3/989: write df/d13/d39/d3d/d43/d71/f83 [330882,83008] 0 2024-06-03T01:21:04.921 INFO:tasks.workunit.client.0.smithi032.stdout:7/542: write dc/d12/d2a/d2f/f39 [2043754,47403] 0 2024-06-03T01:21:04.923 INFO:tasks.workunit.client.0.smithi032.stdout:2/485: write d5/da/d1a/d2a/d36/d3f/f4f [1314302,80953] 0 2024-06-03T01:21:04.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/489: mkdir d4/db/da/dae 0 2024-06-03T01:21:04.925 INFO:tasks.workunit.client.0.smithi032.stdout:4/490: write d4/db/da/d37/f40 [4983076,60797] 0 2024-06-03T01:21:04.929 INFO:tasks.workunit.client.0.smithi032.stdout:5/541: write da/dd/df/d10/d16/d18/f50 [1210623,109619] 0 2024-06-03T01:21:04.936 INFO:tasks.workunit.client.0.smithi032.stdout:8/387: dwrite dc/f13 [0,4194304] 0 2024-06-03T01:21:04.939 INFO:tasks.workunit.client.0.smithi032.stdout:1/412: rename d18/d29 to d18/d3f/d5f/d6b/d78 0 2024-06-03T01:21:04.939 INFO:tasks.workunit.client.0.smithi032.stdout:1/413: chown d18/d3f/d5f/d6b/d78/d3d 24 1 2024-06-03T01:21:04.941 INFO:tasks.workunit.client.0.smithi032.stdout:6/433: creat d8/d11/d20/d2b/d39/f82 x:0 0 0 2024-06-03T01:21:04.943 INFO:tasks.workunit.client.0.smithi032.stdout:1/414: read d18/d27/d2b/f54 [2097694,30463] 0 2024-06-03T01:21:04.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/465: dwrite de/d29/d2e/d31/d78/f86 [4194304,4194304] 0 2024-06-03T01:21:04.948 INFO:tasks.workunit.client.0.smithi032.stdout:8/388: dread dc/d25/d28/d32/f36 [0,4194304] 0 2024-06-03T01:21:04.952 INFO:tasks.workunit.client.0.smithi032.stdout:2/486: dwrite d5/d38/d46/d5c/f8b [0,4194304] 0 2024-06-03T01:21:04.952 INFO:tasks.workunit.client.0.smithi032.stdout:2/487: chown d5/l2d 162083857 1 2024-06-03T01:21:04.966 INFO:tasks.workunit.client.0.smithi032.stdout:7/543: symlink dc/d12/d6a/d86/d88/d8b/laf 0 2024-06-03T01:21:04.966 INFO:tasks.workunit.client.0.smithi032.stdout:4/491: creat d4/db/da/d1d/faf x:0 0 0 2024-06-03T01:21:04.968 INFO:tasks.workunit.client.0.smithi032.stdout:5/542: mknod da/dd/df/d10/d16/d18/d1a/d30/d51/d9e/ca3 0 2024-06-03T01:21:04.968 INFO:tasks.workunit.client.0.smithi032.stdout:1/415: dwrite d18/f5d [0,4194304] 0 2024-06-03T01:21:04.977 INFO:tasks.workunit.client.0.smithi032.stdout:6/434: rename d8/le to d8/d11/d20/d7f/l83 0 2024-06-03T01:21:04.983 INFO:tasks.workunit.client.0.smithi032.stdout:8/389: truncate dc/d25/f55 165356 0 2024-06-03T01:21:04.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/466: symlink de/d29/d2e/d31/d3c/l94 0 2024-06-03T01:21:04.986 INFO:tasks.workunit.client.1.smithi044.stdout:3/990: sync 2024-06-03T01:21:04.987 INFO:tasks.workunit.client.1.smithi044.stdout:3/991: truncate df/d1e/d3a/d84/fc0 818985 0 2024-06-03T01:21:04.987 INFO:tasks.workunit.client.1.smithi044.stdout:3/992: fsync df/d13/f1d 0 2024-06-03T01:21:04.987 INFO:tasks.workunit.client.1.smithi044.stdout:3/993: write df/d13/d1b/d4b/d69/dd1/f115 [968634,58508] 0 2024-06-03T01:21:04.987 INFO:tasks.workunit.client.0.smithi032.stdout:2/488: dwrite d5/da/f74 [0,4194304] 0 2024-06-03T01:21:04.987 INFO:tasks.workunit.client.0.smithi032.stdout:2/489: chown d5/da/d1a/d2a/c9c 0 1 2024-06-03T01:21:04.989 INFO:tasks.workunit.client.1.smithi044.stdout:3/994: rmdir df/d1e/d2a/d111 0 2024-06-03T01:21:04.992 INFO:tasks.workunit.client.1.smithi044.stdout:3/995: symlink df/d1e/d78/dda/d12c/dbc/l14c 0 2024-06-03T01:21:04.994 INFO:tasks.workunit.client.1.smithi044.stdout:3/996: symlink df/d1e/d2a/d9b/l14d 0 2024-06-03T01:21:04.994 INFO:tasks.workunit.client.1.smithi044.stdout:3/997: stat df/d13/d1b/d56/ffb 0 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:7/544: rmdir dc/d12/d2a/d2f/d63 39 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:7/545: fsync dc/d12/d22/d42/d61/f95 0 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:7/546: fsync dc/d12/d1b/fa5 0 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:4/492: creat d4/db/da/d37/d90/d91/fb0 x:0 0 0 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:4/493: chown d4/db/da/d37/f40 6337811 1 2024-06-03T01:21:04.995 INFO:tasks.workunit.client.0.smithi032.stdout:4/494: write d4/d7f/fad [678603,77014] 0 2024-06-03T01:21:04.996 INFO:tasks.workunit.client.1.smithi044.stdout:3/998: mknod df/d13/d1b/d56/d5c/de3/c14e 0 2024-06-03T01:21:04.998 INFO:tasks.workunit.client.0.smithi032.stdout:5/543: dwrite da/dd/f68 [0,4194304] 0 2024-06-03T01:21:04.999 INFO:tasks.workunit.client.1.smithi044.stdout:3/999: unlink df/d13/d39/d3d/d43/d71/fa2 0 2024-06-03T01:21:05.001 INFO:tasks.workunit.client.0.smithi032.stdout:1/416: rmdir d18/d3f/d5f/d6b 39 2024-06-03T01:21:05.003 INFO:tasks.workunit.client.0.smithi032.stdout:2/490: write d5/f3c [1048308,9792] 0 2024-06-03T01:21:05.003 INFO:tasks.workunit.client.0.smithi032.stdout:2/491: write d5/d38/d46/d53/d57/d6d/f98 [7426,40960] 0 2024-06-03T01:21:05.004 INFO:tasks.workunit.client.1.smithi044.stderr:+ rm -rf -- ./tmp.57AOhJ9MAj 2024-06-03T01:21:05.006 INFO:tasks.workunit.client.0.smithi032.stdout:8/390: creat dc/d25/d28/d51/d5d/d5e/f84 x:0 0 0 2024-06-03T01:21:05.006 INFO:tasks.workunit.client.0.smithi032.stdout:8/391: fdatasync dc/d25/d28/d32/f50 0 2024-06-03T01:21:05.008 INFO:tasks.workunit.client.0.smithi032.stdout:7/547: rename dc/d12/d1b/c46 to dc/d12/d78/cb0 0 2024-06-03T01:21:05.008 INFO:tasks.workunit.client.0.smithi032.stdout:7/548: read dc/d12/d1b/f60 [203792,67501] 0 2024-06-03T01:21:05.010 INFO:tasks.workunit.client.0.smithi032.stdout:5/544: creat da/dd/df/d10/d16/d25/fa4 x:0 0 0 2024-06-03T01:21:05.012 INFO:tasks.workunit.client.0.smithi032.stdout:7/549: symlink dc/d12/d6a/d86/d89/lb1 0 2024-06-03T01:21:05.017 INFO:tasks.workunit.client.0.smithi032.stdout:5/545: mkdir da/dd/df/d10/d16/d18/d1a/d30/d58/da5 0 2024-06-03T01:21:05.022 INFO:tasks.workunit.client.0.smithi032.stdout:3/467: dwrite de/d29/f4b [0,4194304] 0 2024-06-03T01:21:05.022 INFO:tasks.workunit.client.0.smithi032.stdout:3/468: stat de/d29/d2e/d31/d3c/f44 0 2024-06-03T01:21:05.023 INFO:tasks.workunit.client.0.smithi032.stdout:3/469: creat de/d29/f95 x:0 0 0 2024-06-03T01:21:05.023 INFO:tasks.workunit.client.0.smithi032.stdout:1/417: dwrite d18/d3f/d5f/d6b/d78/f5c [0,4194304] 0 2024-06-03T01:21:05.023 INFO:tasks.workunit.client.0.smithi032.stdout:6/435: dwrite d8/d11/f35 [4194304,4194304] 0 2024-06-03T01:21:05.032 INFO:tasks.workunit.client.0.smithi032.stdout:6/436: unlink d8/d11/f49 0 2024-06-03T01:21:05.034 INFO:tasks.workunit.client.0.smithi032.stdout:3/470: rmdir de/d29/d2e/d31/d23 39 2024-06-03T01:21:05.034 INFO:tasks.workunit.client.0.smithi032.stdout:3/471: write de/d29/d2e/d31/d3c/d50/d56/f5c [88799,49938] 0 2024-06-03T01:21:05.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/392: dwrite dc/d25/f35 [0,4194304] 0 2024-06-03T01:21:05.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/393: stat dc/d25/d28/d3c/f5a 0 2024-06-03T01:21:05.036 INFO:tasks.workunit.client.0.smithi032.stdout:6/437: mkdir d8/d11/d20/d3b/d84 0 2024-06-03T01:21:05.036 INFO:tasks.workunit.client.0.smithi032.stdout:6/438: chown d8/d11/d20/d31 26290827 1 2024-06-03T01:21:05.037 INFO:tasks.workunit.client.0.smithi032.stdout:1/418: write d18/d32/f52 [8777873,109572] 0 2024-06-03T01:21:05.037 INFO:tasks.workunit.client.0.smithi032.stdout:3/472: creat de/d29/d2e/d31/f96 x:0 0 0 2024-06-03T01:21:05.038 INFO:tasks.workunit.client.0.smithi032.stdout:1/419: fdatasync d18/d3f/d5f/d6b/d78/f45 0 2024-06-03T01:21:05.038 INFO:tasks.workunit.client.0.smithi032.stdout:8/394: mknod dc/d25/d28/d3c/c85 0 2024-06-03T01:21:05.038 INFO:tasks.workunit.client.0.smithi032.stdout:8/395: creat dc/d25/d28/d58/f86 x:0 0 0 2024-06-03T01:21:05.040 INFO:tasks.workunit.client.0.smithi032.stdout:3/473: creat de/d29/d2e/d31/d23/d53/d69/d8c/f97 x:0 0 0 2024-06-03T01:21:05.045 INFO:tasks.workunit.client.0.smithi032.stdout:1/420: symlink d18/d27/l79 0 2024-06-03T01:21:05.046 INFO:tasks.workunit.client.0.smithi032.stdout:5/546: dwrite da/dd/df/d10/d16/d25/f66 [0,4194304] 0 2024-06-03T01:21:05.046 INFO:tasks.workunit.client.0.smithi032.stdout:5/547: fsync da/dd/df/d10/d16/d18/f27 0 2024-06-03T01:21:05.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/439: write d8/d11/d20/d2b/d39/d19/f4d [3555312,34871] 0 2024-06-03T01:21:05.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/440: dread - d8/d11/d20/d7f/f78 zero size 2024-06-03T01:21:05.049 INFO:tasks.workunit.client.0.smithi032.stdout:8/396: symlink dc/d25/d28/d51/d5d/l87 0 2024-06-03T01:21:05.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/474: mknod de/d29/d2e/d31/d23/c98 0 2024-06-03T01:21:05.054 INFO:tasks.workunit.client.0.smithi032.stdout:1/421: mkdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a 0 2024-06-03T01:21:05.056 INFO:tasks.workunit.client.0.smithi032.stdout:5/548: write da/dd/f68 [3055095,23141] 0 2024-06-03T01:21:05.056 INFO:tasks.workunit.client.0.smithi032.stdout:5/549: write da/f17 [437675,101520] 0 2024-06-03T01:21:05.057 INFO:tasks.workunit.client.0.smithi032.stdout:6/441: rename d8/d11/d20/d3b/d66/l7b to d8/d11/d20/d21/l85 0 2024-06-03T01:21:05.059 INFO:tasks.workunit.client.0.smithi032.stdout:3/475: symlink de/d29/d2e/d31/d24/l99 0 2024-06-03T01:21:05.059 INFO:tasks.workunit.client.0.smithi032.stdout:3/476: write de/d29/d2e/d31/d24/f25 [2230063,10493] 0 2024-06-03T01:21:05.060 INFO:tasks.workunit.client.0.smithi032.stdout:8/397: write dc/de/d1e/f2e [262871,28818] 0 2024-06-03T01:21:05.061 INFO:tasks.workunit.client.0.smithi032.stdout:1/422: mkdir d18/d3f/d5f/d7b 0 2024-06-03T01:21:05.061 INFO:tasks.workunit.client.0.smithi032.stdout:1/423: write d18/f34 [788703,54922] 0 2024-06-03T01:21:05.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/550: mknod da/dd/df/d10/d16/d18/d1a/d30/d51/d9e/ca6 0 2024-06-03T01:21:05.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/551: dread - da/dd/df/d10/d16/d18/d35/f98 zero size 2024-06-03T01:21:05.066 INFO:tasks.workunit.client.0.smithi032.stdout:5/552: write da/f2a [11464,1715] 0 2024-06-03T01:21:05.066 INFO:tasks.workunit.client.0.smithi032.stdout:5/553: write da/dd/df/f33 [4682219,17082] 0 2024-06-03T01:21:05.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/477: rename de/d29/f52 to de/d29/d2e/d31/d3c/d50/d66/f9a 0 2024-06-03T01:21:05.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/478: chown de/d29/d2e/l6f 76 1 2024-06-03T01:21:05.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/479: fdatasync de/d29/f95 0 2024-06-03T01:21:05.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/480: write de/d29/d2e/d31/f32 [5065147,7163] 0 2024-06-03T01:21:05.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/398: dwrite dc/d14/f22 [0,4194304] 0 2024-06-03T01:21:05.081 INFO:tasks.workunit.client.0.smithi032.stdout:1/424: getdents d18/d3f/d5f/d6b 0 2024-06-03T01:21:05.083 INFO:tasks.workunit.client.0.smithi032.stdout:6/442: link d8/d11/d20/d2b/d39/f1f d8/d55/f86 0 2024-06-03T01:21:05.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:04 smithi032 ceph-mon[29360]: pgmap v7: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 249 MiB/s rd, 296 MiB/s wr, 418 op/s 2024-06-03T01:21:05.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:04 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2843681611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:21:05.088 INFO:tasks.workunit.client.0.smithi032.stdout:5/554: symlink da/dd/df/d10/d16/d25/d74/la7 0 2024-06-03T01:21:05.088 INFO:tasks.workunit.client.0.smithi032.stdout:5/555: write da/dd/df/d10/f14 [1151907,87394] 0 2024-06-03T01:21:05.095 INFO:tasks.workunit.client.0.smithi032.stdout:8/399: dwrite dc/d25/d28/d33/f48 [0,4194304] 0 2024-06-03T01:21:05.099 INFO:tasks.workunit.client.0.smithi032.stdout:1/425: symlink d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a/l7c 0 2024-06-03T01:21:05.099 INFO:tasks.workunit.client.0.smithi032.stdout:1/426: dread - d18/d3f/d5f/d6b/d78/d3d/d47/f77 zero size 2024-06-03T01:21:05.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/443: mknod d8/d11/d20/d31/d3f/c87 0 2024-06-03T01:21:05.101 INFO:tasks.workunit.client.0.smithi032.stdout:6/444: dread - d8/d50/f62 zero size 2024-06-03T01:21:05.102 INFO:tasks.workunit.client.0.smithi032.stdout:5/556: mkdir da/dd/df/d10/d16/d18/d1a/d30/d58/da8 0 2024-06-03T01:21:05.105 INFO:tasks.workunit.client.0.smithi032.stdout:8/400: link dc/de/f37 dc/d25/f88 0 2024-06-03T01:21:05.106 INFO:tasks.workunit.client.0.smithi032.stdout:1/427: mknod d18/d3f/d5f/d6b/d78/d3d/c7d 0 2024-06-03T01:21:05.106 INFO:tasks.workunit.client.0.smithi032.stdout:1/428: write d18/d3f/d5f/d6b/d78/d3d/d47/f51 [1100349,125298] 0 2024-06-03T01:21:05.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/557: rename da/dd/df/d10/d16/d18/d35/d44/f6f to da/dd/df/d10/d16/fa9 0 2024-06-03T01:21:05.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/558: dread - da/dd/df/d10/d16/d25/fa4 zero size 2024-06-03T01:21:05.115 INFO:tasks.workunit.client.0.smithi032.stdout:1/429: dread d18/d32/d39/f4a [0,4194304] 0 2024-06-03T01:21:05.115 INFO:tasks.workunit.client.0.smithi032.stdout:1/430: write d18/f19 [3993353,103633] 0 2024-06-03T01:21:05.116 INFO:tasks.workunit.client.0.smithi032.stdout:1/431: chown d18/l1f 214469 1 2024-06-03T01:21:05.116 INFO:tasks.workunit.client.0.smithi032.stdout:8/401: dread dc/d25/d28/d58/f79 [0,4194304] 0 2024-06-03T01:21:05.117 INFO:tasks.workunit.client.0.smithi032.stdout:6/445: fsync d8/d11/d20/d2b/d39/f1f 0 2024-06-03T01:21:05.125 INFO:tasks.workunit.client.0.smithi032.stdout:5/559: dwrite da/dd/df/d10/d16/d18/f27 [0,4194304] 0 2024-06-03T01:21:05.127 INFO:tasks.workunit.client.0.smithi032.stdout:8/402: unlink dc/d14/c1d 0 2024-06-03T01:21:05.141 INFO:tasks.workunit.client.0.smithi032.stdout:5/560: dread da/f15 [4194304,4194304] 0 2024-06-03T01:21:05.142 INFO:tasks.workunit.client.0.smithi032.stdout:5/561: creat da/dd/df/d10/d16/d18/faa x:0 0 0 2024-06-03T01:21:05.143 INFO:tasks.workunit.client.0.smithi032.stdout:5/562: mkdir da/dab 0 2024-06-03T01:21:05.143 INFO:tasks.workunit.client.0.smithi032.stdout:1/432: dwrite d18/f1c [0,4194304] 0 2024-06-03T01:21:05.144 INFO:tasks.workunit.client.0.smithi032.stdout:8/403: dwrite dc/d25/d28/d3c/f77 [0,4194304] 0 2024-06-03T01:21:05.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/446: dwrite d8/d55/f86 [0,4194304] 0 2024-06-03T01:21:05.145 INFO:tasks.workunit.client.0.smithi032.stdout:5/563: rename da/dd/df/d10/l1f to da/dd/df/lac 0 2024-06-03T01:21:05.145 INFO:tasks.workunit.client.0.smithi032.stdout:5/564: fsync da/dd/df/d10/d16/f8a 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/433: mknod d18/d3f/d5f/c7e 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/434: creat d18/d27/d2b/f7f x:0 0 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/435: fdatasync fa 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/436: creat d18/f80 x:0 0 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/437: write d18/d3f/f55 [4765308,128147] 0 2024-06-03T01:21:05.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/438: stat d18/c20 0 2024-06-03T01:21:05.149 INFO:tasks.workunit.client.0.smithi032.stdout:1/439: creat d18/d3f/d5f/d6b/f81 x:0 0 0 2024-06-03T01:21:05.149 INFO:tasks.workunit.client.0.smithi032.stdout:1/440: write d18/d3f/d5f/d6b/d78/f45 [1314764,125453] 0 2024-06-03T01:21:05.150 INFO:tasks.workunit.client.0.smithi032.stdout:1/441: write d18/d3f/d5f/d6b/d78/d3d/d47/f51 [848551,50908] 0 2024-06-03T01:21:05.155 INFO:tasks.workunit.client.0.smithi032.stdout:1/442: dread d18/d2d/f50 [0,4194304] 0 2024-06-03T01:21:05.155 INFO:tasks.workunit.client.0.smithi032.stdout:1/443: write d18/f19 [3904410,70350] 0 2024-06-03T01:21:05.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/447: write d8/d11/d20/d2b/d39/d19/f40 [1853281,86410] 0 2024-06-03T01:21:05.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/448: mknod d8/d11/c88 0 2024-06-03T01:21:05.164 INFO:tasks.workunit.client.0.smithi032.stdout:8/404: write dc/de/f37 [1519626,31099] 0 2024-06-03T01:21:05.164 INFO:tasks.workunit.client.0.smithi032.stdout:5/565: dread da/dd/df/d10/d16/d18/d35/d44/f4f [0,4194304] 0 2024-06-03T01:21:05.168 INFO:tasks.workunit.client.0.smithi032.stdout:8/405: unlink dc/d14/f24 0 2024-06-03T01:21:05.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/406: dread dc/d25/d28/d3c/f5a [0,4194304] 0 2024-06-03T01:21:05.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/407: chown dc/d25/d28/c68 189676380 1 2024-06-03T01:21:05.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/408: chown dc/d25/d28/d51/d5d 463914 1 2024-06-03T01:21:05.173 INFO:tasks.workunit.client.0.smithi032.stdout:6/449: dread d8/d11/d20/d3b/f41 [0,4194304] 0 2024-06-03T01:21:05.173 INFO:tasks.workunit.client.0.smithi032.stdout:6/450: write d8/d11/d20/d31/d3f/f69 [50001,3380] 0 2024-06-03T01:21:05.175 INFO:tasks.workunit.client.0.smithi032.stdout:6/451: mkdir d8/d11/d20/d89 0 2024-06-03T01:21:05.176 INFO:tasks.workunit.client.0.smithi032.stdout:6/452: rename d8/d11/d20/d7f/l83 to d8/d11/d20/d2b/d39/d7e/l8a 0 2024-06-03T01:21:05.179 INFO:tasks.workunit.client.0.smithi032.stdout:6/453: rename d8/d11/d20/c2a to d8/d50/c8b 0 2024-06-03T01:21:05.181 INFO:tasks.workunit.client.0.smithi032.stdout:5/566: dwrite da/dd/df/d10/d16/fa9 [0,4194304] 0 2024-06-03T01:21:05.187 INFO:tasks.workunit.client.0.smithi032.stdout:6/454: dread d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:05.187 INFO:tasks.workunit.client.0.smithi032.stdout:6/455: write d8/d11/d20/d31/d3f/f69 [277402,41923] 0 2024-06-03T01:21:05.187 INFO:tasks.workunit.client.0.smithi032.stdout:6/456: chown d8/d11/d20/d7f/f78 21881 1 2024-06-03T01:21:05.188 INFO:tasks.workunit.client.0.smithi032.stdout:5/567: symlink da/dd/df/d10/d16/d18/d35/d70/lad 0 2024-06-03T01:21:05.188 INFO:tasks.workunit.client.0.smithi032.stdout:5/568: truncate da/dd/df/d10/d16/d18/d35/d70/f95 425044 0 2024-06-03T01:21:05.191 INFO:tasks.workunit.client.0.smithi032.stdout:6/457: link d8/d11/d20/d31/f45 d8/d11/d20/f8c 0 2024-06-03T01:21:05.191 INFO:tasks.workunit.client.0.smithi032.stdout:6/458: truncate d8/d11/d20/d2b/d39/d19/f4d 4229051 0 2024-06-03T01:21:05.198 INFO:tasks.workunit.client.0.smithi032.stdout:5/569: write f4 [6571913,19173] 0 2024-06-03T01:21:05.198 INFO:tasks.workunit.client.0.smithi032.stdout:5/570: dread - da/dd/df/f9d zero size 2024-06-03T01:21:05.207 INFO:tasks.workunit.client.0.smithi032.stdout:6/459: link d8/c4b d8/d11/d20/d31/c8d 0 2024-06-03T01:21:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:04 smithi044 ceph-mon[36956]: pgmap v7: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 249 MiB/s rd, 296 MiB/s wr, 418 op/s 2024-06-03T01:21:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:04 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2843681611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:21:05.224 INFO:tasks.workunit.client.0.smithi032.stdout:5/571: dwrite da/dd/df/d10/d16/d18/d1a/d30/d51/f57 [0,4194304] 0 2024-06-03T01:21:05.250 INFO:tasks.workunit.client.0.smithi032.stdout:0/490: sync 2024-06-03T01:21:05.250 INFO:tasks.workunit.client.0.smithi032.stdout:0/491: chown d1/d46/f48 1279 1 2024-06-03T01:21:05.250 INFO:tasks.workunit.client.0.smithi032.stdout:0/492: dread - d1/d46/f98 zero size 2024-06-03T01:21:05.280 INFO:tasks.workunit.client.0.smithi032.stdout:6/460: mkdir d8/d11/d20/d2b/d39/d7e/d8e 0 2024-06-03T01:21:05.295 INFO:tasks.workunit.client.0.smithi032.stdout:6/461: unlink d8/fd 0 2024-06-03T01:21:05.296 INFO:tasks.workunit.client.0.smithi032.stdout:0/493: getdents d1/da/d28/d3b/d68/d83 0 2024-06-03T01:21:05.298 INFO:tasks.workunit.client.0.smithi032.stdout:6/462: rmdir d8/d11/d20/d2b/d39/d7e 39 2024-06-03T01:21:05.299 INFO:tasks.workunit.client.0.smithi032.stdout:0/494: link d1/da/d10/d29/f73 d1/da/d28/d3b/d68/fa3 0 2024-06-03T01:21:05.300 INFO:tasks.workunit.client.0.smithi032.stdout:0/495: stat d1/f6 0 2024-06-03T01:21:05.301 INFO:tasks.workunit.client.0.smithi032.stdout:6/463: rmdir d8/d11/d20/d2b/d39/d7e/d8e 0 2024-06-03T01:21:05.302 INFO:tasks.workunit.client.0.smithi032.stdout:0/496: rename d1/da/d10/d53/f6e to d1/da/d10/d53/fa4 0 2024-06-03T01:21:05.304 INFO:tasks.workunit.client.0.smithi032.stdout:6/464: mkdir d8/d5f/d8f 0 2024-06-03T01:21:05.306 INFO:tasks.workunit.client.0.smithi032.stdout:0/497: unlink d1/da/d10/d80/c84 0 2024-06-03T01:21:05.306 INFO:tasks.workunit.client.0.smithi032.stdout:0/498: write d1/da/d10/d53/fa4 [664283,75695] 0 2024-06-03T01:21:05.310 INFO:tasks.workunit.client.0.smithi032.stdout:6/465: rename d8/d4c/c74 to d8/d11/d20/d36/c90 0 2024-06-03T01:21:05.314 INFO:tasks.workunit.client.0.smithi032.stdout:6/466: rmdir d8/d5f/d8f 0 2024-06-03T01:21:05.314 INFO:tasks.workunit.client.0.smithi032.stdout:6/467: write d8/d50/f62 [974480,44998] 0 2024-06-03T01:21:05.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/499: rename d1/da/d10/f2a to d1/da/d28/d3b/d5d/fa5 0 2024-06-03T01:21:05.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/500: fsync d1/da/d10/d53/fa4 0 2024-06-03T01:21:05.348 INFO:tasks.workunit.client.0.smithi032.stdout:0/501: rename d1/da/d10/c2c to d1/da/d28/ca6 0 2024-06-03T01:21:05.349 INFO:tasks.workunit.client.0.smithi032.stdout:0/502: fsync d1/da/d10/f6f 0 2024-06-03T01:21:05.349 INFO:tasks.workunit.client.0.smithi032.stdout:0/503: dread - d1/da/d28/d3b/d5d/d8d/d9c/fa0 zero size 2024-06-03T01:21:05.349 INFO:tasks.workunit.client.0.smithi032.stdout:0/504: write d1/f1c [273798,41654] 0 2024-06-03T01:21:05.349 INFO:tasks.workunit.client.0.smithi032.stdout:0/505: readlink d1/da/d10/l22 0 2024-06-03T01:21:05.353 INFO:tasks.workunit.client.0.smithi032.stdout:5/572: symlink da/dd/df/d10/d16/d18/d1a/d30/lae 0 2024-06-03T01:21:05.354 INFO:tasks.workunit.client.0.smithi032.stdout:5/573: unlink da/dd/f48 0 2024-06-03T01:21:05.384 INFO:tasks.workunit.client.0.smithi032.stdout:5/574: link da/dd/df/d10/l13 da/dd/df/d10/d16/d18/d1a/d30/d58/da8/laf 0 2024-06-03T01:21:05.384 INFO:tasks.workunit.client.0.smithi032.stdout:5/575: fsync da/f73 0 2024-06-03T01:21:05.386 INFO:tasks.workunit.client.0.smithi032.stdout:0/506: dwrite d1/f43 [0,4194304] 0 2024-06-03T01:21:05.396 INFO:tasks.workunit.client.0.smithi032.stdout:0/507: dread d1/f35 [0,4194304] 0 2024-06-03T01:21:05.397 INFO:tasks.workunit.client.0.smithi032.stdout:0/508: link d1/da/c23 d1/da/d28/d3b/d68/d83/ca7 0 2024-06-03T01:21:05.397 INFO:tasks.workunit.client.0.smithi032.stdout:0/509: chown d1/da/f9e 386 1 2024-06-03T01:21:05.399 INFO:tasks.workunit.client.0.smithi032.stdout:5/576: dwrite da/dd/df/d10/d16/d18/d1a/d30/f94 [0,4194304] 0 2024-06-03T01:21:05.401 INFO:tasks.workunit.client.0.smithi032.stdout:5/577: write da/dd/df/d10/d16/d18/d35/d70/f95 [417417,111230] 0 2024-06-03T01:21:05.405 INFO:tasks.workunit.client.0.smithi032.stdout:5/578: creat da/dd/df/d10/d16/d18/d1a/fb0 x:0 0 0 2024-06-03T01:21:05.405 INFO:tasks.workunit.client.0.smithi032.stdout:5/579: creat da/dd/df/d10/d16/d18/d1a/d30/d51/fb1 x:0 0 0 2024-06-03T01:21:05.409 INFO:tasks.workunit.client.0.smithi032.stdout:5/580: creat da/dd/df/d10/d16/d18/d1a/d30/d58/fb2 x:0 0 0 2024-06-03T01:21:05.409 INFO:tasks.workunit.client.0.smithi032.stdout:5/581: write da/dd/df/d10/d16/d25/d74/f88 [297428,110832] 0 2024-06-03T01:21:05.410 INFO:tasks.workunit.client.0.smithi032.stdout:5/582: write da/dd/df/d10/d16/d18/faa [985132,117785] 0 2024-06-03T01:21:05.410 INFO:tasks.workunit.client.0.smithi032.stdout:5/583: truncate da/dd/df/d10/d16/d18/d1a/fb0 983178 0 2024-06-03T01:21:05.412 INFO:tasks.workunit.client.0.smithi032.stdout:5/584: rename da/dd/df/d10/d16/fa1 to da/dd/df/d10/d16/d18/d35/d70/fb3 0 2024-06-03T01:21:05.414 INFO:tasks.workunit.client.0.smithi032.stdout:0/510: dwrite d1/da/d10/d29/f96 [0,4194304] 0 2024-06-03T01:21:05.414 INFO:tasks.workunit.client.0.smithi032.stdout:5/585: symlink da/dd/df/lb4 0 2024-06-03T01:21:05.414 INFO:tasks.workunit.client.0.smithi032.stdout:5/586: stat da/dd/df/d10/d16/d18/d1a/d30/d58/fb2 0 2024-06-03T01:21:05.442 INFO:tasks.workunit.client.0.smithi032.stdout:0/511: dwrite d1/da/d10/d29/f96 [0,4194304] 0 2024-06-03T01:21:05.442 INFO:tasks.workunit.client.0.smithi032.stdout:0/512: stat d1/da/d10/d80/d9f 0 2024-06-03T01:21:05.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/587: dwrite da/dd/df/d10/d16/d18/d35/d44/f8f [0,4194304] 0 2024-06-03T01:21:05.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/588: creat da/dd/df/d10/d16/d18/d1a/d30/d58/fb5 x:0 0 0 2024-06-03T01:21:05.443 INFO:tasks.workunit.client.0.smithi032.stdout:0/513: link d1/da/d28/d3b/d5d/c77 d1/da/d28/d3b/ca8 0 2024-06-03T01:21:05.448 INFO:tasks.workunit.client.0.smithi032.stdout:0/514: mkdir d1/da/d28/d3b/d5d/da9 0 2024-06-03T01:21:05.448 INFO:tasks.workunit.client.0.smithi032.stdout:0/515: truncate d1/d46/d3a/f6d 461313 0 2024-06-03T01:21:05.448 INFO:tasks.workunit.client.0.smithi032.stdout:0/516: fsync d1/da/f8f 0 2024-06-03T01:21:05.450 INFO:tasks.workunit.client.0.smithi032.stdout:0/517: unlink d1/da/d28/d3b/d68/f85 0 2024-06-03T01:21:05.450 INFO:tasks.workunit.client.0.smithi032.stdout:0/518: chown d1/d46/d3a/f6d 256 1 2024-06-03T01:21:05.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/519: mkdir d1/da/d28/d3b/d5d/da9/daa 0 2024-06-03T01:21:05.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/520: write d1/da/d10/f5b [1074191,11781] 0 2024-06-03T01:21:05.454 INFO:tasks.workunit.client.0.smithi032.stdout:0/521: unlink d1/da/d10/l20 0 2024-06-03T01:21:05.454 INFO:tasks.workunit.client.0.smithi032.stdout:0/522: chown d1/da/c5a 12 1 2024-06-03T01:21:05.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/523: fsync d1/da/d10/f5b 0 2024-06-03T01:21:05.459 INFO:tasks.workunit.client.0.smithi032.stdout:5/589: dwrite da/f12 [0,4194304] 0 2024-06-03T01:21:05.463 INFO:tasks.workunit.client.0.smithi032.stdout:5/590: symlink da/dd/df/d10/d16/d18/d1a/d30/d51/d9e/lb6 0 2024-06-03T01:21:05.465 INFO:tasks.workunit.client.0.smithi032.stdout:0/524: write d1/f35 [2112096,30181] 0 2024-06-03T01:21:05.467 INFO:tasks.workunit.client.0.smithi032.stdout:0/525: mknod d1/da/d28/cab 0 2024-06-03T01:21:05.485 INFO:tasks.workunit.client.0.smithi032.stdout:0/526: dwrite d1/da/d10/f4f [0,4194304] 0 2024-06-03T01:21:05.485 INFO:tasks.workunit.client.0.smithi032.stdout:5/591: dwrite da/dd/df/d10/d16/d18/d35/f5d [0,4194304] 0 2024-06-03T01:21:05.487 INFO:tasks.workunit.client.0.smithi032.stdout:0/527: mkdir d1/da/d10/d80/d9f/dac 0 2024-06-03T01:21:05.489 INFO:tasks.workunit.client.0.smithi032.stdout:0/528: link d1/da/d10/d29/l7d d1/da/d28/d3b/d5d/da9/daa/lad 0 2024-06-03T01:21:05.489 INFO:tasks.workunit.client.0.smithi032.stdout:0/529: stat d1/l7 0 2024-06-03T01:21:05.489 INFO:tasks.workunit.client.0.smithi032.stdout:0/530: write d1/f17 [87810,74435] 0 2024-06-03T01:21:05.492 INFO:tasks.workunit.client.0.smithi032.stdout:0/531: rename d1/f1c to d1/da/d10/d53/fae 0 2024-06-03T01:21:05.505 INFO:tasks.workunit.client.0.smithi032.stdout:5/592: dwrite da/dd/f92 [0,4194304] 0 2024-06-03T01:21:05.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/593: getdents da/dd/df/d10 0 2024-06-03T01:21:05.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/594: chown da/dd/df/d10/d16/d25/d74/la7 427199280 1 2024-06-03T01:21:05.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/595: creat da/dd/df/d10/d16/d18/d35/d44/fb7 x:0 0 0 2024-06-03T01:21:05.508 INFO:tasks.workunit.client.0.smithi032.stdout:5/596: mknod da/dd/df/d10/d16/d18/d1a/d30/d51/cb8 0 2024-06-03T01:21:05.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/597: mkdir da/dd/df/d10/d16/d18/db9 0 2024-06-03T01:21:05.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/598: creat da/dd/df/d10/d16/d18/d1a/d30/d51/fba x:0 0 0 2024-06-03T01:21:05.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/599: chown da/dd/df/f5c 1745180729 1 2024-06-03T01:21:05.512 INFO:tasks.workunit.client.0.smithi032.stdout:5/600: symlink da/dd/df/d10/d16/d18/d1a/d30/d51/lbb 0 2024-06-03T01:21:05.512 INFO:tasks.workunit.client.0.smithi032.stdout:5/601: chown da/dd/df/d10/d16/l2e 307 1 2024-06-03T01:21:05.530 INFO:tasks.workunit.client.0.smithi032.stdout:9/579: sync 2024-06-03T01:21:05.534 INFO:tasks.workunit.client.0.smithi032.stdout:5/602: dwrite da/dd/df/d10/d16/d18/d1a/d30/d51/f57 [4194304,4194304] 0 2024-06-03T01:21:05.573 INFO:tasks.workunit.client.0.smithi032.stdout:9/580: dwrite d0/d23/d25/f8d [0,4194304] 0 2024-06-03T01:21:05.573 INFO:tasks.workunit.client.0.smithi032.stdout:9/581: truncate d0/d23/d24/f55 260115 0 2024-06-03T01:21:05.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/582: link d0/fe d0/d11/d1d/d38/d4f/db0/d41/fc0 0 2024-06-03T01:21:05.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/583: fsync d0/d11/d1d/d38/d4f/f60 0 2024-06-03T01:21:05.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/584: write d0/f6 [854731,58302] 0 2024-06-03T01:21:05.576 INFO:tasks.workunit.client.0.smithi032.stdout:5/603: rename da/dd/df/d10/d16 to da/dd/df/d10/dbc 0 2024-06-03T01:21:05.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/604: symlink da/dd/df/d10/dbc/d18/d1a/d30/d51/d9e/lbd 0 2024-06-03T01:21:05.579 INFO:tasks.workunit.client.0.smithi032.stdout:5/605: creat da/dd/df/d10/dbc/d18/d1a/d30/d58/da5/fbe x:0 0 0 2024-06-03T01:21:05.579 INFO:tasks.workunit.client.0.smithi032.stdout:5/606: write da/dd/df/d10/dbc/d18/d35/f98 [596204,93573] 0 2024-06-03T01:21:05.579 INFO:tasks.workunit.client.0.smithi032.stdout:5/607: write da/dd/df/d10/dbc/d25/f56 [888512,64894] 0 2024-06-03T01:21:05.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/608: link da/dd/df/d10/dbc/d18/d1a/d30/d51/f57 da/dd/df/fbf 0 2024-06-03T01:21:05.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/609: read da/f3e [721960,80140] 0 2024-06-03T01:21:05.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/610: write da/dd/df/d10/dbc/d18/f69 [1519878,98790] 0 2024-06-03T01:21:05.582 INFO:tasks.workunit.client.0.smithi032.stdout:9/585: dread d0/d11/d1d/da6/fa2 [0,4194304] 0 2024-06-03T01:21:05.583 INFO:tasks.workunit.client.0.smithi032.stdout:5/611: rename da/dd/df/d10/dbc/d18/d1a/d30/l43 to da/dd/df/d10/dbc/d18/d35/d70/lc0 0 2024-06-03T01:21:05.583 INFO:tasks.workunit.client.0.smithi032.stdout:9/586: mkdir d0/da4/dc1 0 2024-06-03T01:21:05.583 INFO:tasks.workunit.client.0.smithi032.stdout:9/587: fdatasync d0/d11/d1d/d38/d4f/f60 0 2024-06-03T01:21:05.584 INFO:tasks.workunit.client.0.smithi032.stdout:5/612: creat da/dd/df/d10/dbc/d18/fc1 x:0 0 0 2024-06-03T01:21:05.584 INFO:tasks.workunit.client.0.smithi032.stdout:5/613: truncate da/dd/df/d10/f4d 1994164 0 2024-06-03T01:21:05.585 INFO:tasks.workunit.client.0.smithi032.stdout:5/614: mknod da/dd/df/cc2 0 2024-06-03T01:21:05.585 INFO:tasks.workunit.client.0.smithi032.stdout:5/615: readlink da/dd/df/d10/dbc/d18/l76 0 2024-06-03T01:21:05.591 INFO:tasks.workunit.client.0.smithi032.stdout:5/616: rename da/dd/df/d10/dbc/d18/d1a/d30/d58 to da/dd/df/d10/dbc/d18/d35/d44/dc3 0 2024-06-03T01:21:05.592 INFO:tasks.workunit.client.0.smithi032.stdout:9/588: write d0/d11/d1d/d38/d4f/db0/d41/fc0 [6536215,61747] 0 2024-06-03T01:21:05.592 INFO:tasks.workunit.client.0.smithi032.stdout:9/589: chown d0/f57 5802 1 2024-06-03T01:21:05.592 INFO:tasks.workunit.client.0.smithi032.stdout:9/590: dread - d0/d11/d1d/fbc zero size 2024-06-03T01:21:05.594 INFO:tasks.workunit.client.0.smithi032.stdout:9/591: symlink d0/da4/lc2 0 2024-06-03T01:21:05.594 INFO:tasks.workunit.client.0.smithi032.stdout:9/592: write d0/f57 [2319742,126902] 0 2024-06-03T01:21:05.595 INFO:tasks.workunit.client.0.smithi032.stdout:9/593: creat d0/d11/d1d/d38/d4f/fc3 x:0 0 0 2024-06-03T01:21:05.596 INFO:tasks.workunit.client.0.smithi032.stdout:9/594: mknod d0/d11/d9c/cc4 0 2024-06-03T01:21:05.602 INFO:tasks.workunit.client.0.smithi032.stdout:5/617: read da/dd/df/d10/f99 [782928,19881] 0 2024-06-03T01:21:05.602 INFO:tasks.workunit.client.0.smithi032.stdout:5/618: stat da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fbe 0 2024-06-03T01:21:05.603 INFO:tasks.workunit.client.0.smithi032.stdout:5/619: symlink da/dd/df/d10/dbc/lc4 0 2024-06-03T01:21:05.605 INFO:tasks.workunit.client.0.smithi032.stdout:5/620: stat da/dd/df/d10/dbc/d18/d35/d44/dc3/da8/laf 0 2024-06-03T01:21:05.617 INFO:tasks.workunit.client.0.smithi032.stdout:9/595: dwrite d0/d11/d1d/d38/d4f/db0/fa0 [0,4194304] 0 2024-06-03T01:21:05.617 INFO:tasks.workunit.client.0.smithi032.stdout:9/596: dread - d0/d23/fb5 zero size 2024-06-03T01:21:05.621 INFO:tasks.workunit.client.0.smithi032.stdout:5/621: dwrite da/dd/df/d10/f39 [0,4194304] 0 2024-06-03T01:21:05.624 INFO:tasks.workunit.client.0.smithi032.stdout:9/597: dread d0/d11/d1d/f99 [0,4194304] 0 2024-06-03T01:21:05.624 INFO:tasks.workunit.client.0.smithi032.stdout:9/598: chown d0/d23/lbf 196495520 1 2024-06-03T01:21:05.625 INFO:tasks.workunit.client.0.smithi032.stdout:9/599: link d0/d11/d1d/d38/d4f/f7a d0/d11/d1d/d38/d4f/db0/d41/fc5 0 2024-06-03T01:21:05.625 INFO:tasks.workunit.client.0.smithi032.stdout:9/600: write d0/f19 [4463488,78253] 0 2024-06-03T01:21:05.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/601: readlink d0/d23/d25/l30 0 2024-06-03T01:21:05.627 INFO:tasks.workunit.client.0.smithi032.stdout:5/622: dread da/dd/df/d10/dbc/d18/d35/d44/f4f [0,4194304] 0 2024-06-03T01:21:05.628 INFO:tasks.workunit.client.0.smithi032.stdout:9/602: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6 0 2024-06-03T01:21:05.630 INFO:tasks.workunit.client.0.smithi032.stdout:5/623: creat da/dd/df/d10/dbc/d18/d35/d44/fc5 x:0 0 0 2024-06-03T01:21:05.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/603: creat d0/d11/d1d/da6/d71/fc7 x:0 0 0 2024-06-03T01:21:05.633 INFO:tasks.workunit.client.0.smithi032.stdout:5/624: symlink da/dd/df/d10/dbc/d18/d35/d44/lc6 0 2024-06-03T01:21:05.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/625: truncate da/dd/df/d10/dbc/d18/d35/d70/fb3 706984 0 2024-06-03T01:21:05.635 INFO:tasks.workunit.client.0.smithi032.stdout:9/604: unlink d0/fa 0 2024-06-03T01:21:05.635 INFO:tasks.workunit.client.0.smithi032.stdout:9/605: readlink d0/l7 0 2024-06-03T01:21:05.637 INFO:tasks.workunit.client.0.smithi032.stdout:5/626: symlink da/dd/lc7 0 2024-06-03T01:21:05.639 INFO:tasks.workunit.client.0.smithi032.stdout:5/627: mkdir da/dd/df/d10/dbc/d18/d1a/d30/d51/dc8 0 2024-06-03T01:21:05.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/628: symlink da/dd/df/d10/dbc/d25/lc9 0 2024-06-03T01:21:05.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/629: write da/dd/f68 [1249416,94211] 0 2024-06-03T01:21:05.648 INFO:tasks.workunit.client.0.smithi032.stdout:4/495: sync 2024-06-03T01:21:05.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/630: symlink da/dd/df/d10/dbc/d18/d1a/d30/lca 0 2024-06-03T01:21:05.651 INFO:tasks.workunit.client.0.smithi032.stdout:4/496: symlink d4/db/da/d37/d90/d91/lb1 0 2024-06-03T01:21:05.652 INFO:tasks.workunit.client.0.smithi032.stdout:4/497: symlink d4/db/da/d5d/lb2 0 2024-06-03T01:21:05.653 INFO:tasks.workunit.client.0.smithi032.stdout:4/498: write d4/db/da/f3f [330770,42520] 0 2024-06-03T01:21:05.654 INFO:tasks.workunit.client.0.smithi032.stdout:4/499: mkdir d4/db/da/d37/d90/d91/db3 0 2024-06-03T01:21:05.654 INFO:tasks.workunit.client.0.smithi032.stdout:4/500: chown d4/db/l62 2016 1 2024-06-03T01:21:05.656 INFO:tasks.workunit.client.0.smithi032.stdout:4/501: creat d4/db/da/d2d/d27/d8c/fb4 x:0 0 0 2024-06-03T01:21:05.675 INFO:tasks.workunit.client.0.smithi032.stdout:5/631: dwrite da/f12 [0,4194304] 0 2024-06-03T01:21:05.675 INFO:tasks.workunit.client.0.smithi032.stdout:4/502: dwrite d4/db/da/f49 [0,4194304] 0 2024-06-03T01:21:05.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/632: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fcb x:0 0 0 2024-06-03T01:21:05.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/633: truncate da/dd/df/d10/dbc/d18/fc1 171074 0 2024-06-03T01:21:05.677 INFO:tasks.workunit.client.0.smithi032.stdout:5/634: fdatasync da/dd/f92 0 2024-06-03T01:21:05.680 INFO:tasks.workunit.client.0.smithi032.stdout:5/635: link da/dd/df/d10/dbc/l65 da/dd/df/d10/dbc/d18/lcc 0 2024-06-03T01:21:05.694 INFO:tasks.workunit.client.0.smithi032.stdout:5/636: dwrite da/dd/df/d10/dbc/d18/d35/d44/f72 [0,4194304] 0 2024-06-03T01:21:05.701 INFO:tasks.workunit.client.0.smithi032.stdout:5/637: mkdir da/dab/dcd 0 2024-06-03T01:21:05.702 INFO:tasks.workunit.client.0.smithi032.stdout:5/638: mknod da/dd/df/d10/cce 0 2024-06-03T01:21:05.702 INFO:tasks.workunit.client.0.smithi032.stdout:5/639: chown da/dd/df/d10/dbc/l55 163132517 1 2024-06-03T01:21:05.705 INFO:tasks.workunit.client.0.smithi032.stdout:5/640: dread da/dd/df/d10/dbc/d18/d35/f98 [0,4194304] 0 2024-06-03T01:21:05.707 INFO:tasks.workunit.client.0.smithi032.stdout:5/641: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/fcf x:0 0 0 2024-06-03T01:21:05.708 INFO:tasks.workunit.client.0.smithi032.stdout:5/642: getdents da/dd/df/d10/dbc/d25 0 2024-06-03T01:21:05.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/643: symlink da/dd/df/d10/dbc/ld0 0 2024-06-03T01:21:05.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/644: write da/dd/df/d10/dbc/d18/d1a/d30/d51/fba [558536,66514] 0 2024-06-03T01:21:05.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/645: chown da/dd/df/fbf 49137716 1 2024-06-03T01:21:05.710 INFO:tasks.workunit.client.0.smithi032.stdout:5/646: chown da/dd/df/d10/dbc/l55 1 1 2024-06-03T01:21:05.711 INFO:tasks.workunit.client.0.smithi032.stdout:5/647: mknod da/dd/df/d10/dbc/d18/db9/cd1 0 2024-06-03T01:21:05.713 INFO:tasks.workunit.client.0.smithi032.stdout:5/648: unlink da/dd/df/d10/dbc/d18/d1a/d30/d51/f57 0 2024-06-03T01:21:05.720 INFO:tasks.workunit.client.0.smithi032.stdout:5/649: write da/dd/df/d10/dbc/d18/d35/d44/f8f [393127,33280] 0 2024-06-03T01:21:05.724 INFO:tasks.workunit.client.0.smithi032.stdout:5/650: getdents da/dd/df/d10/dbc 0 2024-06-03T01:21:05.724 INFO:tasks.workunit.client.0.smithi032.stdout:5/651: truncate da/dd/df/d10/dbc/d18/d35/d44/fb7 774938 0 2024-06-03T01:21:05.725 INFO:tasks.workunit.client.0.smithi032.stdout:5/652: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fd2 x:0 0 0 2024-06-03T01:21:05.725 INFO:tasks.workunit.client.0.smithi032.stdout:5/653: creat da/dd/fd3 x:0 0 0 2024-06-03T01:21:05.734 INFO:tasks.workunit.client.0.smithi032.stdout:5/654: write da/dd/df/d10/dbc/f8a [2399369,10396] 0 2024-06-03T01:21:05.739 INFO:tasks.workunit.client.0.smithi032.stdout:5/655: creat da/dd/df/d10/dbc/d18/d1a/d30/fd4 x:0 0 0 2024-06-03T01:21:05.796 INFO:tasks.workunit.client.0.smithi032.stdout:2/492: sync 2024-06-03T01:21:05.805 INFO:tasks.workunit.client.0.smithi032.stdout:2/493: dread d5/da/f1e [0,4194304] 0 2024-06-03T01:21:05.805 INFO:tasks.workunit.client.0.smithi032.stdout:2/494: chown d5/da/l6e 7065 1 2024-06-03T01:21:05.807 INFO:tasks.workunit.client.0.smithi032.stdout:2/495: dread d5/da/d1a/d2a/d36/f66 [0,4194304] 0 2024-06-03T01:21:05.809 INFO:tasks.workunit.client.0.smithi032.stdout:2/496: dread d5/da/d1a/f6f [0,4194304] 0 2024-06-03T01:21:05.809 INFO:tasks.workunit.client.0.smithi032.stdout:2/497: symlink d5/da/d1a/d2a/d49/d67/la2 0 2024-06-03T01:21:05.811 INFO:tasks.workunit.client.0.smithi032.stdout:2/498: write d5/f3c [663960,4232] 0 2024-06-03T01:21:05.812 INFO:tasks.workunit.client.0.smithi032.stdout:2/499: creat d5/d38/d46/d53/d90/fa3 x:0 0 0 2024-06-03T01:21:05.812 INFO:tasks.workunit.client.0.smithi032.stdout:2/500: rename d5/da/d1a to d5/da/d1a/d2a/d49/d8f/da4 22 2024-06-03T01:21:05.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/501: link d5/da/f1e d5/da/d1a/fa5 0 2024-06-03T01:21:05.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/502: fsync d5/da/f74 0 2024-06-03T01:21:05.815 INFO:tasks.workunit.client.0.smithi032.stdout:2/503: creat d5/da/fa6 x:0 0 0 2024-06-03T01:21:05.829 INFO:tasks.workunit.client.0.smithi032.stdout:2/504: dwrite d5/f6b [0,4194304] 0 2024-06-03T01:21:05.834 INFO:tasks.workunit.client.0.smithi032.stdout:2/505: write d5/f25 [1842276,63806] 0 2024-06-03T01:21:05.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/506: chown d5/da/d1a/d2a/d49/d8f/l95 117 1 2024-06-03T01:21:05.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/507: creat d5/d38/d46/d53/fa7 x:0 0 0 2024-06-03T01:21:05.836 INFO:tasks.workunit.client.0.smithi032.stdout:2/508: mkdir d5/da/d1a/d2a/da8 0 2024-06-03T01:21:05.837 INFO:tasks.workunit.client.0.smithi032.stdout:2/509: write d5/da/d1a/d2a/d36/d3f/f78 [612204,62132] 0 2024-06-03T01:21:05.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/510: dread d5/da/d1a/d2a/d36/d3f/f4f [0,4194304] 0 2024-06-03T01:21:05.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/511: dread - d5/da/f59 zero size 2024-06-03T01:21:05.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/512: mkdir d5/d38/d46/d53/da9 0 2024-06-03T01:21:05.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/513: write d5/da/d1a/f68 [1949905,112690] 0 2024-06-03T01:21:05.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/514: fsync d5/f72 0 2024-06-03T01:21:05.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/515: write d5/da/fa6 [399296,130722] 0 2024-06-03T01:21:05.851 INFO:tasks.workunit.client.0.smithi032.stdout:2/516: truncate d5/da/f29 567565 0 2024-06-03T01:21:05.852 INFO:tasks.workunit.client.0.smithi032.stdout:2/517: creat d5/d38/d46/faa x:0 0 0 2024-06-03T01:21:05.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/518: rename d5/l33 to d5/da/d1a/d2a/d49/d7e/d86/lab 0 2024-06-03T01:21:05.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/519: fsync d5/da/d2e/f3d 0 2024-06-03T01:21:05.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/520: write d5/da/d1a/d2a/d49/f88 [92265,84568] 0 2024-06-03T01:21:05.860 INFO:tasks.workunit.client.0.smithi032.stdout:2/521: rename d5/da/d1a/d2a/d49/d7e to d5/d38/d46/d53/d57/dac 0 2024-06-03T01:21:05.860 INFO:tasks.workunit.client.0.smithi032.stdout:2/522: readlink d5/da/l5b 0 2024-06-03T01:21:05.862 INFO:tasks.workunit.client.0.smithi032.stdout:2/523: unlink d5/d38/d46/faa 0 2024-06-03T01:21:05.862 INFO:tasks.workunit.client.0.smithi032.stdout:2/524: write d5/da/d1a/d69/d83/f96 [219380,99649] 0 2024-06-03T01:21:05.862 INFO:tasks.workunit.client.0.smithi032.stdout:2/525: read d5/da/d1a/d2a/d36/f60 [1496007,24169] 0 2024-06-03T01:21:05.863 INFO:tasks.workunit.client.0.smithi032.stdout:2/526: dread d5/d38/d46/d53/d90/f9b [0,4194304] 0 2024-06-03T01:21:05.863 INFO:tasks.workunit.client.0.smithi032.stdout:2/527: write d5/d38/d46/d53/d90/f9b [1315358,26870] 0 2024-06-03T01:21:05.864 INFO:tasks.workunit.client.0.smithi032.stdout:2/528: symlink d5/da/d2e/lad 0 2024-06-03T01:21:05.865 INFO:tasks.workunit.client.0.smithi032.stdout:2/529: mknod d5/d38/d46/d53/da9/cae 0 2024-06-03T01:21:05.865 INFO:tasks.workunit.client.0.smithi032.stdout:2/530: write d5/da/d1a/f4b [685407,118522] 0 2024-06-03T01:21:05.884 INFO:tasks.workunit.client.0.smithi032.stdout:2/531: dwrite d5/da/d1a/d2a/d36/f60 [0,4194304] 0 2024-06-03T01:21:05.884 INFO:tasks.workunit.client.0.smithi032.stdout:2/532: read d5/d38/d46/d53/d90/f9b [220542,114496] 0 2024-06-03T01:21:05.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/533: symlink d5/da/d1a/d2a/d36/laf 0 2024-06-03T01:21:05.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/534: read - d5/da/d1a/d69/d83/fa1 zero size 2024-06-03T01:21:05.888 INFO:tasks.workunit.client.0.smithi032.stdout:2/535: unlink d5/da/d2e/lad 0 2024-06-03T01:21:05.890 INFO:tasks.workunit.client.0.smithi032.stdout:2/536: unlink d5/da/d1a/d2a/d49/d8f/l95 0 2024-06-03T01:21:05.892 INFO:tasks.workunit.client.0.smithi032.stdout:2/537: mknod d5/d38/d46/d5c/d6c/d89/cb0 0 2024-06-03T01:21:05.892 INFO:tasks.workunit.client.0.smithi032.stdout:2/538: write d5/d38/d46/f81 [214641,29006] 0 2024-06-03T01:21:05.893 INFO:tasks.workunit.client.0.smithi032.stdout:2/539: rmdir d5/d38 39 2024-06-03T01:21:05.907 INFO:tasks.workunit.client.0.smithi032.stdout:2/540: dwrite d5/d38/d46/d53/f5e [0,4194304] 0 2024-06-03T01:21:05.910 INFO:tasks.workunit.client.0.smithi032.stdout:2/541: unlink d5/da/d1a/f68 0 2024-06-03T01:21:05.910 INFO:tasks.workunit.client.0.smithi032.stdout:2/542: readlink d5/da/d1a/d2a/d49/d67/la2 0 2024-06-03T01:21:05.911 INFO:tasks.workunit.client.0.smithi032.stdout:2/543: chown d5/da/d2e/c62 71752 1 2024-06-03T01:21:05.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/544: creat d5/da/d1a/d2a/d49/fb1 x:0 0 0 2024-06-03T01:21:05.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/545: readlink d5/l24 0 2024-06-03T01:21:05.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/546: creat d5/da/d1a/d69/d83/fb2 x:0 0 0 2024-06-03T01:21:05.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/547: stat d5/da/d1a/f41 0 2024-06-03T01:21:05.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/548: readlink d5/d38/d46/d53/d57/dac/d86/lab 0 2024-06-03T01:21:05.927 INFO:tasks.workunit.client.0.smithi032.stdout:2/549: dwrite d5/d38/d46/d53/d57/dac/f54 [0,4194304] 0 2024-06-03T01:21:05.928 INFO:tasks.workunit.client.0.smithi032.stdout:2/550: truncate d5/d38/d46/d5c/f8b 1451913 0 2024-06-03T01:21:05.928 INFO:tasks.workunit.client.0.smithi032.stdout:2/551: chown d5/d38/d46/d53 5710929 1 2024-06-03T01:21:05.937 INFO:tasks.workunit.client.0.smithi032.stdout:2/552: write d5/f23 [6970914,94910] 0 2024-06-03T01:21:05.938 INFO:tasks.workunit.client.0.smithi032.stdout:2/553: dread - d5/da/d1a/d2a/d36/d3f/f84 zero size 2024-06-03T01:21:05.954 INFO:tasks.workunit.client.0.smithi032.stdout:2/554: dwrite d5/d38/d46/f81 [0,4194304] 0 2024-06-03T01:21:05.997 INFO:tasks.workunit.client.0.smithi032.stdout:2/555: dwrite d5/da/d1a/d69/d83/fb2 [0,4194304] 0 2024-06-03T01:21:05.997 INFO:tasks.workunit.client.0.smithi032.stdout:2/556: stat d5/da/d1a/d69/l6a 0 2024-06-03T01:21:06.001 INFO:tasks.workunit.client.0.smithi032.stdout:2/557: symlink d5/da/d1a/d2a/d49/d67/lb3 0 2024-06-03T01:21:06.002 INFO:tasks.workunit.client.0.smithi032.stdout:2/558: unlink d5/d38/d46/d53/f9a 0 2024-06-03T01:21:06.003 INFO:tasks.workunit.client.0.smithi032.stdout:2/559: mkdir d5/d38/db4 0 2024-06-03T01:21:06.004 INFO:tasks.workunit.client.0.smithi032.stdout:2/560: rmdir d5/d38/d46/d53/d57/d6d 39 2024-06-03T01:21:06.006 INFO:tasks.workunit.client.0.smithi032.stdout:2/561: mkdir d5/d38/db5 0 2024-06-03T01:21:06.006 INFO:tasks.workunit.client.0.smithi032.stdout:2/562: mkdir d5/da/d1a/d2a/da8/db6 0 2024-06-03T01:21:06.008 INFO:tasks.workunit.client.0.smithi032.stdout:2/563: rename d5/d38/d46/d5c/c9e to d5/d38/d46/cb7 0 2024-06-03T01:21:06.010 INFO:tasks.workunit.client.0.smithi032.stdout:2/564: creat d5/da/d2e/fb8 x:0 0 0 2024-06-03T01:21:06.021 INFO:tasks.workunit.client.0.smithi032.stdout:7/550: sync 2024-06-03T01:21:06.022 INFO:tasks.workunit.client.0.smithi032.stdout:7/551: chown dc/d12/d78/l7f 59451909 1 2024-06-03T01:21:06.033 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:05 smithi032 ceph-mon[29360]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:06.041 INFO:tasks.workunit.client.0.smithi032.stdout:7/552: dwrite dc/d12/d6a/d86/d88/f94 [0,4194304] 0 2024-06-03T01:21:06.045 INFO:tasks.workunit.client.0.smithi032.stdout:7/553: mknod dc/d12/d22/cb2 0 2024-06-03T01:21:06.046 INFO:tasks.workunit.client.0.smithi032.stdout:7/554: symlink dc/d12/d2a/d2f/d63/lb3 0 2024-06-03T01:21:06.046 INFO:tasks.workunit.client.0.smithi032.stdout:7/555: write dc/d12/d6a/d86/d88/f8c [249392,116548] 0 2024-06-03T01:21:06.068 INFO:tasks.workunit.client.0.smithi032.stdout:7/556: dwrite dc/d12/d22/f85 [0,4194304] 0 2024-06-03T01:21:06.071 INFO:tasks.workunit.client.0.smithi032.stdout:7/557: symlink dc/d12/d24/d33/lb4 0 2024-06-03T01:21:06.086 INFO:tasks.workunit.client.0.smithi032.stdout:7/558: dwrite dc/d12/d24/d33/f66 [0,4194304] 0 2024-06-03T01:21:06.088 INFO:tasks.workunit.client.0.smithi032.stdout:7/559: dread dc/d12/d2a/d2f/d76/f80 [0,4194304] 0 2024-06-03T01:21:06.088 INFO:tasks.workunit.client.0.smithi032.stdout:7/560: stat dc/d12/d22/cb2 0 2024-06-03T01:21:06.109 INFO:tasks.workunit.client.0.smithi032.stdout:7/561: dwrite dc/d12/d24/f6c [0,4194304] 0 2024-06-03T01:21:06.109 INFO:tasks.workunit.client.0.smithi032.stdout:7/562: stat dc/c70 0 2024-06-03T01:21:06.109 INFO:tasks.workunit.client.0.smithi032.stdout:7/563: fdatasync dc/d12/d2a/d2f/fa9 0 2024-06-03T01:21:06.109 INFO:tasks.workunit.client.0.smithi032.stdout:7/564: dread - dc/d12/d24/f38 zero size 2024-06-03T01:21:06.111 INFO:tasks.workunit.client.0.smithi032.stdout:7/565: chown dc/d12/d1b/l1c 666148 1 2024-06-03T01:21:06.111 INFO:tasks.workunit.client.0.smithi032.stdout:7/566: chown dc/d12/d24/d7b/f74 112092 1 2024-06-03T01:21:06.112 INFO:tasks.workunit.client.0.smithi032.stdout:7/567: getdents dc/d12/d2a/d2f/d9a 0 2024-06-03T01:21:06.114 INFO:tasks.workunit.client.0.smithi032.stdout:7/568: getdents dc/d12/d2a/d2f/d76 0 2024-06-03T01:21:06.114 INFO:tasks.workunit.client.0.smithi032.stdout:7/569: readlink dc/d12/d2a/d2f/l73 0 2024-06-03T01:21:06.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/570: write dc/d12/d1b/f36 [1809650,33734] 0 2024-06-03T01:21:06.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/571: stat dc/f14 0 2024-06-03T01:21:06.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/572: chown dc/d12/d1b/f2c 153677 1 2024-06-03T01:21:06.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/573: truncate dc/d12/d24/f57 978631 0 2024-06-03T01:21:06.150 INFO:tasks.workunit.client.0.smithi032.stdout:7/574: dwrite dc/d12/d6a/d86/d89/fab [0,4194304] 0 2024-06-03T01:21:06.150 INFO:tasks.workunit.client.0.smithi032.stdout:7/575: readlink dc/d12/d2a/d2f/l73 0 2024-06-03T01:21:06.151 INFO:tasks.workunit.client.0.smithi032.stdout:7/576: creat dc/d12/d6a/d86/d88/d8b/fb5 x:0 0 0 2024-06-03T01:21:06.152 INFO:tasks.workunit.client.0.smithi032.stdout:7/577: unlink dc/d12/d2a/d2f/d63/c75 0 2024-06-03T01:21:06.168 INFO:tasks.workunit.client.0.smithi032.stdout:7/578: dwrite dc/f2d [0,4194304] 0 2024-06-03T01:21:06.168 INFO:tasks.workunit.client.0.smithi032.stdout:7/579: chown dc/d12/l62 64727353 1 2024-06-03T01:21:06.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/580: mknod dc/d12/d2a/d2f/d9a/cb6 0 2024-06-03T01:21:06.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/581: truncate dc/d12/d2a/d2f/d76/f80 356276 0 2024-06-03T01:21:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:7/582: dread dc/d12/f93 [0,4194304] 0 2024-06-03T01:21:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:7/583: creat dc/d12/d24/d7b/d9d/fb7 x:0 0 0 2024-06-03T01:21:06.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/584: mkdir dc/d12/d6a/d86/d88/d8b/db8 0 2024-06-03T01:21:06.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/585: chown dc/d12/d2a/d2f/d63/d69 590604 1 2024-06-03T01:21:06.176 INFO:tasks.workunit.client.0.smithi032.stdout:7/586: dread - dc/d12/d6a/f72 zero size 2024-06-03T01:21:06.178 INFO:tasks.workunit.client.0.smithi032.stdout:7/587: creat dc/d12/fb9 x:0 0 0 2024-06-03T01:21:06.179 INFO:tasks.workunit.client.0.smithi032.stdout:7/588: creat dc/d12/d6a/d86/d88/d8b/db8/fba x:0 0 0 2024-06-03T01:21:06.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/589: mknod dc/d12/d78/cbb 0 2024-06-03T01:21:06.192 INFO:tasks.workunit.client.0.smithi032.stdout:7/590: dread dc/d12/d22/f29 [0,4194304] 0 2024-06-03T01:21:06.198 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:05 smithi044 ceph-mon[36956]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:06.208 INFO:tasks.workunit.client.0.smithi032.stdout:7/591: dwrite dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:06.212 INFO:tasks.workunit.client.0.smithi032.stdout:7/592: rename dc/d12/d24/d33/f4c to dc/d12/d24/d7b/fbc 0 2024-06-03T01:21:06.212 INFO:tasks.workunit.client.0.smithi032.stdout:7/593: chown dc/d12/d2a/d2f/d9a/cb6 482265 1 2024-06-03T01:21:06.212 INFO:tasks.workunit.client.0.smithi032.stdout:7/594: fdatasync dc/d12/d2a/d81/fac 0 2024-06-03T01:21:06.213 INFO:tasks.workunit.client.0.smithi032.stdout:7/595: symlink dc/d12/d24/d7b/d9d/lbd 0 2024-06-03T01:21:06.213 INFO:tasks.workunit.client.0.smithi032.stdout:7/596: readlink dc/d12/d2a/d81/l96 0 2024-06-03T01:21:06.228 INFO:tasks.workunit.client.0.smithi032.stdout:7/597: dwrite dc/d12/d22/d42/fa0 [4194304,4194304] 0 2024-06-03T01:21:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/598: dwrite dc/d12/d24/f57 [0,4194304] 0 2024-06-03T01:21:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/599: fsync dc/d12/d24/d7b/d9d/fa8 0 2024-06-03T01:21:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/600: readlink dc/l53 0 2024-06-03T01:21:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:7/601: fdatasync dc/d12/d1b/fa5 0 2024-06-03T01:21:06.246 INFO:tasks.workunit.client.0.smithi032.stdout:7/602: truncate dc/d12/d22/d42/f50 215401 0 2024-06-03T01:21:06.356 INFO:tasks.workunit.client.0.smithi032.stdout:1/444: sync 2024-06-03T01:21:06.356 INFO:tasks.workunit.client.0.smithi032.stdout:3/481: sync 2024-06-03T01:21:06.360 INFO:tasks.workunit.client.0.smithi032.stdout:3/482: creat de/d29/d2e/d31/d3c/d50/f9b x:0 0 0 2024-06-03T01:21:06.365 INFO:tasks.workunit.client.0.smithi032.stdout:1/445: dread d18/d32/d3c/f61 [0,4194304] 0 2024-06-03T01:21:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:3/483: dread f5 [4194304,4194304] 0 2024-06-03T01:21:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:3/484: chown de/d29/d2e/d31/d3c/d50/d5e/d7e 0 1 2024-06-03T01:21:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:3/485: dread - de/d29/d2e/d31/d3c/d50/d5e/f89 zero size 2024-06-03T01:21:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:3/486: creat de/d29/d2e/d31/d23/d53/d69/d8c/f9c x:0 0 0 2024-06-03T01:21:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:3/487: fdatasync f5 0 2024-06-03T01:21:06.371 INFO:tasks.workunit.client.0.smithi032.stdout:1/446: write d18/d32/d39/f4a [3845381,117393] 0 2024-06-03T01:21:06.371 INFO:tasks.workunit.client.0.smithi032.stdout:1/447: chown d18/d27/d2b 421 1 2024-06-03T01:21:06.371 INFO:tasks.workunit.client.0.smithi032.stdout:1/448: fdatasync d18/f1a 0 2024-06-03T01:21:06.372 INFO:tasks.workunit.client.0.smithi032.stdout:3/488: creat de/d29/d2e/d31/d8d/f9d x:0 0 0 2024-06-03T01:21:06.373 INFO:tasks.workunit.client.0.smithi032.stdout:1/449: mknod d18/d32/d39/c82 0 2024-06-03T01:21:06.375 INFO:tasks.workunit.client.0.smithi032.stdout:3/489: mknod de/d29/d2e/d31/d23/d53/d69/d8c/c9e 0 2024-06-03T01:21:06.377 INFO:tasks.workunit.client.0.smithi032.stdout:1/450: unlink d18/d2d/d5e/l6c 0 2024-06-03T01:21:06.378 INFO:tasks.workunit.client.0.smithi032.stdout:3/490: dread de/f10 [0,4194304] 0 2024-06-03T01:21:06.396 INFO:tasks.workunit.client.0.smithi032.stdout:1/451: dwrite d18/f19 [0,4194304] 0 2024-06-03T01:21:06.403 INFO:tasks.workunit.client.0.smithi032.stdout:1/452: dread d18/d32/f52 [0,4194304] 0 2024-06-03T01:21:06.403 INFO:tasks.workunit.client.0.smithi032.stdout:1/453: chown f17 5150 1 2024-06-03T01:21:06.403 INFO:tasks.workunit.client.0.smithi032.stdout:1/454: dread - d18/d3f/d5f/d6b/d78/d3d/d47/f77 zero size 2024-06-03T01:21:06.404 INFO:tasks.workunit.client.0.smithi032.stdout:1/455: truncate d18/d2d/f69 428204 0 2024-06-03T01:21:06.404 INFO:tasks.workunit.client.0.smithi032.stdout:1/456: chown d18/d3f/d5f/d6b/d78/d3d 0 1 2024-06-03T01:21:06.413 INFO:tasks.workunit.client.0.smithi032.stdout:1/457: dread d18/d3f/d5f/d6b/d78/f5c [0,4194304] 0 2024-06-03T01:21:06.429 INFO:tasks.workunit.client.0.smithi032.stdout:1/458: dwrite d18/d32/d3c/f67 [0,4194304] 0 2024-06-03T01:21:06.429 INFO:tasks.workunit.client.0.smithi032.stdout:1/459: truncate d18/d3f/d5f/d6b/d78/f2e 2313827 0 2024-06-03T01:21:06.431 INFO:tasks.workunit.client.0.smithi032.stdout:1/460: creat d18/d3f/d5f/d6b/d78/f83 x:0 0 0 2024-06-03T01:21:06.445 INFO:tasks.workunit.client.0.smithi032.stdout:1/461: dwrite d18/d2d/f50 [0,4194304] 0 2024-06-03T01:21:06.447 INFO:tasks.workunit.client.0.smithi032.stdout:1/462: creat d18/d3f/d5f/f84 x:0 0 0 2024-06-03T01:21:06.449 INFO:tasks.workunit.client.0.smithi032.stdout:1/463: rename d18/d3f/d5f/d6b/d78/d3d/c66 to d18/d2d/d5e/c85 0 2024-06-03T01:21:06.449 INFO:tasks.workunit.client.0.smithi032.stdout:1/464: fsync d18/f1a 0 2024-06-03T01:21:06.449 INFO:tasks.workunit.client.0.smithi032.stdout:1/465: dread - d18/d32/d39/d60/f72 zero size 2024-06-03T01:21:06.449 INFO:tasks.workunit.client.0.smithi032.stdout:1/466: fdatasync d18/d2d/f50 0 2024-06-03T01:21:06.466 INFO:tasks.workunit.client.0.smithi032.stdout:1/467: dwrite d18/f5d [0,4194304] 0 2024-06-03T01:21:06.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/468: truncate d18/d27/d2b/f54 1115249 0 2024-06-03T01:21:06.479 INFO:tasks.workunit.client.0.smithi032.stdout:1/469: read d18/d3f/f55 [192696,84581] 0 2024-06-03T01:21:06.496 INFO:tasks.workunit.client.0.smithi032.stdout:1/470: dwrite d18/d2d/f69 [0,4194304] 0 2024-06-03T01:21:06.505 INFO:tasks.workunit.client.0.smithi032.stdout:1/471: dread d18/d2d/d5e/f64 [0,4194304] 0 2024-06-03T01:21:06.505 INFO:tasks.workunit.client.0.smithi032.stdout:1/472: write f17 [10354533,42267] 0 2024-06-03T01:21:06.512 INFO:tasks.workunit.client.0.smithi032.stdout:1/473: write d18/d2d/f50 [1260059,23349] 0 2024-06-03T01:21:06.517 INFO:tasks.workunit.client.0.smithi032.stdout:1/474: write d18/d32/d3c/f61 [1518767,114831] 0 2024-06-03T01:21:06.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/475: link d18/d2d/d5e/c85 d18/c86 0 2024-06-03T01:21:06.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/476: truncate d18/f5d 4409422 0 2024-06-03T01:21:06.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/477: chown d18/d32/l4e 488852299 1 2024-06-03T01:21:06.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/478: stat d18/l21 0 2024-06-03T01:21:06.521 INFO:tasks.workunit.client.0.smithi032.stdout:1/479: rmdir d18 39 2024-06-03T01:21:06.522 INFO:tasks.workunit.client.0.smithi032.stdout:1/480: symlink d18/d3f/d5f/d6b/d78/d3d/d47/l87 0 2024-06-03T01:21:06.539 INFO:tasks.workunit.client.0.smithi032.stdout:1/481: dwrite d18/d3f/d5f/f84 [0,4194304] 0 2024-06-03T01:21:06.539 INFO:tasks.workunit.client.0.smithi032.stdout:1/482: chown d18/d3f/d5f/d6b/f73 171 1 2024-06-03T01:21:06.541 INFO:tasks.workunit.client.0.smithi032.stdout:1/483: symlink d18/d27/d2b/l88 0 2024-06-03T01:21:06.542 INFO:tasks.workunit.client.0.smithi032.stdout:1/484: mkdir d18/d27/d89 0 2024-06-03T01:21:06.549 INFO:tasks.workunit.client.0.smithi032.stdout:1/485: dread d18/d3f/d5f/f84 [0,4194304] 0 2024-06-03T01:21:06.552 INFO:tasks.workunit.client.0.smithi032.stdout:1/486: rename d18/d27/d2b/l88 to d18/d3f/d5f/d6b/d78/l8a 0 2024-06-03T01:21:06.554 INFO:tasks.workunit.client.0.smithi032.stdout:1/487: rename d18/d27/d2b/f54 to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a/f8b 0 2024-06-03T01:21:06.555 INFO:tasks.workunit.client.0.smithi032.stdout:1/488: read d18/d3f/f55 [743693,72538] 0 2024-06-03T01:21:06.556 INFO:tasks.workunit.client.0.smithi032.stdout:1/489: creat d18/d3f/d5f/d6b/d78/d3d/f8c x:0 0 0 2024-06-03T01:21:06.576 INFO:tasks.workunit.client.0.smithi032.stdout:1/490: dwrite d18/f5d [0,4194304] 0 2024-06-03T01:21:06.576 INFO:tasks.workunit.client.0.smithi032.stdout:1/491: chown d18/d3f/d5f/d6b/d78/f5c 1 1 2024-06-03T01:21:06.577 INFO:tasks.workunit.client.0.smithi032.stdout:1/492: symlink d18/d32/l8d 0 2024-06-03T01:21:06.577 INFO:tasks.workunit.client.0.smithi032.stdout:1/493: dread - d18/d3f/d5f/d6b/d78/f83 zero size 2024-06-03T01:21:06.578 INFO:tasks.workunit.client.0.smithi032.stdout:1/494: creat d18/f8e x:0 0 0 2024-06-03T01:21:06.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/495: readlink d18/d32/l56 0 2024-06-03T01:21:06.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/496: readlink d18/d3f/d5f/d6b/d78/d3d/d47/l87 0 2024-06-03T01:21:06.594 INFO:tasks.workunit.client.0.smithi032.stdout:1/497: dwrite d18/f1a [0,4194304] 0 2024-06-03T01:21:06.598 INFO:tasks.workunit.client.0.smithi032.stdout:2/565: sync 2024-06-03T01:21:06.598 INFO:tasks.workunit.client.0.smithi032.stdout:8/409: sync 2024-06-03T01:21:06.599 INFO:tasks.workunit.client.0.smithi032.stdout:4/503: sync 2024-06-03T01:21:06.599 INFO:tasks.workunit.client.0.smithi032.stdout:6/468: sync 2024-06-03T01:21:06.599 INFO:tasks.workunit.client.0.smithi032.stdout:1/498: dread d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a/f8b [0,4194304] 0 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:7/603: sync 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:9/606: sync 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:0/532: sync 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:0/533: fdatasync d1/da/d10/d53/fae 0 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:0/534: dread - d1/da/d10/d29/f45 zero size 2024-06-03T01:21:06.600 INFO:tasks.workunit.client.0.smithi032.stdout:5/656: sync 2024-06-03T01:21:06.602 INFO:tasks.workunit.client.0.smithi032.stdout:4/504: unlink d4/d57/d6c/f73 0 2024-06-03T01:21:06.602 INFO:tasks.workunit.client.0.smithi032.stdout:4/505: dread - d4/db/da/d2d/d27/d8c/fb4 zero size 2024-06-03T01:21:06.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/410: unlink dc/d25/d28/d51/d5d/d5e/f73 0 2024-06-03T01:21:06.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/411: fsync dc/de/d1e/f2e 0 2024-06-03T01:21:06.605 INFO:tasks.workunit.client.0.smithi032.stdout:3/491: sync 2024-06-03T01:21:06.607 INFO:tasks.workunit.client.0.smithi032.stdout:2/566: creat d5/d38/d46/d53/d57/dac/d86/fb9 x:0 0 0 2024-06-03T01:21:06.607 INFO:tasks.workunit.client.0.smithi032.stdout:6/469: truncate d8/d11/d20/d7f/f78 293228 0 2024-06-03T01:21:06.608 INFO:tasks.workunit.client.0.smithi032.stdout:1/499: getdents d18/d32 0 2024-06-03T01:21:06.609 INFO:tasks.workunit.client.0.smithi032.stdout:9/607: mknod d0/d11/d76/cc8 0 2024-06-03T01:21:06.609 INFO:tasks.workunit.client.0.smithi032.stdout:9/608: readlink d0/d11/d79/l98 0 2024-06-03T01:21:06.610 INFO:tasks.workunit.client.0.smithi032.stdout:0/535: rmdir d1/da/d28/d3b/d50 39 2024-06-03T01:21:06.610 INFO:tasks.workunit.client.0.smithi032.stdout:2/567: dread d5/f3c [0,4194304] 0 2024-06-03T01:21:06.611 INFO:tasks.workunit.client.0.smithi032.stdout:7/604: dread dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:06.612 INFO:tasks.workunit.client.0.smithi032.stdout:8/412: creat dc/d25/d64/f89 x:0 0 0 2024-06-03T01:21:06.612 INFO:tasks.workunit.client.0.smithi032.stdout:8/413: read dc/de/f37 [3886379,79556] 0 2024-06-03T01:21:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/609: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9 0 2024-06-03T01:21:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/610: fdatasync d0/d11/d1d/f7e 0 2024-06-03T01:21:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/611: chown d0/d11/d1d/f99 31515454 1 2024-06-03T01:21:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/612: write d0/d11/d1d/da6/d71/f96 [1564212,86426] 0 2024-06-03T01:21:06.615 INFO:tasks.workunit.client.0.smithi032.stdout:0/536: creat d1/faf x:0 0 0 2024-06-03T01:21:06.615 INFO:tasks.workunit.client.0.smithi032.stdout:0/537: write d1/da/d28/d3b/d68/d83/f95 [68697,126118] 0 2024-06-03T01:21:06.615 INFO:tasks.workunit.client.0.smithi032.stdout:0/538: stat d1/l7 0 2024-06-03T01:21:06.616 INFO:tasks.workunit.client.0.smithi032.stdout:2/568: symlink d5/da/d1a/d2a/lba 0 2024-06-03T01:21:06.618 INFO:tasks.workunit.client.0.smithi032.stdout:8/414: creat dc/d14/d31/d3b/d74/d76/f8a x:0 0 0 2024-06-03T01:21:06.619 INFO:tasks.workunit.client.0.smithi032.stdout:3/492: dread de/d29/d2e/d31/d3c/f61 [0,4194304] 0 2024-06-03T01:21:06.621 INFO:tasks.workunit.client.0.smithi032.stdout:1/500: rmdir d18/d2d/d5e 39 2024-06-03T01:21:06.621 INFO:tasks.workunit.client.0.smithi032.stdout:1/501: chown d18/d3f/d5f/d6b/d78/d3d/l4d 0 1 2024-06-03T01:21:06.621 INFO:tasks.workunit.client.0.smithi032.stdout:1/502: truncate d18/d32/f4b 5008670 0 2024-06-03T01:21:06.623 INFO:tasks.workunit.client.0.smithi032.stdout:9/613: truncate d0/d23/d25/f52 3790899 0 2024-06-03T01:21:06.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/539: mkdir d1/da/d28/d3b/d68/d82/db0 0 2024-06-03T01:21:06.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/506: rename d4/db/da/d37/d90/d91 to d4/db/da/d2d/d27/d8c/db5 0 2024-06-03T01:21:06.628 INFO:tasks.workunit.client.0.smithi032.stdout:2/569: truncate d5/d38/d46/d5c/f8b 687454 0 2024-06-03T01:21:06.630 INFO:tasks.workunit.client.0.smithi032.stdout:1/503: mknod d18/d27/d89/c8f 0 2024-06-03T01:21:06.630 INFO:tasks.workunit.client.0.smithi032.stdout:1/504: chown d18/d3f/d5f/d6b/d78/f5c 0 1 2024-06-03T01:21:06.630 INFO:tasks.workunit.client.0.smithi032.stdout:1/505: fsync d18/d2d/d5e/f64 0 2024-06-03T01:21:06.631 INFO:tasks.workunit.client.0.smithi032.stdout:9/614: truncate d0/d23/d24/f2a 1809495 0 2024-06-03T01:21:06.631 INFO:tasks.workunit.client.0.smithi032.stdout:9/615: readlink d0/d11/d1d/d38/d4f/db0/d41/l74 0 2024-06-03T01:21:06.631 INFO:tasks.workunit.client.0.smithi032.stdout:9/616: write d0/d11/d1d/d38/d4f/fa1 [1570998,3216] 0 2024-06-03T01:21:06.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/540: mknod d1/da/d28/d3b/d68/cb1 0 2024-06-03T01:21:06.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/541: write d1/da/f16 [1657594,34513] 0 2024-06-03T01:21:06.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/542: dread - d1/da/d10/d53/f7e zero size 2024-06-03T01:21:06.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/543: fsync d1/da/d28/f33 0 2024-06-03T01:21:06.635 INFO:tasks.workunit.client.0.smithi032.stdout:6/470: rename d8/d55/f7a to d8/d11/d20/d7f/f91 0 2024-06-03T01:21:06.635 INFO:tasks.workunit.client.0.smithi032.stdout:6/471: chown d8/d11/d20/d2b/d39/d19/l1a 11 1 2024-06-03T01:21:06.636 INFO:tasks.workunit.client.0.smithi032.stdout:2/570: mknod d5/da/d1a/d2a/d49/cbb 0 2024-06-03T01:21:06.637 INFO:tasks.workunit.client.0.smithi032.stdout:1/506: rmdir d18/d2d/d5e 39 2024-06-03T01:21:06.637 INFO:tasks.workunit.client.0.smithi032.stdout:1/507: chown d18/d27/d2b 2280305 1 2024-06-03T01:21:06.637 INFO:tasks.workunit.client.0.smithi032.stdout:4/507: dread d4/db/da/d1d/f21 [0,4194304] 0 2024-06-03T01:21:06.639 INFO:tasks.workunit.client.0.smithi032.stdout:8/415: dwrite dc/d25/d28/d58/f75 [0,4194304] 0 2024-06-03T01:21:06.639 INFO:tasks.workunit.client.0.smithi032.stdout:9/617: creat d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/fca x:0 0 0 2024-06-03T01:21:06.639 INFO:tasks.workunit.client.0.smithi032.stdout:9/618: chown d0/d11/f48 23915 1 2024-06-03T01:21:06.640 INFO:tasks.workunit.client.0.smithi032.stdout:0/544: truncate d1/da/d10/f4f 3777740 0 2024-06-03T01:21:06.640 INFO:tasks.workunit.client.0.smithi032.stdout:0/545: chown d1/da/d28/ca6 1103622046 1 2024-06-03T01:21:06.643 INFO:tasks.workunit.client.0.smithi032.stdout:3/493: rename de/d29/d2e/d31/f38 to de/d29/d2e/d31/d23/f9f 0 2024-06-03T01:21:06.646 INFO:tasks.workunit.client.0.smithi032.stdout:6/472: symlink d8/d11/d20/d2b/d39/l92 0 2024-06-03T01:21:06.646 INFO:tasks.workunit.client.0.smithi032.stdout:6/473: fdatasync d8/d11/d20/d36/f38 0 2024-06-03T01:21:06.647 INFO:tasks.workunit.client.0.smithi032.stdout:2/571: symlink d5/d38/d46/d53/d57/d6d/lbc 0 2024-06-03T01:21:06.648 INFO:tasks.workunit.client.0.smithi032.stdout:2/572: chown d5/f3c 425 1 2024-06-03T01:21:06.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/619: dread d0/d11/d76/f7c [0,4194304] 0 2024-06-03T01:21:06.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/620: fsync d0/d23/d24/f9d 0 2024-06-03T01:21:06.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/621: write d0/d11/d1d/da6/d71/fc7 [741103,12022] 0 2024-06-03T01:21:06.649 INFO:tasks.workunit.client.0.smithi032.stdout:1/508: rmdir d18/d32 39 2024-06-03T01:21:06.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/508: dread d4/db/da/d1d/f7e [0,4194304] 0 2024-06-03T01:21:06.652 INFO:tasks.workunit.client.0.smithi032.stdout:8/416: getdents dc/d14/d31 0 2024-06-03T01:21:06.652 INFO:tasks.workunit.client.0.smithi032.stdout:9/622: fdatasync d0/d11/d1d/da6/d71/f96 0 2024-06-03T01:21:06.654 INFO:tasks.workunit.client.0.smithi032.stdout:0/546: rmdir d1/da/d28/d3b/d68/d82/db0 0 2024-06-03T01:21:06.657 INFO:tasks.workunit.client.0.smithi032.stdout:3/494: creat de/d29/d2e/fa0 x:0 0 0 2024-06-03T01:21:06.657 INFO:tasks.workunit.client.0.smithi032.stdout:3/495: readlink de/d29/d2e/d31/d23/d53/l8a 0 2024-06-03T01:21:06.659 INFO:tasks.workunit.client.0.smithi032.stdout:6/474: creat d8/d11/d20/d2b/f93 x:0 0 0 2024-06-03T01:21:06.659 INFO:tasks.workunit.client.0.smithi032.stdout:4/509: dread d4/db/f1c [0,4194304] 0 2024-06-03T01:21:06.662 INFO:tasks.workunit.client.0.smithi032.stdout:1/509: write d18/f5d [501098,124955] 0 2024-06-03T01:21:06.663 INFO:tasks.workunit.client.0.smithi032.stdout:9/623: rmdir d0/d11/d1d/d38 39 2024-06-03T01:21:06.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/417: truncate dc/f13 2897461 0 2024-06-03T01:21:06.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/418: chown dc/d25/d28/d32/f50 1656639938 1 2024-06-03T01:21:06.666 INFO:tasks.workunit.client.0.smithi032.stdout:0/547: rmdir d1/da/d28/d3b/d5d/da9 39 2024-06-03T01:21:06.666 INFO:tasks.workunit.client.0.smithi032.stdout:0/548: chown d1/da/d28/d3b/d50 128146 1 2024-06-03T01:21:06.670 INFO:tasks.workunit.client.0.smithi032.stdout:6/475: rename d8/d11/d20/d2b/d39/d19/f1d to d8/d11/d20/d3b/d66/f94 0 2024-06-03T01:21:06.673 INFO:tasks.workunit.client.0.smithi032.stdout:4/510: rmdir d4/db/da/d2d/d27/d8c/d9a/da2 0 2024-06-03T01:21:06.676 INFO:tasks.workunit.client.0.smithi032.stdout:2/573: rmdir d5/da/d2e 39 2024-06-03T01:21:06.678 INFO:tasks.workunit.client.0.smithi032.stdout:1/510: truncate d18/d32/f52 6724694 0 2024-06-03T01:21:06.681 INFO:tasks.workunit.client.0.smithi032.stdout:9/624: mknod d0/d11/d9c/ccb 0 2024-06-03T01:21:06.682 INFO:tasks.workunit.client.0.smithi032.stdout:3/496: dwrite de/d29/d2e/d31/f96 [0,4194304] 0 2024-06-03T01:21:06.682 INFO:tasks.workunit.client.0.smithi032.stdout:3/497: chown de/d29/d2e/d31/d23 997428 1 2024-06-03T01:21:06.682 INFO:tasks.workunit.client.0.smithi032.stdout:8/419: mkdir dc/d25/d64/d8b 0 2024-06-03T01:21:06.686 INFO:tasks.workunit.client.0.smithi032.stdout:9/625: symlink d0/d23/d24/lcc 0 2024-06-03T01:21:06.686 INFO:tasks.workunit.client.0.smithi032.stdout:9/626: stat d0/d11/d1d/d38/d4f/db0/d41/c81 0 2024-06-03T01:21:06.686 INFO:tasks.workunit.client.0.smithi032.stdout:3/498: read de/d29/d2e/d31/d3c/f54 [115383,36809] 0 2024-06-03T01:21:06.686 INFO:tasks.workunit.client.0.smithi032.stdout:3/499: write de/d29/d2e/d31/d3c/f44 [5272853,69443] 0 2024-06-03T01:21:06.689 INFO:tasks.workunit.client.0.smithi032.stdout:6/476: dread d8/d11/d20/d7f/f91 [0,4194304] 0 2024-06-03T01:21:06.690 INFO:tasks.workunit.client.0.smithi032.stdout:0/549: dwrite d1/da/d10/f6f [0,4194304] 0 2024-06-03T01:21:06.692 INFO:tasks.workunit.client.0.smithi032.stdout:8/420: symlink dc/l8c 0 2024-06-03T01:21:06.696 INFO:tasks.workunit.client.0.smithi032.stdout:4/511: getdents d4/d57/d6c 0 2024-06-03T01:21:06.697 INFO:tasks.workunit.client.0.smithi032.stdout:4/512: dread - d4/d7f/f92 zero size 2024-06-03T01:21:06.697 INFO:tasks.workunit.client.0.smithi032.stdout:6/477: symlink d8/d11/d20/d21/l95 0 2024-06-03T01:21:06.704 INFO:tasks.workunit.client.0.smithi032.stdout:8/421: dread dc/f15 [0,4194304] 0 2024-06-03T01:21:06.706 INFO:tasks.workunit.client.0.smithi032.stdout:0/550: mkdir d1/da/db2 0 2024-06-03T01:21:06.706 INFO:tasks.workunit.client.0.smithi032.stdout:0/551: readlink d1/da/d10/d80/l9d 0 2024-06-03T01:21:06.706 INFO:tasks.workunit.client.0.smithi032.stdout:4/513: dread d4/db/da/d1d/f21 [0,4194304] 0 2024-06-03T01:21:06.709 INFO:tasks.workunit.client.0.smithi032.stdout:8/422: mkdir dc/d25/d28/d58/d8d 0 2024-06-03T01:21:06.709 INFO:tasks.workunit.client.0.smithi032.stdout:8/423: fsync dc/f13 0 2024-06-03T01:21:06.710 INFO:tasks.workunit.client.0.smithi032.stdout:0/552: unlink d1/f64 0 2024-06-03T01:21:06.710 INFO:tasks.workunit.client.0.smithi032.stdout:0/553: chown d1/c75 24 1 2024-06-03T01:21:06.711 INFO:tasks.workunit.client.0.smithi032.stdout:0/554: read d1/da/d28/f56 [535659,103487] 0 2024-06-03T01:21:06.711 INFO:tasks.workunit.client.0.smithi032.stdout:4/514: creat d4/db/da/fb6 x:0 0 0 2024-06-03T01:21:06.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/555: getdents d1/da/d10 0 2024-06-03T01:21:06.716 INFO:tasks.workunit.client.0.smithi032.stdout:4/515: link d4/db/f36 d4/db/da/d1d/fb7 0 2024-06-03T01:21:06.717 INFO:tasks.workunit.client.0.smithi032.stdout:0/556: symlink d1/da/d28/d3b/d5d/da9/lb3 0 2024-06-03T01:21:06.718 INFO:tasks.workunit.client.0.smithi032.stdout:4/516: rename d4/db/da/d2d/d27/f8a to d4/db/da/d37/dab/fb8 0 2024-06-03T01:21:06.719 INFO:tasks.workunit.client.0.smithi032.stdout:0/557: creat d1/da/d10/d29/fb4 x:0 0 0 2024-06-03T01:21:06.720 INFO:tasks.workunit.client.0.smithi032.stdout:4/517: unlink d4/db/da/d2d/c4d 0 2024-06-03T01:21:06.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/558: mkdir d1/da/d28/d3b/d68/db5 0 2024-06-03T01:21:06.722 INFO:tasks.workunit.client.0.smithi032.stdout:0/559: chown d1/da/d28/f56 19263 1 2024-06-03T01:21:06.722 INFO:tasks.workunit.client.0.smithi032.stdout:8/424: dread dc/d25/d28/d3c/f3f [0,4194304] 0 2024-06-03T01:21:06.723 INFO:tasks.workunit.client.0.smithi032.stdout:4/518: unlink d4/db/da/d2d/f2a 0 2024-06-03T01:21:06.724 INFO:tasks.workunit.client.0.smithi032.stdout:0/560: dread d1/da/d10/f7a [0,4194304] 0 2024-06-03T01:21:06.725 INFO:tasks.workunit.client.0.smithi032.stdout:8/425: rename c7 to dc/d25/d28/d51/c8e 0 2024-06-03T01:21:06.725 INFO:tasks.workunit.client.0.smithi032.stdout:8/426: fsync dc/d14/f83 0 2024-06-03T01:21:06.725 INFO:tasks.workunit.client.0.smithi032.stdout:0/561: getdents d1/da/d28/d3b/d68/d83 0 2024-06-03T01:21:06.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/562: truncate d1/da/d28/d3b/d5d/f94 309272 0 2024-06-03T01:21:06.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/563: fdatasync d1/da/d28/d3b/d50/f5c 0 2024-06-03T01:21:06.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/564: readlink d1/da/d28/d3b/d50/l5e 0 2024-06-03T01:21:06.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/565: write d1/da/d10/d53/f7e [942614,106344] 0 2024-06-03T01:21:06.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/566: readlink d1/l63 0 2024-06-03T01:21:06.728 INFO:tasks.workunit.client.0.smithi032.stdout:4/519: dread d4/db/da/d1d/f7e [0,4194304] 0 2024-06-03T01:21:06.731 INFO:tasks.workunit.client.0.smithi032.stdout:4/520: rename d4/db/da/d2d/d27/d53/l8f to d4/db/da/d2d/d27/lb9 0 2024-06-03T01:21:06.731 INFO:tasks.workunit.client.0.smithi032.stdout:4/521: chown d4/db/da/d37/f3b 536586 1 2024-06-03T01:21:06.732 INFO:tasks.workunit.client.0.smithi032.stdout:4/522: creat d4/db/da/d1d/d6b/fba x:0 0 0 2024-06-03T01:21:06.733 INFO:tasks.workunit.client.0.smithi032.stdout:3/500: dwrite de/d29/d2e/d31/d23/d53/d69/f8e [4194304,4194304] 0 2024-06-03T01:21:06.733 INFO:tasks.workunit.client.0.smithi032.stdout:1/511: dwrite d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a/f8b [0,4194304] 0 2024-06-03T01:21:06.734 INFO:tasks.workunit.client.0.smithi032.stdout:1/512: chown d18/d3f/d5f/d6b/d78/d3d/l4d 1 1 2024-06-03T01:21:06.734 INFO:tasks.workunit.client.0.smithi032.stdout:2/574: dwrite d5/d38/d46/d53/d57/dac/f54 [0,4194304] 0 2024-06-03T01:21:06.734 INFO:tasks.workunit.client.0.smithi032.stdout:2/575: fdatasync d5/da/d2e/fb8 0 2024-06-03T01:21:06.735 INFO:tasks.workunit.client.0.smithi032.stdout:9/627: dwrite d0/d11/d9c/fa3 [0,4194304] 0 2024-06-03T01:21:06.735 INFO:tasks.workunit.client.0.smithi032.stdout:4/523: mknod d4/db/da/d5d/cbb 0 2024-06-03T01:21:06.735 INFO:tasks.workunit.client.0.smithi032.stdout:6/478: dwrite d8/d11/d20/d3b/f80 [0,4194304] 0 2024-06-03T01:21:06.737 INFO:tasks.workunit.client.0.smithi032.stdout:3/501: mkdir de/d29/d2e/d31/da1 0 2024-06-03T01:21:06.737 INFO:tasks.workunit.client.0.smithi032.stdout:3/502: fdatasync de/f5b 0 2024-06-03T01:21:06.737 INFO:tasks.workunit.client.0.smithi032.stdout:3/503: truncate de/d29/d2e/d31/d23/f9f 4592770 0 2024-06-03T01:21:06.745 INFO:tasks.workunit.client.0.smithi032.stdout:1/513: dread d18/d3f/d5f/d6b/d78/f2e [0,4194304] 0 2024-06-03T01:21:06.745 INFO:tasks.workunit.client.0.smithi032.stdout:1/514: read d18/d3f/d5f/d6b/d78/f45 [2606102,128670] 0 2024-06-03T01:21:06.745 INFO:tasks.workunit.client.0.smithi032.stdout:8/427: write dc/d25/d28/d58/f75 [4151786,97759] 0 2024-06-03T01:21:06.745 INFO:tasks.workunit.client.0.smithi032.stdout:4/524: dread d4/d7f/f8b [0,4194304] 0 2024-06-03T01:21:06.748 INFO:tasks.workunit.client.0.smithi032.stdout:2/576: creat d5/d38/d46/d53/d57/d6d/fbd x:0 0 0 2024-06-03T01:21:06.748 INFO:tasks.workunit.client.0.smithi032.stdout:2/577: fsync d5/da/d1a/d2a/d36/d3f/f4f 0 2024-06-03T01:21:06.748 INFO:tasks.workunit.client.0.smithi032.stdout:2/578: fsync d5/da/f39 0 2024-06-03T01:21:06.749 INFO:tasks.workunit.client.0.smithi032.stdout:6/479: write d8/d11/d20/d36/f38 [1619821,76625] 0 2024-06-03T01:21:06.749 INFO:tasks.workunit.client.0.smithi032.stdout:6/480: creat d8/d11/d20/d31/f96 x:0 0 0 2024-06-03T01:21:06.751 INFO:tasks.workunit.client.0.smithi032.stdout:9/628: mknod d0/d23/ccd 0 2024-06-03T01:21:06.755 INFO:tasks.workunit.client.0.smithi032.stdout:0/567: dwrite d1/f21 [0,4194304] 0 2024-06-03T01:21:06.756 INFO:tasks.workunit.client.0.smithi032.stdout:1/515: creat d18/d2d/f90 x:0 0 0 2024-06-03T01:21:06.761 INFO:tasks.workunit.client.0.smithi032.stdout:8/428: mknod dc/d25/d28/d32/c8f 0 2024-06-03T01:21:06.769 INFO:tasks.workunit.client.0.smithi032.stdout:0/568: write d1/da/d10/d29/f96 [1129794,75494] 0 2024-06-03T01:21:06.769 INFO:tasks.workunit.client.0.smithi032.stdout:8/429: dread dc/f13 [0,4194304] 0 2024-06-03T01:21:06.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/579: mkdir d5/d38/d46/dbe 0 2024-06-03T01:21:06.773 INFO:tasks.workunit.client.0.smithi032.stdout:6/481: creat d8/d11/d20/d31/d3f/f97 x:0 0 0 2024-06-03T01:21:06.775 INFO:tasks.workunit.client.0.smithi032.stdout:9/629: dwrite d0/d11/d76/fa7 [0,4194304] 0 2024-06-03T01:21:06.775 INFO:tasks.workunit.client.0.smithi032.stdout:9/630: creat d0/d23/fce x:0 0 0 2024-06-03T01:21:06.781 INFO:tasks.workunit.client.0.smithi032.stdout:1/516: mkdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91 0 2024-06-03T01:21:06.781 INFO:tasks.workunit.client.0.smithi032.stdout:1/517: dread - d18/d3f/d5f/d6b/d78/f83 zero size 2024-06-03T01:21:06.781 INFO:tasks.workunit.client.0.smithi032.stdout:1/518: truncate d18/d2d/f90 997498 0 2024-06-03T01:21:06.781 INFO:tasks.workunit.client.0.smithi032.stdout:4/525: getdents d4/db/da/d1d/d6b 0 2024-06-03T01:21:06.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/526: dread d4/db/da/f1b [0,4194304] 0 2024-06-03T01:21:06.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/527: chown d4/db/da/d2d/d27/lb9 129284 1 2024-06-03T01:21:06.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/528: write d4/db/f2e [933302,128664] 0 2024-06-03T01:21:06.784 INFO:tasks.workunit.client.0.smithi032.stdout:0/569: unlink d1/f9 0 2024-06-03T01:21:06.787 INFO:tasks.workunit.client.0.smithi032.stdout:8/430: mknod dc/d14/d31/d3b/c90 0 2024-06-03T01:21:06.787 INFO:tasks.workunit.client.0.smithi032.stdout:8/431: dread - dc/d25/d28/d32/f60 zero size 2024-06-03T01:21:06.788 INFO:tasks.workunit.client.0.smithi032.stdout:0/570: dread d1/da/d10/f7a [0,4194304] 0 2024-06-03T01:21:06.789 INFO:tasks.workunit.client.0.smithi032.stdout:2/580: rmdir d5/d38/d46/d5c/d6c 39 2024-06-03T01:21:06.793 INFO:tasks.workunit.client.0.smithi032.stdout:9/631: symlink d0/d11/lcf 0 2024-06-03T01:21:06.793 INFO:tasks.workunit.client.0.smithi032.stdout:9/632: readlink d0/d23/l2d 0 2024-06-03T01:21:06.793 INFO:tasks.workunit.client.0.smithi032.stdout:9/633: chown d0/d11/l7d 52918 1 2024-06-03T01:21:06.797 INFO:tasks.workunit.client.0.smithi032.stdout:4/529: dwrite d4/d5c/f5e [0,4194304] 0 2024-06-03T01:21:06.798 INFO:tasks.workunit.client.0.smithi032.stdout:1/519: link d18/l21 d18/l92 0 2024-06-03T01:21:06.800 INFO:tasks.workunit.client.0.smithi032.stdout:8/432: creat dc/de/f91 x:0 0 0 2024-06-03T01:21:06.802 INFO:tasks.workunit.client.0.smithi032.stdout:0/571: rename d1/l38 to d1/da/d28/d3b/d5d/lb6 0 2024-06-03T01:21:06.806 INFO:tasks.workunit.client.0.smithi032.stdout:9/634: creat d0/d11/d1d/da6/fd0 x:0 0 0 2024-06-03T01:21:06.809 INFO:tasks.workunit.client.0.smithi032.stdout:4/530: symlink d4/db/da/d5d/lbc 0 2024-06-03T01:21:06.809 INFO:tasks.workunit.client.0.smithi032.stdout:4/531: chown d4/db/f36 1445 1 2024-06-03T01:21:06.809 INFO:tasks.workunit.client.0.smithi032.stdout:4/532: dread - d4/d7f/f92 zero size 2024-06-03T01:21:06.821 INFO:tasks.workunit.client.0.smithi032.stdout:1/520: symlink d18/d2d/l93 0 2024-06-03T01:21:06.824 INFO:tasks.workunit.client.0.smithi032.stdout:0/572: symlink d1/da/d28/d3b/d68/d83/lb7 0 2024-06-03T01:21:06.827 INFO:tasks.workunit.client.0.smithi032.stdout:2/581: rmdir d5/d38/d46/d53/d57/d6d 39 2024-06-03T01:21:06.833 INFO:tasks.workunit.client.0.smithi032.stdout:4/533: rmdir d4/db/da/dae 0 2024-06-03T01:21:06.833 INFO:tasks.workunit.client.0.smithi032.stdout:4/534: write d4/db/da/d37/f84 [5199312,32583] 0 2024-06-03T01:21:06.835 INFO:tasks.workunit.client.0.smithi032.stdout:9/635: getdents d0/d23 0 2024-06-03T01:21:06.836 INFO:tasks.workunit.client.0.smithi032.stdout:9/636: write d0/d11/d1d/f99 [1897712,34947] 0 2024-06-03T01:21:06.836 INFO:tasks.workunit.client.0.smithi032.stdout:9/637: readlink d0/d11/d1d/d38/d4f/db0/lb4 0 2024-06-03T01:21:06.836 INFO:tasks.workunit.client.0.smithi032.stdout:1/521: dwrite d18/d27/d2b/f7f [0,4194304] 0 2024-06-03T01:21:06.837 INFO:tasks.workunit.client.0.smithi032.stdout:4/535: creat d4/db/da/d2d/d27/d8c/db5/fbd x:0 0 0 2024-06-03T01:21:06.925 INFO:tasks.workunit.client.0.smithi032.stdout:0/573: dwrite d1/da/d10/f7a [0,4194304] 0 2024-06-03T01:21:06.928 INFO:tasks.workunit.client.0.smithi032.stdout:2/582: dwrite d5/d38/d46/d53/d90/f9b [0,4194304] 0 2024-06-03T01:21:06.930 INFO:tasks.workunit.client.0.smithi032.stdout:8/433: rmdir dc/d25/d3a 0 2024-06-03T01:21:06.930 INFO:tasks.workunit.client.0.smithi032.stdout:8/434: write dc/de/f91 [678223,100867] 0 2024-06-03T01:21:06.930 INFO:tasks.workunit.client.0.smithi032.stdout:8/435: dread - dc/d25/d28/d51/d5d/d5e/f84 zero size 2024-06-03T01:21:06.934 INFO:tasks.workunit.client.0.smithi032.stdout:9/638: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1 0 2024-06-03T01:21:06.936 INFO:tasks.workunit.client.0.smithi032.stdout:1/522: getdents d18/d27 0 2024-06-03T01:21:06.938 INFO:tasks.workunit.client.0.smithi032.stdout:4/536: creat d4/d78/fbe x:0 0 0 2024-06-03T01:21:06.938 INFO:tasks.workunit.client.0.smithi032.stdout:4/537: truncate d4/db/da/d37/f70 14135 0 2024-06-03T01:21:06.939 INFO:tasks.workunit.client.0.smithi032.stdout:4/538: chown d4/db/da/d5d/l7a 1184 1 2024-06-03T01:21:06.939 INFO:tasks.workunit.client.0.smithi032.stdout:4/539: chown d4/db/da/f3f 0 1 2024-06-03T01:21:06.942 INFO:tasks.workunit.client.0.smithi032.stdout:2/583: mkdir d5/d38/d46/d53/dbf 0 2024-06-03T01:21:06.945 INFO:tasks.workunit.client.0.smithi032.stdout:9/639: creat d0/d11/fd2 x:0 0 0 2024-06-03T01:21:06.946 INFO:tasks.workunit.client.0.smithi032.stdout:0/574: dwrite d1/f17 [0,4194304] 0 2024-06-03T01:21:06.947 INFO:tasks.workunit.client.0.smithi032.stdout:1/523: truncate d18/d3f/d5f/d6b/d78/f45 2854152 0 2024-06-03T01:21:06.948 INFO:tasks.workunit.client.0.smithi032.stdout:1/524: write d18/d3f/d5f/d6b/d78/d3d/d47/f51 [1729497,8708] 0 2024-06-03T01:21:06.948 INFO:tasks.workunit.client.0.smithi032.stdout:1/525: stat l4 0 2024-06-03T01:21:06.948 INFO:tasks.workunit.client.0.smithi032.stdout:1/526: chown l4 0 1 2024-06-03T01:21:06.950 INFO:tasks.workunit.client.0.smithi032.stdout:2/584: symlink d5/d38/d46/d53/da9/lc0 0 2024-06-03T01:21:06.950 INFO:tasks.workunit.client.0.smithi032.stdout:2/585: getdents d5/d38/d46/dbe 0 2024-06-03T01:21:06.952 INFO:tasks.workunit.client.0.smithi032.stdout:8/436: mknod dc/d14/c92 0 2024-06-03T01:21:06.954 INFO:tasks.workunit.client.0.smithi032.stdout:2/586: dread d5/d38/d46/d53/d57/d6d/f98 [0,4194304] 0 2024-06-03T01:21:06.954 INFO:tasks.workunit.client.0.smithi032.stdout:2/587: write d5/d38/d46/d53/d57/dac/d86/f99 [414219,4853] 0 2024-06-03T01:21:06.954 INFO:tasks.workunit.client.0.smithi032.stdout:9/640: write d0/d11/d1d/d38/d4f/db0/fa0 [3351247,72678] 0 2024-06-03T01:21:06.955 INFO:tasks.workunit.client.0.smithi032.stdout:0/575: getdents d1/da/d28/d3b/d50 0 2024-06-03T01:21:06.955 INFO:tasks.workunit.client.0.smithi032.stdout:0/576: readlink d1/da/d28/d3b/d5d/da9/daa/lad 0 2024-06-03T01:21:06.955 INFO:tasks.workunit.client.0.smithi032.stdout:0/577: write d1/faf [334021,127211] 0 2024-06-03T01:21:06.955 INFO:tasks.workunit.client.0.smithi032.stdout:0/578: chown d1/da/d28/d3b/d50/f58 2876 1 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/588: symlink d5/da/d1a/d2a/d49/d67/lc1 0 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/589: write d5/da/d1a/d2a/d49/fb1 [430403,81840] 0 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/590: readlink d5/d38/d46/d53/da9/lc0 0 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/591: chown d5/d38/d46/d53 2 1 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/592: write d5/da/d1a/d2a/d49/fb1 [694998,92482] 0 2024-06-03T01:21:06.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/593: fsync d5/da/d1a/d2a/f50 0 2024-06-03T01:21:06.960 INFO:tasks.workunit.client.0.smithi032.stdout:4/540: dwrite d4/db/da/d37/f84 [0,4194304] 0 2024-06-03T01:21:06.960 INFO:tasks.workunit.client.0.smithi032.stdout:9/641: creat d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/fd3 x:0 0 0 2024-06-03T01:21:06.961 INFO:tasks.workunit.client.0.smithi032.stdout:0/579: symlink d1/da/db2/lb8 0 2024-06-03T01:21:06.964 INFO:tasks.workunit.client.0.smithi032.stdout:2/594: mknod d5/d38/d46/d53/da9/cc2 0 2024-06-03T01:21:06.965 INFO:tasks.workunit.client.0.smithi032.stdout:4/541: symlink d4/db/da/d37/lbf 0 2024-06-03T01:21:06.966 INFO:tasks.workunit.client.0.smithi032.stdout:8/437: write dc/d25/d28/d32/f36 [1504813,15408] 0 2024-06-03T01:21:06.967 INFO:tasks.workunit.client.0.smithi032.stdout:9/642: mkdir d0/d11/d9c/dd4 0 2024-06-03T01:21:06.971 INFO:tasks.workunit.client.0.smithi032.stdout:0/580: rename d1/d46/d3a/f6d to d1/da/d10/fb9 0 2024-06-03T01:21:06.972 INFO:tasks.workunit.client.0.smithi032.stdout:8/438: dread dc/d25/f88 [0,4194304] 0 2024-06-03T01:21:06.973 INFO:tasks.workunit.client.0.smithi032.stdout:2/595: mkdir d5/da/d1a/d2a/d36/dc3 0 2024-06-03T01:21:06.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/542: getdents d4/db/da/d37/d90/da1 0 2024-06-03T01:21:06.976 INFO:tasks.workunit.client.0.smithi032.stdout:4/543: readlink d4/l5 0 2024-06-03T01:21:06.976 INFO:tasks.workunit.client.0.smithi032.stdout:4/544: write d4/db/da/fb6 [455387,104711] 0 2024-06-03T01:21:06.980 INFO:tasks.workunit.client.0.smithi032.stdout:2/596: creat d5/d38/d46/d53/da9/fc4 x:0 0 0 2024-06-03T01:21:06.982 INFO:tasks.workunit.client.0.smithi032.stdout:9/643: dread d0/d23/d25/f8e [0,4194304] 0 2024-06-03T01:21:06.982 INFO:tasks.workunit.client.0.smithi032.stdout:8/439: rmdir dc/d25/d28/d51 39 2024-06-03T01:21:06.983 INFO:tasks.workunit.client.0.smithi032.stdout:2/597: truncate d5/f3c 685787 0 2024-06-03T01:21:06.985 INFO:tasks.workunit.client.0.smithi032.stdout:8/440: mkdir dc/d93 0 2024-06-03T01:21:06.985 INFO:tasks.workunit.client.0.smithi032.stdout:8/441: write dc/d25/d28/d32/f50 [128247,118943] 0 2024-06-03T01:21:06.985 INFO:tasks.workunit.client.0.smithi032.stdout:9/644: write d0/f6 [4424994,86703] 0 2024-06-03T01:21:06.988 INFO:tasks.workunit.client.0.smithi032.stdout:2/598: unlink d5/c40 0 2024-06-03T01:21:06.988 INFO:tasks.workunit.client.0.smithi032.stdout:2/599: stat d5/d38/l42 0 2024-06-03T01:21:06.991 INFO:tasks.workunit.client.0.smithi032.stdout:8/442: rename dc/d14/d31/d3b/d80 to dc/d25/d28/d58/d8d/d94 0 2024-06-03T01:21:06.991 INFO:tasks.workunit.client.0.smithi032.stdout:2/600: truncate d5/da/f2f 6159871 0 2024-06-03T01:21:06.991 INFO:tasks.workunit.client.0.smithi032.stdout:2/601: truncate d5/da/d1a/d69/d83/fa1 180202 0 2024-06-03T01:21:06.994 INFO:tasks.workunit.client.0.smithi032.stdout:9/645: write d0/d23/f3d [1880843,9131] 0 2024-06-03T01:21:06.995 INFO:tasks.workunit.client.0.smithi032.stdout:9/646: stat d0/da4/lab 0 2024-06-03T01:21:06.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/602: link d5/da/f1e d5/d38/d46/d5c/fc5 0 2024-06-03T01:21:06.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/603: write d5/da/f1e [6238892,102608] 0 2024-06-03T01:21:06.998 INFO:tasks.workunit.client.0.smithi032.stdout:9/647: mkdir d0/d11/d1d/dd5 0 2024-06-03T01:21:07.001 INFO:tasks.workunit.client.0.smithi032.stdout:2/604: mknod d5/d38/d46/d53/da9/cc6 0 2024-06-03T01:21:07.001 INFO:tasks.workunit.client.0.smithi032.stdout:0/581: dwrite d1/f21 [0,4194304] 0 2024-06-03T01:21:07.007 INFO:tasks.workunit.client.0.smithi032.stdout:1/527: dwrite d18/d3f/d5f/d6b/d78/f2e [0,4194304] 0 2024-06-03T01:21:07.008 INFO:tasks.workunit.client.0.smithi032.stdout:4/545: dwrite d4/f6 [0,4194304] 0 2024-06-03T01:21:07.008 INFO:tasks.workunit.client.0.smithi032.stdout:9/648: dread d0/f57 [0,4194304] 0 2024-06-03T01:21:07.009 INFO:tasks.workunit.client.0.smithi032.stdout:9/649: fdatasync d0/d23/f85 0 2024-06-03T01:21:07.016 INFO:tasks.workunit.client.0.smithi032.stdout:4/546: symlink d4/db/da/d5d/lc0 0 2024-06-03T01:21:07.021 INFO:tasks.workunit.client.0.smithi032.stdout:9/650: mkdir d0/d23/d25/dd6 0 2024-06-03T01:21:07.022 INFO:tasks.workunit.client.0.smithi032.stdout:1/528: write d18/d32/f52 [5346886,8540] 0 2024-06-03T01:21:07.022 INFO:tasks.workunit.client.0.smithi032.stdout:2/605: dread d5/da/f1e [4194304,4194304] 0 2024-06-03T01:21:07.026 INFO:tasks.workunit.client.0.smithi032.stdout:0/582: dwrite d1/da/f1a [0,4194304] 0 2024-06-03T01:21:07.027 INFO:tasks.workunit.client.0.smithi032.stdout:0/583: fdatasync d1/da/d28/d3b/d5d/fa5 0 2024-06-03T01:21:07.027 INFO:tasks.workunit.client.0.smithi032.stdout:8/443: dwrite dc/de/f91 [0,4194304] 0 2024-06-03T01:21:07.029 INFO:tasks.workunit.client.0.smithi032.stdout:4/547: mknod d4/d57/cc1 0 2024-06-03T01:21:07.037 INFO:tasks.workunit.client.0.smithi032.stdout:1/529: mkdir d18/d3f/d5f/d94 0 2024-06-03T01:21:07.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/606: symlink d5/d38/d46/d5c/d6c/d80/lc7 0 2024-06-03T01:21:07.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/607: fdatasync d5/d38/d46/d53/d57/dac/f73 0 2024-06-03T01:21:07.042 INFO:tasks.workunit.client.0.smithi032.stdout:8/444: dwrite dc/d25/d28/d3c/f5a [0,4194304] 0 2024-06-03T01:21:07.044 INFO:tasks.workunit.client.0.smithi032.stdout:0/584: rename d1/da/d28/l90 to d1/da/d28/lba 0 2024-06-03T01:21:07.052 INFO:tasks.workunit.client.0.smithi032.stdout:4/548: symlink d4/d78/lc2 0 2024-06-03T01:21:07.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/608: symlink d5/d38/db5/lc8 0 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/585: symlink d1/da/d10/d80/d9f/dac/lbb 0 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/586: fsync d1/faf 0 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/587: readlink d1/da/d28/d3b/d5d/d8d/l92 0 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/588: creat d1/da/d10/d29/fbc x:0 0 0 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/589: chown d1/da/d10/d29/l36 317282 1 2024-06-03T01:21:07.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/590: write d1/da/d28/d3b/f76 [1254149,39450] 0 2024-06-03T01:21:07.080 INFO:tasks.workunit.client.0.smithi032.stdout:0/591: write d1/f43 [1369118,52722] 0 2024-06-03T01:21:07.080 INFO:tasks.workunit.client.0.smithi032.stdout:0/592: fdatasync d1/d46/f48 0 2024-06-03T01:21:07.117 INFO:tasks.workunit.client.0.smithi032.stdout:4/549: dwrite d4/f6 [4194304,4194304] 0 2024-06-03T01:21:07.117 INFO:tasks.workunit.client.0.smithi032.stdout:4/550: dread - d4/db/da/d2d/d27/f89 zero size 2024-06-03T01:21:07.117 INFO:tasks.workunit.client.0.smithi032.stdout:4/551: creat d4/db/da/fc3 x:0 0 0 2024-06-03T01:21:07.118 INFO:tasks.workunit.client.0.smithi032.stdout:4/552: read d4/db/da/f1b [490779,95132] 0 2024-06-03T01:21:07.126 INFO:tasks.workunit.client.0.smithi032.stdout:0/593: truncate d1/da/d10/f6f 4075525 0 2024-06-03T01:21:07.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/530: truncate f17 1380099 0 2024-06-03T01:21:07.136 INFO:tasks.workunit.client.0.smithi032.stdout:0/594: write d1/d46/f48 [3197308,109450] 0 2024-06-03T01:21:07.136 INFO:tasks.workunit.client.0.smithi032.stdout:4/553: creat d4/db/da/d2d/d27/d8c/db5/db3/fc4 x:0 0 0 2024-06-03T01:21:07.137 INFO:tasks.workunit.client.0.smithi032.stdout:2/609: mknod d5/da/d1a/d2a/d49/d8f/cc9 0 2024-06-03T01:21:07.137 INFO:tasks.workunit.client.0.smithi032.stdout:2/610: creat d5/da/d1a/d2a/d49/fca x:0 0 0 2024-06-03T01:21:07.138 INFO:tasks.workunit.client.0.smithi032.stdout:2/611: write d5/f58 [347232,115044] 0 2024-06-03T01:21:07.138 INFO:tasks.workunit.client.0.smithi032.stdout:2/612: dread - d5/da/d1a/d2a/d36/d3f/f84 zero size 2024-06-03T01:21:07.139 INFO:tasks.workunit.client.0.smithi032.stdout:4/554: read d4/db/da/fb6 [5304,42049] 0 2024-06-03T01:21:07.139 INFO:tasks.workunit.client.0.smithi032.stdout:4/555: write d4/d78/fbe [840237,58263] 0 2024-06-03T01:21:07.141 INFO:tasks.workunit.client.0.smithi032.stdout:1/531: rename d18/d27/d2b/d5b to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/d95 0 2024-06-03T01:21:07.144 INFO:tasks.workunit.client.0.smithi032.stdout:0/595: creat d1/da/d10/d80/d9f/dac/fbd x:0 0 0 2024-06-03T01:21:07.148 INFO:tasks.workunit.client.0.smithi032.stdout:2/613: creat d5/da/d1a/d2a/fcb x:0 0 0 2024-06-03T01:21:07.152 INFO:tasks.workunit.client.0.smithi032.stdout:4/556: mkdir d4/db/da/d5d/dc5 0 2024-06-03T01:21:07.152 INFO:tasks.workunit.client.0.smithi032.stdout:5/657: sync 2024-06-03T01:21:07.154 INFO:tasks.workunit.client.0.smithi032.stdout:0/596: mkdir d1/da/d28/d3b/d5d/d8d/dbe 0 2024-06-03T01:21:07.157 INFO:tasks.workunit.client.0.smithi032.stdout:2/614: symlink d5/d38/db4/lcc 0 2024-06-03T01:21:07.165 INFO:tasks.workunit.client.0.smithi032.stdout:5/658: rename da/dd/df/d10/dbc/d18/d35/d44/f8f to da/dd/df/d10/dbc/d18/db9/fd5 0 2024-06-03T01:21:07.167 INFO:tasks.workunit.client.0.smithi032.stdout:4/557: symlink d4/db/da/d2d/d27/d8c/db5/lc6 0 2024-06-03T01:21:07.169 INFO:tasks.workunit.client.0.smithi032.stdout:0/597: rename d1/da/f8f to d1/da/d28/d3b/d68/fbf 0 2024-06-03T01:21:07.173 INFO:tasks.workunit.client.0.smithi032.stdout:4/558: truncate d4/db/da/f2f 1907182 0 2024-06-03T01:21:07.178 INFO:tasks.workunit.client.0.smithi032.stdout:5/659: dread da/dd/df/d10/dbc/d18/d35/f5d [0,4194304] 0 2024-06-03T01:21:07.178 INFO:tasks.workunit.client.0.smithi032.stdout:5/660: chown da/dd/df/d10/dbc/d18/d35/d44/dc3/l6b 888 1 2024-06-03T01:21:07.181 INFO:tasks.workunit.client.0.smithi032.stdout:5/661: write da/dd/df/d10/f4d [1133580,118870] 0 2024-06-03T01:21:07.181 INFO:tasks.workunit.client.0.smithi032.stdout:5/662: fdatasync da/dd/df/d10/dbc/d18/fc1 0 2024-06-03T01:21:07.182 INFO:tasks.workunit.client.0.smithi032.stdout:2/615: dwrite d5/da/f2c [0,4194304] 0 2024-06-03T01:21:07.184 INFO:tasks.workunit.client.0.smithi032.stdout:0/598: write d1/da/f1a [2481551,57218] 0 2024-06-03T01:21:07.184 INFO:tasks.workunit.client.0.smithi032.stdout:0/599: fdatasync d1/da/d28/d3b/d68/d82/f9a 0 2024-06-03T01:21:07.184 INFO:tasks.workunit.client.0.smithi032.stdout:0/600: getdents d1/da/d28/d3b/d50 0 2024-06-03T01:21:07.184 INFO:tasks.workunit.client.0.smithi032.stdout:0/601: read d1/da/d10/d29/f96 [725495,4110] 0 2024-06-03T01:21:07.187 INFO:tasks.workunit.client.0.smithi032.stdout:5/663: unlink da/dd/df/d10/dbc/d18/l76 0 2024-06-03T01:21:07.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/664: unlink da/dd/df/c11 0 2024-06-03T01:21:07.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/559: dwrite d4/db/ff [0,4194304] 0 2024-06-03T01:21:07.195 INFO:tasks.workunit.client.0.smithi032.stdout:0/602: dread d1/da/d28/d3b/d68/fa3 [0,4194304] 0 2024-06-03T01:21:07.195 INFO:tasks.workunit.client.0.smithi032.stdout:0/603: chown d1/da/d10/c24 40645 1 2024-06-03T01:21:07.197 INFO:tasks.workunit.client.0.smithi032.stdout:0/604: symlink d1/da/d28/d3b/d68/d83/lc0 0 2024-06-03T01:21:07.199 INFO:tasks.workunit.client.0.smithi032.stdout:2/616: dread d5/f58 [0,4194304] 0 2024-06-03T01:21:07.201 INFO:tasks.workunit.client.0.smithi032.stdout:4/560: dread d4/db/da/f82 [0,4194304] 0 2024-06-03T01:21:07.201 INFO:tasks.workunit.client.0.smithi032.stdout:4/561: write d4/db/f51 [856213,18645] 0 2024-06-03T01:21:07.202 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:06 smithi044 ceph-mon[36956]: pgmap v8: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 193 MiB/s rd, 230 MiB/s wr, 324 op/s 2024-06-03T01:21:07.204 INFO:tasks.workunit.client.0.smithi032.stdout:2/617: fsync d5/da/d1a/f3e 0 2024-06-03T01:21:07.204 INFO:tasks.workunit.client.0.smithi032.stdout:5/665: write da/dd/df/d10/dbc/d25/d74/f93 [2352425,80690] 0 2024-06-03T01:21:07.206 INFO:tasks.workunit.client.0.smithi032.stdout:4/562: creat d4/db/da/d2d/fc7 x:0 0 0 2024-06-03T01:21:07.208 INFO:tasks.workunit.client.0.smithi032.stdout:0/605: write d1/da/d10/f4f [1101056,85252] 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/606: write d1/da/d28/d3b/d68/fa3 [3080821,60285] 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/607: dread - d1/d46/f98 zero size 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/608: write d1/da/d28/d3b/d5d/f94 [1282576,95223] 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/609: fdatasync d1/da/d10/d29/fbc 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/610: read d1/faf [370012,34283] 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:2/618: symlink d5/da/d1a/d2a/lcd 0 2024-06-03T01:21:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:2/619: chown d5/f6b 160 1 2024-06-03T01:21:07.210 INFO:tasks.workunit.client.0.smithi032.stdout:5/666: unlink da/dd/df/d10/dbc/d18/d35/d44/dc3/fb2 0 2024-06-03T01:21:07.211 INFO:tasks.workunit.client.0.smithi032.stdout:5/667: write f4 [4316006,66920] 0 2024-06-03T01:21:07.212 INFO:tasks.workunit.client.0.smithi032.stdout:4/563: symlink d4/lc8 0 2024-06-03T01:21:07.218 INFO:tasks.workunit.client.0.smithi032.stdout:0/611: rename d1/da/d10/d29/d6c/c72 to d1/cc1 0 2024-06-03T01:21:07.228 INFO:tasks.workunit.client.0.smithi032.stdout:0/612: getdents d1 0 2024-06-03T01:21:07.230 INFO:tasks.workunit.client.0.smithi032.stdout:0/613: mkdir d1/da/d28/d3b/d5d/d8d/d9c/dc2 0 2024-06-03T01:21:07.232 INFO:tasks.workunit.client.0.smithi032.stdout:0/614: mkdir d1/d46/dc3 0 2024-06-03T01:21:07.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/615: dread d1/f18 [0,4194304] 0 2024-06-03T01:21:07.236 INFO:tasks.workunit.client.0.smithi032.stdout:0/616: readlink d1/da/d10/l2e 0 2024-06-03T01:21:07.242 INFO:tasks.workunit.client.0.smithi032.stdout:5/668: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/fb5 [0,4194304] 0 2024-06-03T01:21:07.242 INFO:tasks.workunit.client.0.smithi032.stdout:5/669: stat da/dd/df/d10/dbc/d18/f69 0 2024-06-03T01:21:07.242 INFO:tasks.workunit.client.0.smithi032.stdout:4/564: dread d4/db/f68 [0,4194304] 0 2024-06-03T01:21:07.242 INFO:tasks.workunit.client.0.smithi032.stdout:2/620: dwrite d5/da/d1a/d2a/d36/d3f/f4f [0,4194304] 0 2024-06-03T01:21:07.243 INFO:tasks.workunit.client.0.smithi032.stdout:5/670: mknod da/dd/df/d10/dbc/d18/d1a/d30/d51/dc8/cd6 0 2024-06-03T01:21:07.245 INFO:tasks.workunit.client.0.smithi032.stdout:4/565: creat d4/d57/d6c/fc9 x:0 0 0 2024-06-03T01:21:07.245 INFO:tasks.workunit.client.0.smithi032.stdout:4/566: write d4/db/da/fb6 [335348,47744] 0 2024-06-03T01:21:07.246 INFO:tasks.workunit.client.0.smithi032.stdout:2/621: truncate d5/f1c 1172982 0 2024-06-03T01:21:07.248 INFO:tasks.workunit.client.0.smithi032.stdout:5/671: mkdir da/dd/df/d10/dd7 0 2024-06-03T01:21:07.256 INFO:tasks.workunit.client.0.smithi032.stdout:2/622: rename d5/da/d1a/d2a/d49 to d5/d38/d46/d53/d57/d6d/dce 0 2024-06-03T01:21:07.256 INFO:tasks.workunit.client.0.smithi032.stdout:2/623: write d5/da/f59 [915090,36434] 0 2024-06-03T01:21:07.259 INFO:tasks.workunit.client.0.smithi032.stdout:5/672: rename da/dd/df/d10/dbc/d18/d1a/d30/d51/fba to da/dd/df/d10/dbc/d18/d35/d70/fd8 0 2024-06-03T01:21:07.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/617: dwrite d1/f6 [0,4194304] 0 2024-06-03T01:21:07.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/618: chown d1/lf 66761069 1 2024-06-03T01:21:07.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/619: write d1/da/d10/d29/fbc [507535,126143] 0 2024-06-03T01:21:07.261 INFO:tasks.workunit.client.0.smithi032.stdout:2/624: write d5/da/d1a/d69/d83/f96 [711307,93835] 0 2024-06-03T01:21:07.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/625: creat d5/d38/db5/fcf x:0 0 0 2024-06-03T01:21:07.265 INFO:tasks.workunit.client.0.smithi032.stdout:0/620: dread d1/da/d10/d29/fbc [0,4194304] 0 2024-06-03T01:21:07.265 INFO:tasks.workunit.client.0.smithi032.stdout:0/621: dread - d1/da/d10/d29/f45 zero size 2024-06-03T01:21:07.268 INFO:tasks.workunit.client.0.smithi032.stdout:0/622: truncate d1/da/d28/d3b/d5d/f65 1667838 0 2024-06-03T01:21:07.269 INFO:tasks.workunit.client.0.smithi032.stdout:5/673: dread da/dd/df/d10/dbc/fa9 [0,4194304] 0 2024-06-03T01:21:07.269 INFO:tasks.workunit.client.0.smithi032.stdout:5/674: truncate da/dd/df/d10/dbc/d18/d35/d44/fc5 306182 0 2024-06-03T01:21:07.269 INFO:tasks.workunit.client.0.smithi032.stdout:4/567: dwrite d4/d57/f9c [0,4194304] 0 2024-06-03T01:21:07.271 INFO:tasks.workunit.client.0.smithi032.stdout:2/626: write d5/da/d1a/f6f [259402,80347] 0 2024-06-03T01:21:07.273 INFO:tasks.workunit.client.0.smithi032.stdout:4/568: rename d4/f41 to d4/db/da/d2d/fca 0 2024-06-03T01:21:07.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/569: rmdir d4/d7f 39 2024-06-03T01:21:07.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/570: chown d4/db/da/c12 1467417332 1 2024-06-03T01:21:07.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/571: fsync d4/db/f2e 0 2024-06-03T01:21:07.283 INFO:tasks.workunit.client.0.smithi032.stdout:4/572: write d4/d57/d6c/f6e [44909,33593] 0 2024-06-03T01:21:07.285 INFO:tasks.workunit.client.0.smithi032.stdout:4/573: link d4/db/da/d37/c4a d4/db/da/d37/d90/ccb 0 2024-06-03T01:21:07.285 INFO:tasks.workunit.client.0.smithi032.stdout:4/574: dread - d4/db/da/d2d/d27/f89 zero size 2024-06-03T01:21:07.287 INFO:tasks.workunit.client.0.smithi032.stdout:4/575: creat d4/fcc x:0 0 0 2024-06-03T01:21:07.287 INFO:tasks.workunit.client.0.smithi032.stdout:4/576: creat d4/db/fcd x:0 0 0 2024-06-03T01:21:07.287 INFO:tasks.workunit.client.0.smithi032.stdout:4/577: chown d4/d57/c74 3914563 1 2024-06-03T01:21:07.288 INFO:tasks.workunit.client.0.smithi032.stdout:4/578: chown d4/db/da/d2d/d27/d8c/db5 119144173 1 2024-06-03T01:21:07.290 INFO:tasks.workunit.client.0.smithi032.stdout:4/579: mknod d4/d7f/cce 0 2024-06-03T01:21:07.290 INFO:tasks.workunit.client.0.smithi032.stdout:4/580: creat d4/d57/d6c/fcf x:0 0 0 2024-06-03T01:21:07.293 INFO:tasks.workunit.client.0.smithi032.stdout:4/581: unlink d4/db/da/d2d/d27/f2b 0 2024-06-03T01:21:07.295 INFO:tasks.workunit.client.0.smithi032.stdout:4/582: getdents d4/db/da/d2d/d27/d8c/db5/db3 0 2024-06-03T01:21:07.295 INFO:tasks.workunit.client.0.smithi032.stdout:4/583: write d4/db/da/d37/f3b [250134,41040] 0 2024-06-03T01:21:07.297 INFO:tasks.workunit.client.0.smithi032.stdout:4/584: mknod d4/db/da/d37/cd0 0 2024-06-03T01:21:07.297 INFO:tasks.workunit.client.0.smithi032.stdout:4/585: fdatasync d4/db/fcd 0 2024-06-03T01:21:07.297 INFO:tasks.workunit.client.0.smithi032.stdout:0/623: dwrite d1/da/d28/d3b/d68/fa3 [4194304,4194304] 0 2024-06-03T01:21:07.297 INFO:tasks.workunit.client.0.smithi032.stdout:0/624: chown d1/da/d10/l22 97651354 1 2024-06-03T01:21:07.298 INFO:tasks.workunit.client.0.smithi032.stdout:5/675: dwrite da/dd/df/d10/dbc/d18/db9/fd5 [4194304,4194304] 0 2024-06-03T01:21:07.298 INFO:tasks.workunit.client.0.smithi032.stdout:2/627: dwrite d5/da/d1a/d2a/d36/d3f/f63 [0,4194304] 0 2024-06-03T01:21:07.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/676: unlink da/c26 0 2024-06-03T01:21:07.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/677: creat da/dd/df/d10/dbc/d18/d35/d44/fd9 x:0 0 0 2024-06-03T01:21:07.305 INFO:tasks.workunit.client.0.smithi032.stdout:4/586: read d4/d57/f9c [1745939,73728] 0 2024-06-03T01:21:07.318 INFO:tasks.workunit.client.0.smithi032.stdout:2/628: mkdir d5/da/d1a/d2a/da8/dd0 0 2024-06-03T01:21:07.320 INFO:tasks.workunit.client.0.smithi032.stdout:5/678: creat da/dd/df/d10/dbc/d18/fda x:0 0 0 2024-06-03T01:21:07.322 INFO:tasks.workunit.client.0.smithi032.stdout:5/679: rmdir da/dd/df/d10/dbc/d18/d1a 39 2024-06-03T01:21:07.323 INFO:tasks.workunit.client.0.smithi032.stdout:4/587: getdents d4/db 0 2024-06-03T01:21:07.325 INFO:tasks.workunit.client.0.smithi032.stdout:5/680: link da/dd/df/d10/dbc/d25/lc9 da/dab/dcd/ldb 0 2024-06-03T01:21:07.326 INFO:tasks.workunit.client.0.smithi032.stdout:4/588: symlink d4/db/da/d2d/d27/d8c/db5/ld1 0 2024-06-03T01:21:07.328 INFO:tasks.workunit.client.0.smithi032.stdout:5/681: unlink da/dd/df/d10/dbc/d18/f69 0 2024-06-03T01:21:07.330 INFO:tasks.workunit.client.0.smithi032.stdout:4/589: symlink d4/db/da/d5d/ld2 0 2024-06-03T01:21:07.331 INFO:tasks.workunit.client.0.smithi032.stdout:5/682: symlink da/dd/df/d10/dbc/d18/d35/d44/ldc 0 2024-06-03T01:21:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:06 smithi032 ceph-mon[29360]: pgmap v8: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 193 MiB/s rd, 230 MiB/s wr, 324 op/s 2024-06-03T01:21:07.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/590: dread d4/db/da/d1d/f4c [0,4194304] 0 2024-06-03T01:21:07.337 INFO:tasks.workunit.client.0.smithi032.stdout:4/591: creat d4/db/da/d5d/dc5/fd3 x:0 0 0 2024-06-03T01:21:07.344 INFO:tasks.workunit.client.0.smithi032.stdout:2/629: dwrite d5/da/d1a/d69/d83/fb2 [0,4194304] 0 2024-06-03T01:21:07.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/630: read d5/da/f59 [344518,62287] 0 2024-06-03T01:21:07.358 INFO:tasks.workunit.client.0.smithi032.stdout:5/683: dwrite da/dd/df/d10/dbc/d25/d74/f93 [0,4194304] 0 2024-06-03T01:21:07.358 INFO:tasks.workunit.client.0.smithi032.stdout:5/684: fsync da/dd/df/d10/f39 0 2024-06-03T01:21:07.358 INFO:tasks.workunit.client.0.smithi032.stdout:5/685: dread - da/dd/fd3 zero size 2024-06-03T01:21:07.359 INFO:tasks.workunit.client.0.smithi032.stdout:2/631: dread d5/da/d1a/fa5 [0,4194304] 0 2024-06-03T01:21:07.359 INFO:tasks.workunit.client.0.smithi032.stdout:4/592: dwrite d4/db/f1c [0,4194304] 0 2024-06-03T01:21:07.359 INFO:tasks.workunit.client.0.smithi032.stdout:5/686: mknod da/dab/dcd/cdd 0 2024-06-03T01:21:07.361 INFO:tasks.workunit.client.0.smithi032.stdout:2/632: creat d5/d38/d46/d5c/d6c/fd1 x:0 0 0 2024-06-03T01:21:07.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/687: creat da/dd/df/d10/dbc/d25/fde x:0 0 0 2024-06-03T01:21:07.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/688: dread - da/dd/df/d10/dbc/d18/d35/d70/f84 zero size 2024-06-03T01:21:07.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/689: write da/dd/f68 [3073636,70616] 0 2024-06-03T01:21:07.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/690: chown da/dd/df/d10/dbc/d18/d35/d44/dc3/f7f 5 1 2024-06-03T01:21:07.369 INFO:tasks.workunit.client.0.smithi032.stdout:2/633: dread d5/da/f74 [0,4194304] 0 2024-06-03T01:21:07.371 INFO:tasks.workunit.client.0.smithi032.stdout:2/634: mknod d5/d38/d46/d53/d57/d6d/dce/d67/cd2 0 2024-06-03T01:21:07.372 INFO:tasks.workunit.client.0.smithi032.stdout:2/635: link d5/d38/d46/d53/d57/dac/d86/f99 d5/da/fd3 0 2024-06-03T01:21:07.384 INFO:tasks.workunit.client.0.smithi032.stdout:2/636: dread d5/ff [0,4194304] 0 2024-06-03T01:21:07.385 INFO:tasks.workunit.client.0.smithi032.stdout:2/637: truncate d5/d38/db5/fcf 241991 0 2024-06-03T01:21:07.385 INFO:tasks.workunit.client.0.smithi032.stdout:2/638: dread d5/da/d1a/d2a/f50 [0,4194304] 0 2024-06-03T01:21:07.386 INFO:tasks.workunit.client.0.smithi032.stdout:2/639: rmdir d5/da/d1a/d2a/da8/dd0 0 2024-06-03T01:21:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:2/640: rename d5/d38/d46/d53/d57/d6d/dce/d8f/cc9 to d5/d38/d46/d5c/d6c/d80/cd4 0 2024-06-03T01:21:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:4/593: dwrite d4/db/da/d1d/f7e [0,4194304] 0 2024-06-03T01:21:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:5/691: dwrite da/dd/df/d10/dbc/d18/d35/d70/fb3 [0,4194304] 0 2024-06-03T01:21:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:5/692: fdatasync da/dd/df/d10/dbc/d18/d1a/d30/d51/fa2 0 2024-06-03T01:21:07.390 INFO:tasks.workunit.client.0.smithi032.stdout:2/641: symlink d5/d38/d46/d53/d57/d6d/ld5 0 2024-06-03T01:21:07.392 INFO:tasks.workunit.client.0.smithi032.stdout:4/594: rename d4/db/f51 to d4/db/da/d5d/fd4 0 2024-06-03T01:21:07.392 INFO:tasks.workunit.client.0.smithi032.stdout:4/595: dread - d4/d7f/f92 zero size 2024-06-03T01:21:07.394 INFO:tasks.workunit.client.0.smithi032.stdout:5/693: chown da/dd/df/f5c 248252552 1 2024-06-03T01:21:07.394 INFO:tasks.workunit.client.0.smithi032.stdout:5/694: stat da/dd/df/d10/dbc/f8a 0 2024-06-03T01:21:07.394 INFO:tasks.workunit.client.0.smithi032.stdout:5/695: stat da/dd/df/d10/dbc 0 2024-06-03T01:21:07.396 INFO:tasks.workunit.client.0.smithi032.stdout:2/642: creat d5/da/d1a/d2a/d36/d3f/fd6 x:0 0 0 2024-06-03T01:21:07.396 INFO:tasks.workunit.client.0.smithi032.stdout:2/643: chown d5/d38/d46/d53/d90/fa3 884190375 1 2024-06-03T01:21:07.398 INFO:tasks.workunit.client.0.smithi032.stdout:4/596: rmdir d4/db/da/d5d 39 2024-06-03T01:21:07.401 INFO:tasks.workunit.client.0.smithi032.stdout:5/696: rename da/dd/df/d10/dbc/d18/d35/d44/f4f to da/dd/fdf 0 2024-06-03T01:21:07.402 INFO:tasks.workunit.client.0.smithi032.stdout:7/605: sync 2024-06-03T01:21:07.402 INFO:tasks.workunit.client.0.smithi032.stdout:7/606: write dc/d12/f77 [935264,57273] 0 2024-06-03T01:21:07.403 INFO:tasks.workunit.client.0.smithi032.stdout:2/644: symlink d5/d38/d46/d53/d57/dac/d86/ld7 0 2024-06-03T01:21:07.403 INFO:tasks.workunit.client.0.smithi032.stdout:2/645: write d5/da/d2e/f61 [91735,130364] 0 2024-06-03T01:21:07.404 INFO:tasks.workunit.client.0.smithi032.stdout:5/697: truncate da/dd/df/f46 9341149 0 2024-06-03T01:21:07.408 INFO:tasks.workunit.client.0.smithi032.stdout:4/597: rename d4/db/da/d1d/f7e to d4/db/da/d2d/d27/d8c/d9a/fd5 0 2024-06-03T01:21:07.408 INFO:tasks.workunit.client.0.smithi032.stdout:4/598: chown d4/db/da/d2d/d27/d8c/d9a/la4 40898849 1 2024-06-03T01:21:07.411 INFO:tasks.workunit.client.0.smithi032.stdout:7/607: mkdir dc/dbe 0 2024-06-03T01:21:07.415 INFO:tasks.workunit.client.0.smithi032.stdout:2/646: mknod d5/da/d1a/d2a/da8/db6/cd8 0 2024-06-03T01:21:07.415 INFO:tasks.workunit.client.0.smithi032.stdout:2/647: creat d5/d38/d46/d53/da9/fd9 x:0 0 0 2024-06-03T01:21:07.417 INFO:tasks.workunit.client.0.smithi032.stdout:4/599: dread d4/db/da/f2c [0,4194304] 0 2024-06-03T01:21:07.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/648: unlink d5/d38/d46/d53/da9/fc4 0 2024-06-03T01:21:07.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/649: stat d5/l2d 0 2024-06-03T01:21:07.420 INFO:tasks.workunit.client.0.smithi032.stdout:5/698: dwrite da/dd/df/d10/dbc/d18/d35/d70/fb3 [0,4194304] 0 2024-06-03T01:21:07.420 INFO:tasks.workunit.client.0.smithi032.stdout:5/699: write da/dd/fd3 [262271,117888] 0 2024-06-03T01:21:07.424 INFO:tasks.workunit.client.0.smithi032.stdout:4/600: mknod d4/d57/cd6 0 2024-06-03T01:21:07.424 INFO:tasks.workunit.client.0.smithi032.stdout:2/650: mknod d5/d38/cda 0 2024-06-03T01:21:07.424 INFO:tasks.workunit.client.0.smithi032.stdout:2/651: chown d5/d38/d46/d53/d57/d6d/dce/d67/lc1 4515220 1 2024-06-03T01:21:07.428 INFO:tasks.workunit.client.0.smithi032.stdout:2/652: rmdir d5/d38/d46/d53/dbf 0 2024-06-03T01:21:07.431 INFO:tasks.workunit.client.0.smithi032.stdout:7/608: dwrite f6 [4194304,4194304] 0 2024-06-03T01:21:07.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/609: readlink dc/d12/d2a/d2f/l73 0 2024-06-03T01:21:07.432 INFO:tasks.workunit.client.0.smithi032.stdout:2/653: rename d5/da/d1a/d2a/d36/f60 to d5/da/d1a/d2a/d36/dc3/fdb 0 2024-06-03T01:21:07.432 INFO:tasks.workunit.client.0.smithi032.stdout:2/654: getdents d5/d38/d46/dbe 0 2024-06-03T01:21:07.434 INFO:tasks.workunit.client.0.smithi032.stdout:7/610: creat dc/d12/d22/fbf x:0 0 0 2024-06-03T01:21:07.434 INFO:tasks.workunit.client.0.smithi032.stdout:7/611: write dc/d12/d24/d7b/f20 [3537759,22202] 0 2024-06-03T01:21:07.434 INFO:tasks.workunit.client.0.smithi032.stdout:2/655: mknod d5/d38/d46/d53/d57/cdc 0 2024-06-03T01:21:07.437 INFO:tasks.workunit.client.0.smithi032.stdout:7/612: unlink dc/d12/d2a/c3f 0 2024-06-03T01:21:07.437 INFO:tasks.workunit.client.0.smithi032.stdout:4/601: dwrite d4/db/da/d1d/d6b/fba [0,4194304] 0 2024-06-03T01:21:07.440 INFO:tasks.workunit.client.0.smithi032.stdout:4/602: mknod d4/db/da/d37/d90/cd7 0 2024-06-03T01:21:07.444 INFO:tasks.workunit.client.0.smithi032.stdout:4/603: unlink c3 0 2024-06-03T01:21:07.446 INFO:tasks.workunit.client.0.smithi032.stdout:4/604: mknod d4/d78/cd8 0 2024-06-03T01:21:07.456 INFO:tasks.workunit.client.0.smithi032.stdout:2/656: dwrite d5/fe [0,4194304] 0 2024-06-03T01:21:07.457 INFO:tasks.workunit.client.0.smithi032.stdout:7/613: dwrite dc/d12/d6a/d86/d88/d8b/db8/fba [0,4194304] 0 2024-06-03T01:21:07.458 INFO:tasks.workunit.client.0.smithi032.stdout:2/657: truncate d5/da/d2e/f3d 21739 0 2024-06-03T01:21:07.462 INFO:tasks.workunit.client.0.smithi032.stdout:2/658: creat d5/d38/d46/d5c/d6c/d89/fdd x:0 0 0 2024-06-03T01:21:07.463 INFO:tasks.workunit.client.0.smithi032.stdout:2/659: chown d5/da/f29 18 1 2024-06-03T01:21:07.463 INFO:tasks.workunit.client.0.smithi032.stdout:2/660: truncate d5/d38/d46/d53/fa7 392620 0 2024-06-03T01:21:07.463 INFO:tasks.workunit.client.0.smithi032.stdout:2/661: fdatasync d5/da/f2f 0 2024-06-03T01:21:07.463 INFO:tasks.workunit.client.0.smithi032.stdout:4/605: dread d4/db/da/f59 [0,4194304] 0 2024-06-03T01:21:07.464 INFO:tasks.workunit.client.0.smithi032.stdout:4/606: symlink d4/db/da/d2d/d27/ld9 0 2024-06-03T01:21:07.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/614: dwrite dc/d12/d24/f57 [4194304,4194304] 0 2024-06-03T01:21:07.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/615: getdents dc/d12/d22/d9f 0 2024-06-03T01:21:07.475 INFO:tasks.workunit.client.0.smithi032.stdout:7/616: rmdir dc/d12/d6a/d86/d89 39 2024-06-03T01:21:07.477 INFO:tasks.workunit.client.0.smithi032.stdout:7/617: rename dc/d12/d2a/d2f/f56 to dc/d12/d22/fc0 0 2024-06-03T01:21:07.478 INFO:tasks.workunit.client.0.smithi032.stdout:2/662: dwrite d5/da/d1a/f3e [0,4194304] 0 2024-06-03T01:21:07.478 INFO:tasks.workunit.client.0.smithi032.stdout:7/618: unlink dc/d12/d2a/ca2 0 2024-06-03T01:21:07.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/663: mknod d5/d38/d46/cde 0 2024-06-03T01:21:07.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/664: write d5/f23 [1514106,125784] 0 2024-06-03T01:21:07.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/665: creat d5/d38/d46/d5c/d6c/fdf x:0 0 0 2024-06-03T01:21:07.484 INFO:tasks.workunit.client.0.smithi032.stdout:4/607: dwrite d4/db/da/d1d/d6b/fba [4194304,4194304] 0 2024-06-03T01:21:07.484 INFO:tasks.workunit.client.0.smithi032.stdout:4/608: fdatasync d4/db/da/f59 0 2024-06-03T01:21:07.484 INFO:tasks.workunit.client.0.smithi032.stdout:2/666: read d5/da/d1a/d69/d83/fa1 [24361,118587] 0 2024-06-03T01:21:07.490 INFO:tasks.workunit.client.0.smithi032.stdout:2/667: mkdir d5/d38/d46/d53/d57/dac/de0 0 2024-06-03T01:21:07.490 INFO:tasks.workunit.client.0.smithi032.stdout:2/668: truncate d5/da/d1a/d69/d83/f96 1617438 0 2024-06-03T01:21:07.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/669: chown d5/da/d1a/d2a/d36/l8c 543438 1 2024-06-03T01:21:07.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/670: dread d5/da/d1a/d69/d83/fa1 [0,4194304] 0 2024-06-03T01:21:07.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/671: stat d5/da/c87 0 2024-06-03T01:21:07.492 INFO:tasks.workunit.client.0.smithi032.stdout:2/672: write d5/d38/db5/fcf [19830,60475] 0 2024-06-03T01:21:07.502 INFO:tasks.workunit.client.0.smithi032.stdout:7/619: dwrite dc/d12/d1b/f34 [8388608,4194304] 0 2024-06-03T01:21:07.504 INFO:tasks.workunit.client.0.smithi032.stdout:4/609: dwrite d4/db/da/d2d/d27/f89 [0,4194304] 0 2024-06-03T01:21:07.505 INFO:tasks.workunit.client.0.smithi032.stdout:7/620: creat dc/d12/d2a/d2f/d9a/fc1 x:0 0 0 2024-06-03T01:21:07.506 INFO:tasks.workunit.client.0.smithi032.stdout:4/610: mknod d4/db/cda 0 2024-06-03T01:21:07.517 INFO:tasks.workunit.client.0.smithi032.stdout:2/673: dwrite d5/da/d1a/d2a/f50 [0,4194304] 0 2024-06-03T01:21:07.519 INFO:tasks.workunit.client.0.smithi032.stdout:7/621: write dc/d12/d6a/d86/d89/fab [2669579,40779] 0 2024-06-03T01:21:07.520 INFO:tasks.workunit.client.0.smithi032.stdout:2/674: creat d5/d38/d46/d53/d57/dac/de0/fe1 x:0 0 0 2024-06-03T01:21:07.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/622: rename dc/d12/d2a/d2f/fa9 to dc/d12/d24/d7b/d84/fc2 0 2024-06-03T01:21:07.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/623: dread - dc/d12/d2a/d2f/d76/f7d zero size 2024-06-03T01:21:07.529 INFO:tasks.workunit.client.0.smithi032.stdout:4/611: dwrite d4/db/da/d2d/d27/d8c/d9a/fd5 [0,4194304] 0 2024-06-03T01:21:07.566 INFO:tasks.workunit.client.0.smithi032.stdout:2/675: mknod d5/d38/d46/d5c/ce2 0 2024-06-03T01:21:07.566 INFO:tasks.workunit.client.0.smithi032.stdout:2/676: dread - d5/da/d1a/d2a/d36/d3f/f84 zero size 2024-06-03T01:21:07.567 INFO:tasks.workunit.client.0.smithi032.stdout:2/677: dread d5/da/fa6 [0,4194304] 0 2024-06-03T01:21:07.568 INFO:tasks.workunit.client.0.smithi032.stdout:2/678: creat d5/da/d1a/d69/d83/fe3 x:0 0 0 2024-06-03T01:21:07.570 INFO:tasks.workunit.client.0.smithi032.stdout:4/612: mknod d4/db/da/d2d/d27/d8c/db5/cdb 0 2024-06-03T01:21:07.576 INFO:tasks.workunit.client.0.smithi032.stdout:7/624: truncate dc/d12/d1b/f36 3811131 0 2024-06-03T01:21:07.590 INFO:tasks.workunit.client.0.smithi032.stdout:4/613: dwrite d4/f29 [0,4194304] 0 2024-06-03T01:21:07.593 INFO:tasks.workunit.client.0.smithi032.stdout:4/614: dread d4/d7f/fad [0,4194304] 0 2024-06-03T01:21:07.601 INFO:tasks.workunit.client.0.smithi032.stdout:7/625: dwrite dc/f30 [0,4194304] 0 2024-06-03T01:21:07.604 INFO:tasks.workunit.client.0.smithi032.stdout:7/626: mknod dc/dbe/cc3 0 2024-06-03T01:21:07.605 INFO:tasks.workunit.client.0.smithi032.stdout:7/627: mknod dc/d12/d6a/cc4 0 2024-06-03T01:21:07.610 INFO:tasks.workunit.client.0.smithi032.stdout:4/615: dwrite d4/d57/d6c/f6e [0,4194304] 0 2024-06-03T01:21:07.610 INFO:tasks.workunit.client.0.smithi032.stdout:4/616: creat d4/db/da/d5d/d8e/fdc x:0 0 0 2024-06-03T01:21:07.619 INFO:tasks.workunit.client.0.smithi032.stdout:4/617: rmdir d4/db/da 39 2024-06-03T01:21:07.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/618: getdents d4/db/da/d2d/d27/d8c/d9a 0 2024-06-03T01:21:07.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/619: chown d4/db/da/d2d/d27/d8c/db5/ld1 193370 1 2024-06-03T01:21:07.623 INFO:tasks.workunit.client.0.smithi032.stdout:4/620: dread d4/db/f1c [0,4194304] 0 2024-06-03T01:21:07.640 INFO:tasks.workunit.client.0.smithi032.stdout:4/621: dwrite d4/db/da/d2d/d27/f60 [0,4194304] 0 2024-06-03T01:21:07.642 INFO:tasks.workunit.client.0.smithi032.stdout:4/622: rename d4/db/da/d2d/d27/d8c/d9a to d4/db/da/d2d/d27/d8c/db5/db3/ddd 0 2024-06-03T01:21:07.644 INFO:tasks.workunit.client.0.smithi032.stdout:4/623: rename d4/c6a to d4/db/da/d2d/d27/d8c/db5/db3/ddd/cde 0 2024-06-03T01:21:07.657 INFO:tasks.workunit.client.0.smithi032.stdout:4/624: rmdir d4/d57 39 2024-06-03T01:21:07.663 INFO:tasks.workunit.client.0.smithi032.stdout:4/625: write d4/db/da/f59 [3220968,83040] 0 2024-06-03T01:21:07.663 INFO:tasks.workunit.client.0.smithi032.stdout:4/626: dread - d4/db/fcd zero size 2024-06-03T01:21:07.663 INFO:tasks.workunit.client.0.smithi032.stdout:4/627: fdatasync d4/db/da/d1d/faf 0 2024-06-03T01:21:07.665 INFO:tasks.workunit.client.0.smithi032.stdout:4/628: symlink d4/db/da/d5d/d8e/ldf 0 2024-06-03T01:21:07.666 INFO:tasks.workunit.client.0.smithi032.stdout:4/629: mknod d4/d78/ce0 0 2024-06-03T01:21:07.666 INFO:tasks.workunit.client.0.smithi032.stdout:4/630: chown d4/c1e 14 1 2024-06-03T01:21:07.681 INFO:tasks.workunit.client.0.smithi032.stdout:4/631: dwrite d4/db/fa0 [0,4194304] 0 2024-06-03T01:21:07.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/632: mkdir d4/db/da/d2d/de1 0 2024-06-03T01:21:07.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/633: dread - d4/db/da/d5d/d8e/fa3 zero size 2024-06-03T01:21:07.685 INFO:tasks.workunit.client.0.smithi032.stdout:4/634: truncate d4/d5c/f5e 1152821 0 2024-06-03T01:21:07.688 INFO:tasks.workunit.client.0.smithi032.stdout:4/635: creat d4/d5c/fe2 x:0 0 0 2024-06-03T01:21:07.688 INFO:tasks.workunit.client.0.smithi032.stdout:4/636: readlink d4/db/l24 0 2024-06-03T01:21:07.697 INFO:tasks.workunit.client.0.smithi032.stdout:4/637: write d4/db/da/f7d [1913101,51480] 0 2024-06-03T01:21:07.718 INFO:tasks.workunit.client.0.smithi032.stdout:4/638: dwrite d4/db/da/d2d/d27/d53/f65 [0,4194304] 0 2024-06-03T01:21:07.720 INFO:tasks.workunit.client.0.smithi032.stdout:4/639: rename d4/db/da/d2d/d27/d53/f65 to d4/db/da/d1d/d6b/fe3 0 2024-06-03T01:21:07.720 INFO:tasks.workunit.client.0.smithi032.stdout:4/640: creat d4/db/fe4 x:0 0 0 2024-06-03T01:21:07.727 INFO:tasks.workunit.client.0.smithi032.stdout:3/504: sync 2024-06-03T01:21:07.731 INFO:tasks.workunit.client.0.smithi032.stdout:3/505: rename de/d29/d2e/d31/f18 to de/d29/d2e/d31/d3c/d50/d62/fa2 0 2024-06-03T01:21:07.733 INFO:tasks.workunit.client.0.smithi032.stdout:4/641: dread d4/db/da/f7d [0,4194304] 0 2024-06-03T01:21:07.736 INFO:tasks.workunit.client.0.smithi032.stdout:4/642: symlink d4/d7f/le5 0 2024-06-03T01:21:07.737 INFO:tasks.workunit.client.0.smithi032.stdout:4/643: read d4/db/da/f7d [4406281,76906] 0 2024-06-03T01:21:07.739 INFO:tasks.workunit.client.0.smithi032.stdout:4/644: unlink d4/db/da/d2d/d27/d8c/db5/db3/ddd/la4 0 2024-06-03T01:21:07.922 INFO:tasks.workunit.client.0.smithi032.stdout:6/482: sync 2024-06-03T01:21:07.922 INFO:tasks.workunit.client.0.smithi032.stdout:6/483: write d8/d11/d20/d31/f45 [4357464,27108] 0 2024-06-03T01:21:07.927 INFO:tasks.workunit.client.0.smithi032.stdout:6/484: dread d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:21:07.929 INFO:tasks.workunit.client.0.smithi032.stdout:6/485: symlink d8/d11/d20/d2b/d39/d19/l98 0 2024-06-03T01:21:07.929 INFO:tasks.workunit.client.0.smithi032.stdout:6/486: getdents d8/d11/d20/d89 0 2024-06-03T01:21:07.932 INFO:tasks.workunit.client.0.smithi032.stdout:6/487: link d8/d11/d20/d2b/d39/d7e/l8a d8/d5f/l99 0 2024-06-03T01:21:07.932 INFO:tasks.workunit.client.0.smithi032.stdout:6/488: chown d8/d11/d20/c5b 446863 1 2024-06-03T01:21:07.933 INFO:tasks.workunit.client.0.smithi032.stdout:6/489: unlink d8/d11/d20/d21/l95 0 2024-06-03T01:21:07.934 INFO:tasks.workunit.client.0.smithi032.stdout:6/490: readlink d8/d55/l5c 0 2024-06-03T01:21:07.934 INFO:tasks.workunit.client.0.smithi032.stdout:6/491: write d8/d11/d20/f8c [5411811,3210] 0 2024-06-03T01:21:07.936 INFO:tasks.workunit.client.0.smithi032.stdout:6/492: dread d8/d11/d20/d31/d3f/f69 [0,4194304] 0 2024-06-03T01:21:07.937 INFO:tasks.workunit.client.0.smithi032.stdout:6/493: mkdir d8/d11/d20/d21/d9a 0 2024-06-03T01:21:07.938 INFO:tasks.workunit.client.0.smithi032.stdout:6/494: mknod d8/d11/d20/d7f/c9b 0 2024-06-03T01:21:07.939 INFO:tasks.workunit.client.0.smithi032.stdout:6/495: rmdir d8/d11/d20/d2b/d39/d19 39 2024-06-03T01:21:07.940 INFO:tasks.workunit.client.0.smithi032.stdout:6/496: mknod d8/d11/d20/d2b/d39/d7e/c9c 0 2024-06-03T01:21:07.941 INFO:tasks.workunit.client.0.smithi032.stdout:6/497: symlink d8/d11/d20/d7f/l9d 0 2024-06-03T01:21:07.942 INFO:tasks.workunit.client.0.smithi032.stdout:6/498: mknod d8/d11/d20/d2b/c9e 0 2024-06-03T01:21:07.943 INFO:tasks.workunit.client.0.smithi032.stdout:6/499: creat d8/d11/d20/f9f x:0 0 0 2024-06-03T01:21:07.961 INFO:tasks.workunit.client.0.smithi032.stdout:6/500: dwrite d8/d11/d20/d36/f7d [0,4194304] 0 2024-06-03T01:21:07.963 INFO:tasks.workunit.client.0.smithi032.stdout:6/501: link d8/d55/f6a d8/d4c/fa0 0 2024-06-03T01:21:07.967 INFO:tasks.workunit.client.0.smithi032.stdout:6/502: link d8/d11/d20/d2b/d39/d7e/c9c d8/d11/d20/ca1 0 2024-06-03T01:21:07.967 INFO:tasks.workunit.client.0.smithi032.stdout:6/503: stat d8 0 2024-06-03T01:21:07.975 INFO:tasks.workunit.client.0.smithi032.stdout:6/504: write d8/d11/f3e [830195,107551] 0 2024-06-03T01:21:07.976 INFO:tasks.workunit.client.0.smithi032.stdout:6/505: creat d8/d11/d20/d21/fa2 x:0 0 0 2024-06-03T01:21:07.977 INFO:tasks.workunit.client.0.smithi032.stdout:6/506: link d8/d11/d20/d31/f45 d8/d11/d20/d2b/d39/d7e/fa3 0 2024-06-03T01:21:07.977 INFO:tasks.workunit.client.0.smithi032.stdout:6/507: creat d8/d11/d20/d2b/d39/fa4 x:0 0 0 2024-06-03T01:21:07.979 INFO:tasks.workunit.client.0.smithi032.stdout:6/508: truncate f3 7028511 0 2024-06-03T01:21:07.979 INFO:tasks.workunit.client.0.smithi032.stdout:6/509: chown d8/l10 380406102 1 2024-06-03T01:21:07.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/510: truncate f4 2128432 0 2024-06-03T01:21:07.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/511: chown d8/d11/d20/d31/d3f/c87 125 1 2024-06-03T01:21:07.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/512: mkdir d8/d11/d20/d36/da5 0 2024-06-03T01:21:08.000 INFO:tasks.workunit.client.0.smithi032.stdout:6/513: dwrite f4 [0,4194304] 0 2024-06-03T01:21:08.001 INFO:tasks.workunit.client.0.smithi032.stdout:6/514: symlink d8/d11/d20/d21/d9a/la6 0 2024-06-03T01:21:08.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/515: dread d8/d11/f35 [0,4194304] 0 2024-06-03T01:21:08.009 INFO:tasks.workunit.client.0.smithi032.stdout:6/516: link d8/d11/d20/d21/l26 d8/d11/d20/d2b/d39/d19/la7 0 2024-06-03T01:21:08.147 INFO:tasks.workunit.client.0.smithi032.stdout:8/445: sync 2024-06-03T01:21:08.147 INFO:tasks.workunit.client.0.smithi032.stdout:9/651: sync 2024-06-03T01:21:08.151 INFO:tasks.workunit.client.0.smithi032.stdout:9/652: unlink d0/d23/d24/c69 0 2024-06-03T01:21:08.156 INFO:tasks.workunit.client.0.smithi032.stdout:8/446: dread dc/d14/f22 [0,4194304] 0 2024-06-03T01:21:08.164 INFO:tasks.workunit.client.0.smithi032.stdout:8/447: write dc/de/f91 [2523145,77794] 0 2024-06-03T01:21:08.166 INFO:tasks.workunit.client.0.smithi032.stdout:8/448: creat dc/de/f95 x:0 0 0 2024-06-03T01:21:08.166 INFO:tasks.workunit.client.0.smithi032.stdout:8/449: write dc/d25/d28/d33/d78/f7e [687924,60050] 0 2024-06-03T01:21:08.168 INFO:tasks.workunit.client.0.smithi032.stdout:8/450: symlink dc/d25/d64/l96 0 2024-06-03T01:21:08.192 INFO:tasks.workunit.client.0.smithi032.stdout:8/451: dwrite dc/d25/d28/d58/f79 [0,4194304] 0 2024-06-03T01:21:08.192 INFO:tasks.workunit.client.0.smithi032.stdout:8/452: stat dc/d25/d28/d51/d5d/d5e 0 2024-06-03T01:21:08.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/453: dread dc/d25/d28/d58/d8d/d94/f6d [0,4194304] 0 2024-06-03T01:21:08.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/454: fsync dc/d25/d28/d33/f48 0 2024-06-03T01:21:08.210 INFO:tasks.workunit.client.0.smithi032.stdout:8/455: truncate dc/f15 4260226 0 2024-06-03T01:21:08.210 INFO:tasks.workunit.client.0.smithi032.stdout:8/456: fsync dc/d25/d28/d3c/f5a 0 2024-06-03T01:21:08.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/457: dread dc/fd [0,4194304] 0 2024-06-03T01:21:08.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/458: write dc/d25/d28/d3c/f42 [479774,92925] 0 2024-06-03T01:21:08.221 INFO:tasks.workunit.client.0.smithi032.stdout:8/459: read f1 [223342,46989] 0 2024-06-03T01:21:08.244 INFO:tasks.workunit.client.0.smithi032.stdout:8/460: dwrite dc/d25/d64/f89 [0,4194304] 0 2024-06-03T01:21:08.246 INFO:tasks.workunit.client.0.smithi032.stdout:8/461: stat dc/d25/d28/d58/d8d/d94/l81 0 2024-06-03T01:21:08.260 INFO:tasks.workunit.client.0.smithi032.stdout:8/462: write fa [2019452,72716] 0 2024-06-03T01:21:08.261 INFO:tasks.workunit.client.0.smithi032.stdout:8/463: link dc/f13 dc/f97 0 2024-06-03T01:21:08.266 INFO:tasks.workunit.client.0.smithi032.stdout:8/464: read dc/d25/d28/d58/d8d/d94/f6d [1347654,116461] 0 2024-06-03T01:21:08.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/465: dread dc/de/d1e/f2e [0,4194304] 0 2024-06-03T01:21:08.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/466: write dc/de/f91 [3822635,87624] 0 2024-06-03T01:21:08.281 INFO:tasks.workunit.client.0.smithi032.stdout:8/467: mkdir dc/de/d1e/d98 0 2024-06-03T01:21:08.281 INFO:tasks.workunit.client.0.smithi032.stdout:8/468: write dc/d25/d28/d33/d78/f82 [104627,125329] 0 2024-06-03T01:21:08.305 INFO:tasks.workunit.client.0.smithi032.stdout:8/469: dwrite dc/d14/f2d [0,4194304] 0 2024-06-03T01:21:08.318 INFO:tasks.workunit.client.0.smithi032.stdout:8/470: dread dc/fd [4194304,4194304] 0 2024-06-03T01:21:08.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/471: mknod dc/d25/d28/d58/d8d/d94/c99 0 2024-06-03T01:21:08.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/472: chown dc/de/d1e/l4a 819 1 2024-06-03T01:21:08.321 INFO:tasks.workunit.client.0.smithi032.stdout:8/473: mkdir dc/d25/d28/d51/d9a 0 2024-06-03T01:21:08.325 INFO:tasks.workunit.client.0.smithi032.stdout:8/474: rename dc/d14/d31/d3b/d74 to dc/de/d1e/d43/d9b 0 2024-06-03T01:21:08.325 INFO:tasks.workunit.client.0.smithi032.stdout:8/475: read dc/de/d1e/f2e [247543,85443] 0 2024-06-03T01:21:08.325 INFO:tasks.workunit.client.0.smithi032.stdout:8/476: write dc/d25/d28/d32/f50 [962200,24031] 0 2024-06-03T01:21:08.330 INFO:tasks.workunit.client.0.smithi032.stdout:8/477: rmdir dc/de 39 2024-06-03T01:21:08.334 INFO:tasks.workunit.client.0.smithi032.stdout:8/478: creat dc/de/d1e/d43/d9b/f9c x:0 0 0 2024-06-03T01:21:08.335 INFO:tasks.workunit.client.0.smithi032.stdout:8/479: truncate dc/f15 683692 0 2024-06-03T01:21:08.411 INFO:tasks.workunit.client.0.smithi032.stdout:1/532: sync 2024-06-03T01:21:08.414 INFO:tasks.workunit.client.0.smithi032.stdout:8/480: dwrite dc/d25/d28/d32/f7d [0,4194304] 0 2024-06-03T01:21:08.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/533: rename d18/d32/d3c/f61 to d18/d3f/d5f/d94/f96 0 2024-06-03T01:21:08.417 INFO:tasks.workunit.client.0.smithi032.stdout:8/481: stat dc/d25/d28/d58/l6f 0 2024-06-03T01:21:08.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/482: chown dc/l8c 123 1 2024-06-03T01:21:08.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/483: fdatasync dc/d25/d28/d3c/f5a 0 2024-06-03T01:21:08.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/484: write f9 [1440493,20091] 0 2024-06-03T01:21:08.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/485: getdents dc/d14/d31 0 2024-06-03T01:21:08.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/486: write dc/de/d1e/f2e [2174459,82829] 0 2024-06-03T01:21:08.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/534: write d18/d3f/d5f/d6b/d78/f2e [2458412,67273] 0 2024-06-03T01:21:08.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/535: fsync d18/d3f/d5f/d6b/d78/f45 0 2024-06-03T01:21:08.430 INFO:tasks.workunit.client.0.smithi032.stdout:8/487: dread dc/d25/d64/f89 [0,4194304] 0 2024-06-03T01:21:08.430 INFO:tasks.workunit.client.0.smithi032.stdout:1/536: dread d18/f34 [0,4194304] 0 2024-06-03T01:21:08.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/537: creat d18/d3f/d5f/d94/f97 x:0 0 0 2024-06-03T01:21:08.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/538: read d18/d3f/d5f/d6b/d78/d3d/d47/f51 [221552,87655] 0 2024-06-03T01:21:08.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/539: rmdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a 39 2024-06-03T01:21:08.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/540: readlink d18/d32/d3c/l49 0 2024-06-03T01:21:08.454 INFO:tasks.workunit.client.0.smithi032.stdout:8/488: dwrite dc/f97 [0,4194304] 0 2024-06-03T01:21:08.455 INFO:tasks.workunit.client.0.smithi032.stdout:8/489: chown dc/d14/d31/d3b 78 1 2024-06-03T01:21:08.457 INFO:tasks.workunit.client.0.smithi032.stdout:1/541: dwrite d18/d3f/d5f/d6b/d78/d3d/f8c [0,4194304] 0 2024-06-03T01:21:08.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/542: write f17 [1861311,68945] 0 2024-06-03T01:21:08.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/543: dread d18/d2d/d5e/f64 [0,4194304] 0 2024-06-03T01:21:08.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/490: dwrite dc/d14/f83 [0,4194304] 0 2024-06-03T01:21:08.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/491: fsync dc/d25/d28/d51/d5d/d5e/f84 0 2024-06-03T01:21:08.485 INFO:tasks.workunit.client.0.smithi032.stdout:8/492: rmdir dc/de/d1e/d43 39 2024-06-03T01:21:08.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/493: dwrite f9 [0,4194304] 0 2024-06-03T01:21:08.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/494: stat dc/d25/d28/d3c/f77 0 2024-06-03T01:21:08.504 INFO:tasks.workunit.client.0.smithi032.stdout:8/495: link dc/de/d1e/l4a dc/d14/l9d 0 2024-06-03T01:21:08.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/496: creat dc/d25/d28/d58/d8d/d94/d59/f9e x:0 0 0 2024-06-03T01:21:08.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/497: stat dc/de/l2c 0 2024-06-03T01:21:08.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/498: write dc/de/d1e/f2e [2046950,110469] 0 2024-06-03T01:21:08.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/499: chown dc/d25/d28/d51/c53 931176107 1 2024-06-03T01:21:08.506 INFO:tasks.workunit.client.0.smithi032.stdout:8/500: creat dc/de/f9f x:0 0 0 2024-06-03T01:21:08.507 INFO:tasks.workunit.client.0.smithi032.stdout:8/501: stat dc/l56 0 2024-06-03T01:21:08.507 INFO:tasks.workunit.client.0.smithi032.stdout:8/502: chown dc/d14/c2b 0 1 2024-06-03T01:21:08.507 INFO:tasks.workunit.client.0.smithi032.stdout:8/503: creat dc/de/d1e/fa0 x:0 0 0 2024-06-03T01:21:08.508 INFO:tasks.workunit.client.0.smithi032.stdout:8/504: mknod dc/d25/ca1 0 2024-06-03T01:21:08.508 INFO:tasks.workunit.client.0.smithi032.stdout:8/505: creat dc/d25/d28/d32/fa2 x:0 0 0 2024-06-03T01:21:08.508 INFO:tasks.workunit.client.0.smithi032.stdout:8/506: creat dc/de/d1e/fa3 x:0 0 0 2024-06-03T01:21:08.510 INFO:tasks.workunit.client.0.smithi032.stdout:8/507: rmdir dc/de/d1e/d43/d7c 0 2024-06-03T01:21:08.511 INFO:tasks.workunit.client.0.smithi032.stdout:8/508: link dc/de/d1e/f2e dc/d25/d64/fa4 0 2024-06-03T01:21:08.529 INFO:tasks.workunit.client.0.smithi032.stdout:8/509: dwrite dc/d25/d28/d51/d5d/d5e/f84 [0,4194304] 0 2024-06-03T01:21:08.536 INFO:tasks.workunit.client.0.smithi032.stdout:8/510: dread f2 [0,4194304] 0 2024-06-03T01:21:08.536 INFO:tasks.workunit.client.0.smithi032.stdout:8/511: write dc/de/f11 [659401,91362] 0 2024-06-03T01:21:08.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/700: sync 2024-06-03T01:21:08.539 INFO:tasks.workunit.client.0.smithi032.stdout:0/625: sync 2024-06-03T01:21:08.543 INFO:tasks.workunit.client.0.smithi032.stdout:5/701: rename da/dd/df/d10/dbc/d18/d35/d70/lc0 to da/dd/le0 0 2024-06-03T01:21:08.543 INFO:tasks.workunit.client.0.smithi032.stdout:5/702: chown da/dd/df/d10/dbc/d18/d35/f81 42 1 2024-06-03T01:21:08.543 INFO:tasks.workunit.client.0.smithi032.stdout:5/703: dread - da/dd/df/d10/dbc/d18/d35/d70/f84 zero size 2024-06-03T01:21:08.546 INFO:tasks.workunit.client.0.smithi032.stdout:0/626: mkdir d1/da/d28/d3b/d5d/d8d/dbe/dc4 0 2024-06-03T01:21:08.546 INFO:tasks.workunit.client.0.smithi032.stdout:0/627: write d1/da/d28/f56 [1588780,91386] 0 2024-06-03T01:21:08.548 INFO:tasks.workunit.client.0.smithi032.stdout:5/704: mknod da/dd/ce1 0 2024-06-03T01:21:08.549 INFO:tasks.workunit.client.0.smithi032.stdout:0/628: creat d1/da/db2/fc5 x:0 0 0 2024-06-03T01:21:08.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/705: unlink da/dd/df/f9d 0 2024-06-03T01:21:08.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/706: write da/dd/df/d10/dbc/d18/d35/d44/dc3/fcf [893759,114699] 0 2024-06-03T01:21:08.553 INFO:tasks.workunit.client.0.smithi032.stdout:0/629: getdents d1/d46 0 2024-06-03T01:21:08.553 INFO:tasks.workunit.client.0.smithi032.stdout:0/630: chown d1/da/d10/f5b 787 1 2024-06-03T01:21:08.555 INFO:tasks.workunit.client.0.smithi032.stdout:0/631: symlink d1/da/d28/d3b/d5d/lc6 0 2024-06-03T01:21:08.556 INFO:tasks.workunit.client.0.smithi032.stdout:0/632: mknod d1/da/d28/d3b/d5d/d8d/cc7 0 2024-06-03T01:21:08.556 INFO:tasks.workunit.client.0.smithi032.stdout:0/633: stat d1/da/d28/d3b/d5d/da9/lb3 0 2024-06-03T01:21:08.557 INFO:tasks.workunit.client.0.smithi032.stdout:0/634: symlink d1/da/d28/d3b/d5d/da9/daa/lc8 0 2024-06-03T01:21:08.557 INFO:tasks.workunit.client.0.smithi032.stdout:8/512: dwrite dc/d25/d64/fa4 [4194304,4194304] 0 2024-06-03T01:21:08.558 INFO:tasks.workunit.client.0.smithi032.stdout:0/635: symlink d1/da/d28/d3b/d5d/d8d/lc9 0 2024-06-03T01:21:08.565 INFO:tasks.workunit.client.0.smithi032.stdout:8/513: write dc/d25/d28/d3c/f77 [916406,14577] 0 2024-06-03T01:21:08.565 INFO:tasks.workunit.client.0.smithi032.stdout:8/514: read f1 [556926,14479] 0 2024-06-03T01:21:08.565 INFO:tasks.workunit.client.0.smithi032.stdout:0/636: mknod d1/da/d28/d3b/d68/d83/cca 0 2024-06-03T01:21:08.567 INFO:tasks.workunit.client.0.smithi032.stdout:8/515: rmdir dc/de/d1e/d43 39 2024-06-03T01:21:08.569 INFO:tasks.workunit.client.0.smithi032.stdout:0/637: read d1/da/d28/d3b/d5d/f65 [601166,63715] 0 2024-06-03T01:21:08.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/638: write d1/da/d10/d29/f96 [447756,29375] 0 2024-06-03T01:21:08.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/639: chown d1/da/d28/d3b/d5d/d8d/d9c/dc2 3665427 1 2024-06-03T01:21:08.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/640: rmdir d1/da/d10/d80/d9f 39 2024-06-03T01:21:08.581 INFO:tasks.workunit.client.0.smithi032.stdout:8/516: dwrite dc/de/f91 [0,4194304] 0 2024-06-03T01:21:08.582 INFO:tasks.workunit.client.0.smithi032.stdout:8/517: read f8 [31692,50837] 0 2024-06-03T01:21:08.582 INFO:tasks.workunit.client.0.smithi032.stdout:0/641: rename d1/da/d28/c88 to d1/da/d10/d80/d9f/dac/ccb 0 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/642: write d1/d46/f98 [1045882,54724] 0 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/643: rename d1/da/d28/d3b/d5d to d1/da/d28/d3b/d5d/d8d/d9c/dc2/dcc 22 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/644: truncate d1/da/d10/f4f 4087781 0 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:8/518: symlink dc/d93/la5 0 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/645: mknod d1/da/d28/d3b/d5d/d8d/ccd 0 2024-06-03T01:21:08.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/646: read - d1/da/d10/d29/f45 zero size 2024-06-03T01:21:08.584 INFO:tasks.workunit.client.0.smithi032.stdout:0/647: truncate d1/da/f1a 4706405 0 2024-06-03T01:21:08.584 INFO:tasks.workunit.client.0.smithi032.stdout:8/519: rmdir dc/de/d1e/d98 0 2024-06-03T01:21:08.593 INFO:tasks.workunit.client.0.smithi032.stdout:0/648: dread d1/da/f4d [0,4194304] 0 2024-06-03T01:21:08.595 INFO:tasks.workunit.client.0.smithi032.stdout:0/649: truncate d1/da/d10/d53/f7e 184170 0 2024-06-03T01:21:08.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/520: dwrite dc/d25/d28/d3c/f77 [0,4194304] 0 2024-06-03T01:21:08.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/650: dwrite d1/da/d10/f5b [0,4194304] 0 2024-06-03T01:21:08.612 INFO:tasks.workunit.client.0.smithi032.stdout:0/651: mkdir d1/da/d28/d3b/dce 0 2024-06-03T01:21:08.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/652: dread d1/da/d10/f6f [0,4194304] 0 2024-06-03T01:21:08.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/653: truncate d1/da/d10/d53/f60 3532310 0 2024-06-03T01:21:08.621 INFO:tasks.workunit.client.0.smithi032.stdout:0/654: readlink d1/d46/d3a/l8c 0 2024-06-03T01:21:08.621 INFO:tasks.workunit.client.0.smithi032.stdout:8/521: dwrite dc/d25/d28/d33/d78/f82 [0,4194304] 0 2024-06-03T01:21:08.621 INFO:tasks.workunit.client.0.smithi032.stdout:8/522: read - dc/de/f95 zero size 2024-06-03T01:21:08.622 INFO:tasks.workunit.client.0.smithi032.stdout:0/655: creat d1/da/d28/fcf x:0 0 0 2024-06-03T01:21:08.622 INFO:tasks.workunit.client.0.smithi032.stdout:0/656: creat d1/da/d10/d29/fd0 x:0 0 0 2024-06-03T01:21:08.626 INFO:tasks.workunit.client.0.smithi032.stdout:8/523: mkdir dc/d25/d28/d58/d8d/d94/da6 0 2024-06-03T01:21:08.630 INFO:tasks.workunit.client.0.smithi032.stdout:8/524: mknod dc/de/d1e/d43/ca7 0 2024-06-03T01:21:08.630 INFO:tasks.workunit.client.0.smithi032.stdout:8/525: fdatasync dc/de/f91 0 2024-06-03T01:21:08.631 INFO:tasks.workunit.client.0.smithi032.stdout:8/526: creat dc/de/fa8 x:0 0 0 2024-06-03T01:21:08.631 INFO:tasks.workunit.client.0.smithi032.stdout:8/527: write dc/de/f9f [790485,117036] 0 2024-06-03T01:21:08.632 INFO:tasks.workunit.client.0.smithi032.stdout:8/528: symlink dc/de/d1e/d43/la9 0 2024-06-03T01:21:08.632 INFO:tasks.workunit.client.0.smithi032.stdout:8/529: truncate dc/de/f95 910381 0 2024-06-03T01:21:08.639 INFO:tasks.workunit.client.0.smithi032.stdout:8/530: write dc/de/f91 [3415309,66500] 0 2024-06-03T01:21:08.728 INFO:tasks.workunit.client.0.smithi032.stdout:2/679: sync 2024-06-03T01:21:08.731 INFO:tasks.workunit.client.0.smithi032.stdout:2/680: link d5/c30 d5/d38/d46/ce4 0 2024-06-03T01:21:08.731 INFO:tasks.workunit.client.0.smithi032.stdout:2/681: truncate d5/f25 2648530 0 2024-06-03T01:21:08.731 INFO:tasks.workunit.client.0.smithi032.stdout:2/682: fdatasync d5/d38/d46/d53/d57/dac/f75 0 2024-06-03T01:21:08.732 INFO:tasks.workunit.client.0.smithi032.stdout:2/683: dread d5/da/d1a/d2a/d36/f66 [0,4194304] 0 2024-06-03T01:21:08.734 INFO:tasks.workunit.client.0.smithi032.stdout:2/684: rmdir d5/d38/d46/d53/da9 39 2024-06-03T01:21:08.750 INFO:tasks.workunit.client.0.smithi032.stdout:2/685: dwrite d5/da/f59 [0,4194304] 0 2024-06-03T01:21:08.752 INFO:tasks.workunit.client.0.smithi032.stdout:2/686: write d5/da/d1a/f4b [2013833,19060] 0 2024-06-03T01:21:08.753 INFO:tasks.workunit.client.0.smithi032.stdout:2/687: fdatasync d5/d38/d46/d5c/d6c/d89/fdd 0 2024-06-03T01:21:08.753 INFO:tasks.workunit.client.0.smithi032.stdout:2/688: write d5/da/d1a/d69/d83/fa1 [789707,74478] 0 2024-06-03T01:21:08.755 INFO:tasks.workunit.client.0.smithi032.stdout:2/689: truncate d5/d38/d46/d5c/fc5 898861 0 2024-06-03T01:21:08.755 INFO:tasks.workunit.client.0.smithi032.stdout:2/690: creat d5/d38/d46/fe5 x:0 0 0 2024-06-03T01:21:08.758 INFO:tasks.workunit.client.0.smithi032.stdout:2/691: mkdir d5/da/d1a/d2a/da8/de6 0 2024-06-03T01:21:08.758 INFO:tasks.workunit.client.0.smithi032.stdout:2/692: readlink d5/da/d1a/d2a/d36/l4a 0 2024-06-03T01:21:08.758 INFO:tasks.workunit.client.0.smithi032.stdout:2/693: readlink d5/l2d 0 2024-06-03T01:21:08.759 INFO:tasks.workunit.client.0.smithi032.stdout:2/694: read d5/da/d2e/f61 [4944,106807] 0 2024-06-03T01:21:08.766 INFO:tasks.workunit.client.0.smithi032.stdout:2/695: write d5/d38/d46/f81 [111143,16686] 0 2024-06-03T01:21:08.768 INFO:tasks.workunit.client.0.smithi032.stdout:2/696: creat d5/d38/fe7 x:0 0 0 2024-06-03T01:21:08.776 INFO:tasks.workunit.client.0.smithi032.stdout:2/697: dread d5/da/d1a/f3e [0,4194304] 0 2024-06-03T01:21:08.777 INFO:tasks.workunit.client.0.smithi032.stdout:2/698: creat d5/d38/d46/d53/d57/fe8 x:0 0 0 2024-06-03T01:21:08.778 INFO:tasks.workunit.client.0.smithi032.stdout:2/699: creat d5/d38/d46/d53/d57/dac/fe9 x:0 0 0 2024-06-03T01:21:08.780 INFO:tasks.workunit.client.0.smithi032.stdout:2/700: rename d5/d38/d46/d53/d57/dac/d86 to d5/da/d1a/d69/dea 0 2024-06-03T01:21:08.780 INFO:tasks.workunit.client.0.smithi032.stdout:2/701: write d5/da/d1a/d2a/f50 [4938361,55515] 0 2024-06-03T01:21:08.790 INFO:tasks.workunit.client.0.smithi032.stdout:2/702: dread d5/d38/d46/d53/d57/dac/f73 [0,4194304] 0 2024-06-03T01:21:08.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/703: unlink d5/da/d1a/d2a/f50 0 2024-06-03T01:21:08.794 INFO:tasks.workunit.client.0.smithi032.stdout:2/704: unlink d5/l24 0 2024-06-03T01:21:08.802 INFO:tasks.workunit.client.0.smithi032.stdout:2/705: dread d5/f23 [0,4194304] 0 2024-06-03T01:21:08.802 INFO:tasks.workunit.client.0.smithi032.stdout:2/706: creat d5/da/d1a/d2a/d36/d3f/feb x:0 0 0 2024-06-03T01:21:08.804 INFO:tasks.workunit.client.0.smithi032.stdout:2/707: mkdir d5/da/d2e/dec 0 2024-06-03T01:21:08.804 INFO:tasks.workunit.client.0.smithi032.stdout:2/708: chown d5/d38/d46/d53/d57/d6d/fbd 3267 1 2024-06-03T01:21:08.806 INFO:tasks.workunit.client.0.smithi032.stdout:2/709: getdents d5/d38/d46/d5c/d6c/d89 0 2024-06-03T01:21:08.806 INFO:tasks.workunit.client.0.smithi032.stdout:2/710: write d5/da/fa6 [542772,72924] 0 2024-06-03T01:21:08.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/711: dread d5/da/f74 [0,4194304] 0 2024-06-03T01:21:08.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/712: write d5/d38/d46/d5c/d6c/d89/f93 [987472,110291] 0 2024-06-03T01:21:08.815 INFO:tasks.workunit.client.0.smithi032.stdout:2/713: symlink d5/da/d1a/d2a/da8/de6/led 0 2024-06-03T01:21:08.816 INFO:tasks.workunit.client.0.smithi032.stdout:2/714: rmdir d5/d38/d46/d53/d57/d6d 39 2024-06-03T01:21:08.816 INFO:tasks.workunit.client.0.smithi032.stdout:2/715: chown d5/d38/d46/d53/d57/dac/fe9 19 1 2024-06-03T01:21:08.819 INFO:tasks.workunit.client.0.smithi032.stdout:2/716: rmdir d5/da/d1a/d2a 39 2024-06-03T01:21:08.822 INFO:tasks.workunit.client.0.smithi032.stdout:2/717: creat d5/d38/d46/d53/d57/d6d/dce/d8f/fee x:0 0 0 2024-06-03T01:21:08.822 INFO:tasks.workunit.client.0.smithi032.stdout:2/718: fsync d5/f23 0 2024-06-03T01:21:08.823 INFO:tasks.workunit.client.0.smithi032.stdout:2/719: getdents d5/d38 0 2024-06-03T01:21:08.844 INFO:tasks.workunit.client.0.smithi032.stdout:2/720: dwrite d5/da/d1a/fa5 [0,4194304] 0 2024-06-03T01:21:08.845 INFO:tasks.workunit.client.0.smithi032.stdout:2/721: mkdir d5/d38/d46/d53/d90/def 0 2024-06-03T01:21:08.847 INFO:tasks.workunit.client.0.smithi032.stdout:2/722: symlink d5/d38/d46/lf0 0 2024-06-03T01:21:08.849 INFO:tasks.workunit.client.0.smithi032.stdout:2/723: link d5/d38/d46/d53/da9/cae d5/d38/d46/d5c/d6c/cf1 0 2024-06-03T01:21:08.868 INFO:tasks.workunit.client.0.smithi032.stdout:2/724: dwrite d5/da/d1a/d2a/d36/d3f/f4f [0,4194304] 0 2024-06-03T01:21:08.869 INFO:tasks.workunit.client.0.smithi032.stdout:2/725: mkdir d5/d38/d46/df2 0 2024-06-03T01:21:08.869 INFO:tasks.workunit.client.0.smithi032.stdout:2/726: chown d5/d38/d46/d5c/f8b 0 1 2024-06-03T01:21:08.872 INFO:tasks.workunit.client.0.smithi032.stdout:2/727: link d5/d38/d46/d53/d57/d6d/dce/d8f/fee d5/d38/d46/d53/d57/ff3 0 2024-06-03T01:21:08.872 INFO:tasks.workunit.client.0.smithi032.stdout:2/728: readlink d5/d38/d46/d53/d57/d6d/dce/d67/la2 0 2024-06-03T01:21:08.872 INFO:tasks.workunit.client.0.smithi032.stdout:2/729: write d5/da/f1e [4595444,34369] 0 2024-06-03T01:21:08.889 INFO:tasks.workunit.client.0.smithi032.stdout:2/730: dwrite d5/d38/d46/d53/fa7 [0,4194304] 0 2024-06-03T01:21:08.910 INFO:tasks.workunit.client.0.smithi032.stdout:2/731: dwrite d5/da/fa6 [0,4194304] 0 2024-06-03T01:21:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 256 MiB/s rd, 311 MiB/s wr, 464 op/s 2024-06-03T01:21:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: Standby manager daemon smithi032.kfcfaa started 2024-06-03T01:21:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/crt"}]: dispatch 2024-06-03T01:21:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:21:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/key"}]: dispatch 2024-06-03T01:21:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:08 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 256 MiB/s rd, 311 MiB/s wr, 464 op/s 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: Standby manager daemon smithi032.kfcfaa started 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/crt"}]: dispatch 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/key"}]: dispatch 2024-06-03T01:21:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:08 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/369704542' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:21:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:09 smithi044 ceph-mon[36956]: mgrmap e24: smithi044.wszjlw(active, since 12s), standbys: smithi032.kfcfaa 2024-06-03T01:21:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:09 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:21:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:09 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:09 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:10.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:09 smithi032 ceph-mon[29360]: mgrmap e24: smithi044.wszjlw(active, since 12s), standbys: smithi032.kfcfaa 2024-06-03T01:21:10.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:09 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:21:10.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:09 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:10.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:09 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:10.232 INFO:tasks.workunit.client.0.smithi032.stdout:6/517: sync 2024-06-03T01:21:10.232 INFO:tasks.workunit.client.0.smithi032.stdout:5/707: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/657: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/653: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:1/544: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:3/506: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/658: dread - d1/da/db2/fc5 zero size 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:5/708: truncate da/dd/df/f5c 1144294 0 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/659: chown d1/da/d10/l97 192951743 1 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/654: write d0/d23/d25/f8e [3678982,65154] 0 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:3/507: write de/d29/d2e/d31/d78/f80 [963398,39820] 0 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:8/531: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:4/645: sync 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:4/646: readlink d4/db/da/d37/l45 0 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:4/647: readlink d4/db/da/d37/d90/da1/l83 0 2024-06-03T01:21:10.233 INFO:tasks.workunit.client.0.smithi032.stdout:4/648: fsync d4/db/da/f3f 0 2024-06-03T01:21:10.234 INFO:tasks.workunit.client.0.smithi032.stdout:7/628: sync 2024-06-03T01:21:10.234 INFO:tasks.workunit.client.0.smithi032.stdout:2/732: sync 2024-06-03T01:21:10.235 INFO:tasks.workunit.client.0.smithi032.stdout:3/508: dread de/d29/d2e/d31/d3c/d50/d66/f9a [0,4194304] 0 2024-06-03T01:21:10.259 INFO:tasks.workunit.client.0.smithi032.stdout:5/709: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/f7f [0,4194304] 0 2024-06-03T01:21:10.259 INFO:tasks.workunit.client.0.smithi032.stdout:8/532: dwrite dc/d25/d28/d51/d5d/d5e/f84 [0,4194304] 0 2024-06-03T01:21:10.259 INFO:tasks.workunit.client.0.smithi032.stdout:5/710: fdatasync da/dd/df/d10/dbc/d18/d35/d44/fc5 0 2024-06-03T01:21:10.265 INFO:tasks.workunit.client.0.smithi032.stdout:6/518: creat d8/d11/fa8 x:0 0 0 2024-06-03T01:21:10.266 INFO:tasks.workunit.client.0.smithi032.stdout:0/660: symlink d1/da/ld1 0 2024-06-03T01:21:10.266 INFO:tasks.workunit.client.0.smithi032.stdout:0/661: stat d1/da/d28/d3b/f76 0 2024-06-03T01:21:10.269 INFO:tasks.workunit.client.0.smithi032.stdout:9/655: mknod d0/d11/d1d/d38/d4f/db0/d41/dc6/cd7 0 2024-06-03T01:21:10.269 INFO:tasks.workunit.client.0.smithi032.stdout:9/656: creat d0/d11/d1d/d38/d4f/db0/d41/fd8 x:0 0 0 2024-06-03T01:21:10.271 INFO:tasks.workunit.client.0.smithi032.stdout:7/629: rename dc/d12/fb9 to dc/d12/fc5 0 2024-06-03T01:21:10.271 INFO:tasks.workunit.client.0.smithi032.stdout:7/630: stat dc/dbe 0 2024-06-03T01:21:10.272 INFO:tasks.workunit.client.0.smithi032.stdout:2/733: mknod d5/d38/d46/df2/cf4 0 2024-06-03T01:21:10.275 INFO:tasks.workunit.client.0.smithi032.stdout:3/509: creat de/d29/d2e/d31/fa3 x:0 0 0 2024-06-03T01:21:10.275 INFO:tasks.workunit.client.0.smithi032.stdout:3/510: fdatasync de/d29/d2e/d31/d3c/d50/d56/f5c 0 2024-06-03T01:21:10.275 INFO:tasks.workunit.client.0.smithi032.stdout:3/511: readlink de/d29/d2e/d31/d3c/l77 0 2024-06-03T01:21:10.283 INFO:tasks.workunit.client.0.smithi032.stdout:5/711: truncate da/dd/df/d10/dbc/d25/f8b 1555206 0 2024-06-03T01:21:10.284 INFO:tasks.workunit.client.0.smithi032.stdout:0/662: mkdir d1/da/dd2 0 2024-06-03T01:21:10.285 INFO:tasks.workunit.client.0.smithi032.stdout:0/663: chown d1/da/d10/l97 189 1 2024-06-03T01:21:10.285 INFO:tasks.workunit.client.0.smithi032.stdout:4/649: symlink d4/db/da/d1d/le6 0 2024-06-03T01:21:10.288 INFO:tasks.workunit.client.0.smithi032.stdout:8/533: dwrite dc/d14/f2d [0,4194304] 0 2024-06-03T01:21:10.291 INFO:tasks.workunit.client.0.smithi032.stdout:2/734: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/df5 0 2024-06-03T01:21:10.293 INFO:tasks.workunit.client.0.smithi032.stdout:4/650: dread d4/db/da/f82 [0,4194304] 0 2024-06-03T01:21:10.298 INFO:tasks.workunit.client.0.smithi032.stdout:6/519: dwrite d8/d11/d20/d3b/f6d [0,4194304] 0 2024-06-03T01:21:10.298 INFO:tasks.workunit.client.0.smithi032.stdout:8/534: dread dc/d25/d28/d32/f36 [0,4194304] 0 2024-06-03T01:21:10.299 INFO:tasks.workunit.client.0.smithi032.stdout:2/735: write d5/f58 [5949512,73767] 0 2024-06-03T01:21:10.299 INFO:tasks.workunit.client.0.smithi032.stdout:7/631: dwrite dc/d12/d24/d7b/f20 [0,4194304] 0 2024-06-03T01:21:10.299 INFO:tasks.workunit.client.0.smithi032.stdout:3/512: dwrite de/d29/d2e/d31/d3c/d50/d56/f5c [0,4194304] 0 2024-06-03T01:21:10.300 INFO:tasks.workunit.client.0.smithi032.stdout:3/513: chown de/d29/d2e/d31/d24/f25 4058 1 2024-06-03T01:21:10.303 INFO:tasks.workunit.client.0.smithi032.stdout:0/664: link d1/da/d28/f56 d1/da/d28/d3b/d5d/d8d/dbe/dc4/fd3 0 2024-06-03T01:21:10.306 INFO:tasks.workunit.client.0.smithi032.stdout:8/535: read dc/fd [2068042,86235] 0 2024-06-03T01:21:10.310 INFO:tasks.workunit.client.0.smithi032.stdout:4/651: dwrite d4/db/da/d2d/d27/d53/f66 [0,4194304] 0 2024-06-03T01:21:10.318 INFO:tasks.workunit.client.0.smithi032.stdout:4/652: read d4/db/da/d37/f84 [1347072,88589] 0 2024-06-03T01:21:10.318 INFO:tasks.workunit.client.0.smithi032.stdout:2/736: rename d5/da/d1a/d69/d83/f96 to d5/ff6 0 2024-06-03T01:21:10.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/632: creat dc/d12/d2a/d2f/d76/fc6 x:0 0 0 2024-06-03T01:21:10.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/633: chown dc/d12/d24/d33/f92 7690 1 2024-06-03T01:21:10.322 INFO:tasks.workunit.client.0.smithi032.stdout:0/665: mkdir d1/da/d28/d3b/dce/dd4 0 2024-06-03T01:21:10.322 INFO:tasks.workunit.client.0.smithi032.stdout:6/520: dread d8/fb [4194304,4194304] 0 2024-06-03T01:21:10.323 INFO:tasks.workunit.client.0.smithi032.stdout:4/653: dread d4/db/da/d1d/f4c [0,4194304] 0 2024-06-03T01:21:10.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/514: rename de/d29/d2e/d31/fa3 to de/d29/d2e/d31/d23/d53/d79/fa4 0 2024-06-03T01:21:10.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/515: stat de/d29/d2e/f37 0 2024-06-03T01:21:10.331 INFO:tasks.workunit.client.0.smithi032.stdout:3/516: chown de/d29/f4b 2090276 1 2024-06-03T01:21:10.332 INFO:tasks.workunit.client.0.smithi032.stdout:7/634: creat dc/dbe/fc7 x:0 0 0 2024-06-03T01:21:10.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/666: symlink d1/da/d28/d3b/d5d/da9/daa/ld5 0 2024-06-03T01:21:10.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/667: read - d1/da/d10/d29/fb4 zero size 2024-06-03T01:21:10.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/668: read d1/da/f1a [2428948,16527] 0 2024-06-03T01:21:10.336 INFO:tasks.workunit.client.0.smithi032.stdout:4/654: symlink d4/d7f/le7 0 2024-06-03T01:21:10.337 INFO:tasks.workunit.client.0.smithi032.stdout:0/669: dread d1/da/d28/d3b/d5d/f65 [0,4194304] 0 2024-06-03T01:21:10.337 INFO:tasks.workunit.client.0.smithi032.stdout:0/670: write d1/da/d10/f2d [4770355,1938] 0 2024-06-03T01:21:10.340 INFO:tasks.workunit.client.0.smithi032.stdout:4/655: truncate d4/db/da/f82 807114 0 2024-06-03T01:21:10.340 INFO:tasks.workunit.client.0.smithi032.stdout:4/656: dread - d4/db/da/fc3 zero size 2024-06-03T01:21:10.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/671: mkdir d1/da/dd2/dd6 0 2024-06-03T01:21:10.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/672: chown d1/da/ld1 0 1 2024-06-03T01:21:10.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/673: rename d1/da/d28/fcf to d1/da/d28/d3b/d68/db5/fd7 0 2024-06-03T01:21:10.346 INFO:tasks.workunit.client.0.smithi032.stdout:0/674: creat d1/da/d28/d3b/d5d/da9/fd8 x:0 0 0 2024-06-03T01:21:10.346 INFO:tasks.workunit.client.0.smithi032.stdout:0/675: truncate d1/da/d10/d29/fd0 965425 0 2024-06-03T01:21:10.348 INFO:tasks.workunit.client.0.smithi032.stdout:0/676: rename d1/da/d28/d3b/d68/d83/ca7 to d1/da/d28/cd9 0 2024-06-03T01:21:10.351 INFO:tasks.workunit.client.0.smithi032.stdout:0/677: unlink d1/da/f16 0 2024-06-03T01:21:10.380 INFO:tasks.workunit.client.0.smithi032.stdout:9/657: rmdir d0/d11/d1d/d38/d4f/db0/d41 39 2024-06-03T01:21:10.380 INFO:tasks.workunit.client.0.smithi032.stdout:9/658: write d0/f19 [5342823,53446] 0 2024-06-03T01:21:10.383 INFO:tasks.workunit.client.0.smithi032.stdout:9/659: symlink d0/da4/dc1/ld9 0 2024-06-03T01:21:10.392 INFO:tasks.workunit.client.0.smithi032.stdout:7/635: mkdir dc/d12/d6a/d86/dc8 0 2024-06-03T01:21:10.406 INFO:tasks.workunit.client.0.smithi032.stdout:7/636: mknod dc/d12/d6a/cc9 0 2024-06-03T01:21:10.408 INFO:tasks.workunit.client.0.smithi032.stdout:7/637: unlink dc/d12/d1b/f34 0 2024-06-03T01:21:10.408 INFO:tasks.workunit.client.0.smithi032.stdout:7/638: chown dc/l53 495 1 2024-06-03T01:21:10.408 INFO:tasks.workunit.client.0.smithi032.stdout:7/639: readlink dc/d12/d2a/la6 0 2024-06-03T01:21:10.410 INFO:tasks.workunit.client.0.smithi032.stdout:7/640: unlink dc/c7a 0 2024-06-03T01:21:10.410 INFO:tasks.workunit.client.0.smithi032.stdout:7/641: write dc/d12/d2a/d2f/d76/f80 [698893,7197] 0 2024-06-03T01:21:10.411 INFO:tasks.workunit.client.0.smithi032.stdout:1/545: mkdir d18/d3f/d5f/d98 0 2024-06-03T01:21:10.411 INFO:tasks.workunit.client.0.smithi032.stdout:1/546: fsync d18/d3f/d5f/d6b/d78/f5c 0 2024-06-03T01:21:10.412 INFO:tasks.workunit.client.0.smithi032.stdout:7/642: creat dc/d12/d6a/d86/d89/fca x:0 0 0 2024-06-03T01:21:10.414 INFO:tasks.workunit.client.0.smithi032.stdout:1/547: creat d18/d3f/d5f/d6b/d78/d3d/d6a/f99 x:0 0 0 2024-06-03T01:21:10.414 INFO:tasks.workunit.client.0.smithi032.stdout:1/548: fdatasync d18/d2d/f50 0 2024-06-03T01:21:10.414 INFO:tasks.workunit.client.0.smithi032.stdout:1/549: mknod d18/c9a 0 2024-06-03T01:21:10.417 INFO:tasks.workunit.client.0.smithi032.stdout:1/550: rename d18/d2d/f90 to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b 0 2024-06-03T01:21:10.419 INFO:tasks.workunit.client.0.smithi032.stdout:1/551: rmdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/d95 0 2024-06-03T01:21:10.420 INFO:tasks.workunit.client.0.smithi032.stdout:1/552: mknod d18/d32/d39/d60/c9c 0 2024-06-03T01:21:10.422 INFO:tasks.workunit.client.0.smithi032.stdout:1/553: truncate d18/f1c 3558758 0 2024-06-03T01:21:10.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/554: rename d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d7a to d18/d3f/d5f/d6b/d9d 0 2024-06-03T01:21:10.425 INFO:tasks.workunit.client.0.smithi032.stdout:1/555: mknod d18/d27/d89/c9e 0 2024-06-03T01:21:10.426 INFO:tasks.workunit.client.0.smithi032.stdout:5/712: unlink da/dd/df/d10/l2f 0 2024-06-03T01:21:10.438 INFO:tasks.workunit.client.0.smithi032.stdout:5/713: read da/dd/f92 [73491,13024] 0 2024-06-03T01:21:10.439 INFO:tasks.workunit.client.0.smithi032.stdout:1/556: dwrite d18/d3f/d5f/d6b/d78/f2e [0,4194304] 0 2024-06-03T01:21:10.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/714: dread da/dd/f92 [0,4194304] 0 2024-06-03T01:21:10.445 INFO:tasks.workunit.client.0.smithi032.stdout:5/715: mkdir da/dab/de2 0 2024-06-03T01:21:10.445 INFO:tasks.workunit.client.0.smithi032.stdout:5/716: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fe3 x:0 0 0 2024-06-03T01:21:10.447 INFO:tasks.workunit.client.0.smithi032.stdout:5/717: creat da/dd/df/d10/dbc/d18/db9/fe4 x:0 0 0 2024-06-03T01:21:10.447 INFO:tasks.workunit.client.0.smithi032.stdout:5/718: chown da/dd/df/d10/dbc/d18/d35/d44/dc3/da8 65717981 1 2024-06-03T01:21:10.447 INFO:tasks.workunit.client.0.smithi032.stdout:5/719: symlink da/dd/df/d10/dbc/d18/d35/d44/dc3/da8/le5 0 2024-06-03T01:21:10.448 INFO:tasks.workunit.client.0.smithi032.stdout:5/720: unlink da/dd/df/d10/dbc/d18/d1a/d30/d51/lbb 0 2024-06-03T01:21:10.473 INFO:tasks.workunit.client.0.smithi032.stdout:5/721: rename da/dd/df/d10/dbc/d18/d1a/d30/d51 to da/dd/de6 0 2024-06-03T01:21:10.476 INFO:tasks.workunit.client.0.smithi032.stdout:5/722: dread da/dd/df/d10/dbc/d18/d35/f98 [0,4194304] 0 2024-06-03T01:21:10.477 INFO:tasks.workunit.client.0.smithi032.stdout:5/723: fdatasync da/dd/df/d10/f99 0 2024-06-03T01:21:10.477 INFO:tasks.workunit.client.0.smithi032.stdout:5/724: stat da/dd/df/d10/dbc/d18/d35/d44/dc3/da8 0 2024-06-03T01:21:10.477 INFO:tasks.workunit.client.0.smithi032.stdout:5/725: write da/dd/df/d10/dbc/d25/d74/f8e [614309,76633] 0 2024-06-03T01:21:10.478 INFO:tasks.workunit.client.0.smithi032.stdout:5/726: symlink da/dd/df/d10/dbc/d18/d35/d70/le7 0 2024-06-03T01:21:10.478 INFO:tasks.workunit.client.0.smithi032.stdout:5/727: read da/dd/df/d10/dbc/d18/f50 [3493462,38783] 0 2024-06-03T01:21:10.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/728: symlink da/dd/de6/le8 0 2024-06-03T01:21:10.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/729: read - da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fe3 zero size 2024-06-03T01:21:10.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/730: truncate da/dd/df/d10/dbc/d18/d35/d44/dc3/fcf 1345714 0 2024-06-03T01:21:10.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/731: write da/dd/de6/f7b [18050,99996] 0 2024-06-03T01:21:10.481 INFO:tasks.workunit.client.0.smithi032.stdout:2/737: dwrite d5/da/f59 [0,4194304] 0 2024-06-03T01:21:10.489 INFO:tasks.workunit.client.0.smithi032.stdout:6/521: dwrite d8/d4c/fa0 [0,4194304] 0 2024-06-03T01:21:10.489 INFO:tasks.workunit.client.0.smithi032.stdout:4/657: dwrite d4/db/da/d5d/dc5/fd3 [0,4194304] 0 2024-06-03T01:21:10.491 INFO:tasks.workunit.client.0.smithi032.stdout:9/660: dwrite d0/d11/d1d/fbb [0,4194304] 0 2024-06-03T01:21:10.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/517: dwrite de/d29/d2e/d31/d23/d53/d69/f8e [4194304,4194304] 0 2024-06-03T01:21:10.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/518: dread - de/d29/d2e/d31/d23/f6e zero size 2024-06-03T01:21:10.492 INFO:tasks.workunit.client.0.smithi032.stdout:7/643: dwrite dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:10.492 INFO:tasks.workunit.client.0.smithi032.stdout:0/678: dwrite d1/da/d28/f33 [0,4194304] 0 2024-06-03T01:21:10.501 INFO:tasks.workunit.client.0.smithi032.stdout:7/644: dread dc/d12/d24/f6c [0,4194304] 0 2024-06-03T01:21:10.502 INFO:tasks.workunit.client.0.smithi032.stdout:5/732: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fd2 [0,4194304] 0 2024-06-03T01:21:10.520 INFO:tasks.workunit.client.0.smithi032.stdout:7/645: dwrite dc/dbe/fc7 [0,4194304] 0 2024-06-03T01:21:10.520 INFO:tasks.workunit.client.0.smithi032.stdout:7/646: dread - dc/d12/d22/fad zero size 2024-06-03T01:21:10.636 INFO:tasks.workunit.client.0.smithi032.stdout:6/522: creat d8/d11/d20/d36/da5/fa9 x:0 0 0 2024-06-03T01:21:10.719 INFO:tasks.workunit.client.0.smithi032.stdout:4/658: creat d4/fe8 x:0 0 0 2024-06-03T01:21:10.723 INFO:tasks.workunit.client.0.smithi032.stdout:9/661: mkdir d0/d11/d1d/dda 0 2024-06-03T01:21:10.726 INFO:tasks.workunit.client.0.smithi032.stdout:3/519: rename de/f4e to de/d29/d2e/d31/d24/fa5 0 2024-06-03T01:21:10.729 INFO:tasks.workunit.client.0.smithi032.stdout:0/679: creat d1/d46/dc3/fda x:0 0 0 2024-06-03T01:21:10.729 INFO:tasks.workunit.client.0.smithi032.stdout:0/680: readlink d1/da/d28/la2 0 2024-06-03T01:21:10.732 INFO:tasks.workunit.client.0.smithi032.stdout:5/733: truncate da/dd/df/d10/f42 4162206 0 2024-06-03T01:21:10.736 INFO:tasks.workunit.client.0.smithi032.stdout:7/647: creat dc/d12/d1b/fcb x:0 0 0 2024-06-03T01:21:10.737 INFO:tasks.workunit.client.0.smithi032.stdout:6/523: mkdir d8/d4c/daa 0 2024-06-03T01:21:10.737 INFO:tasks.workunit.client.0.smithi032.stdout:6/524: stat d8/d4c/daa 0 2024-06-03T01:21:10.738 INFO:tasks.workunit.client.0.smithi032.stdout:9/662: mkdir d0/d11/d1d/d38/d4f/db0/ddb 0 2024-06-03T01:21:10.739 INFO:tasks.workunit.client.0.smithi032.stdout:2/738: dwrite d5/da/f29 [0,4194304] 0 2024-06-03T01:21:10.740 INFO:tasks.workunit.client.0.smithi032.stdout:2/739: write d5/da/d1a/d2a/d36/f66 [737062,92722] 0 2024-06-03T01:21:10.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/659: rename d4/db/fe4 to d4/d57/d6c/fe9 0 2024-06-03T01:21:10.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/660: fsync d4/db/da/d2d/d27/d8c/db5/fb0 0 2024-06-03T01:21:10.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/661: chown d4/db/da/d37/f48 914514823 1 2024-06-03T01:21:10.742 INFO:tasks.workunit.client.0.smithi032.stdout:4/662: dread - d4/f43 zero size 2024-06-03T01:21:10.743 INFO:tasks.workunit.client.0.smithi032.stdout:4/663: dread d4/db/da/f1b [0,4194304] 0 2024-06-03T01:21:10.744 INFO:tasks.workunit.client.0.smithi032.stdout:0/681: symlink d1/da/d10/ldb 0 2024-06-03T01:21:10.747 INFO:tasks.workunit.client.0.smithi032.stdout:7/648: symlink dc/d12/lcc 0 2024-06-03T01:21:10.752 INFO:tasks.workunit.client.0.smithi032.stdout:2/740: unlink d5/da/d2e/f32 0 2024-06-03T01:21:10.753 INFO:tasks.workunit.client.0.smithi032.stdout:5/734: dwrite da/f73 [0,4194304] 0 2024-06-03T01:21:10.753 INFO:tasks.workunit.client.0.smithi032.stdout:5/735: chown da/dd/df/d10/dbc/lc4 317117934 1 2024-06-03T01:21:10.756 INFO:tasks.workunit.client.0.smithi032.stdout:9/663: rename d0/d11/d1d/d38/d4f/db0/d41/l53 to d0/d11/d1d/ldc 0 2024-06-03T01:21:10.758 INFO:tasks.workunit.client.0.smithi032.stdout:3/520: getdents de/d29/d2e/d31/d3c/d50/d66 0 2024-06-03T01:21:10.758 INFO:tasks.workunit.client.0.smithi032.stdout:3/521: truncate de/d29/d2e/d31/d23/f6e 327102 0 2024-06-03T01:21:10.769 INFO:tasks.workunit.client.0.smithi032.stdout:5/736: link da/dd/df/d10/dbc/d25/d74/c80 da/dd/df/d10/dbc/d18/d35/d44/dc3/ce9 0 2024-06-03T01:21:10.770 INFO:tasks.workunit.client.0.smithi032.stdout:0/682: dwrite d1/da/d28/f33 [0,4194304] 0 2024-06-03T01:21:10.770 INFO:tasks.workunit.client.0.smithi032.stdout:0/683: chown d1/da/f1a 2290 1 2024-06-03T01:21:10.770 INFO:tasks.workunit.client.0.smithi032.stdout:0/684: creat d1/da/db2/fdc x:0 0 0 2024-06-03T01:21:10.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/741: dwrite d5/da/d1a/d69/d83/fe3 [0,4194304] 0 2024-06-03T01:21:10.771 INFO:tasks.workunit.client.0.smithi032.stdout:0/685: write d1/d46/f98 [1036826,38417] 0 2024-06-03T01:21:10.772 INFO:tasks.workunit.client.0.smithi032.stdout:7/649: rename dc/d12/d6a/d86/d88/f8c to dc/d12/d6a/fcd 0 2024-06-03T01:21:10.776 INFO:tasks.workunit.client.0.smithi032.stdout:3/522: unlink de/d29/d2e/d31/d23/c98 0 2024-06-03T01:21:10.776 INFO:tasks.workunit.client.0.smithi032.stdout:3/523: creat de/d29/d2e/d31/d3c/d50/d5e/fa6 x:0 0 0 2024-06-03T01:21:10.776 INFO:tasks.workunit.client.0.smithi032.stdout:3/524: truncate de/d29/d2e/d31/d23/f4a 782038 0 2024-06-03T01:21:10.776 INFO:tasks.workunit.client.0.smithi032.stdout:2/742: dread d5/d38/d46/d53/d57/dac/f73 [0,4194304] 0 2024-06-03T01:21:10.780 INFO:tasks.workunit.client.0.smithi032.stdout:4/664: mkdir d4/db/da/d2d/d27/d53/dea 0 2024-06-03T01:21:10.781 INFO:tasks.workunit.client.0.smithi032.stdout:2/743: write d5/da/d1a/d69/dea/f99 [572461,111891] 0 2024-06-03T01:21:10.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/650: dread dc/d12/d1b/f2c [0,4194304] 0 2024-06-03T01:21:10.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/651: stat dc/d12/d6a/d86/d89/la4 0 2024-06-03T01:21:10.786 INFO:tasks.workunit.client.0.smithi032.stdout:3/525: dread de/d29/d2e/d31/f21 [0,4194304] 0 2024-06-03T01:21:10.787 INFO:tasks.workunit.client.0.smithi032.stdout:5/737: dwrite da/dd/df/d10/dbc/d18/d1a/d30/f94 [0,4194304] 0 2024-06-03T01:21:10.787 INFO:tasks.workunit.client.0.smithi032.stdout:0/686: unlink d1/da/d10/d80/c93 0 2024-06-03T01:21:10.788 INFO:tasks.workunit.client.0.smithi032.stdout:9/664: rename d0/d11/d1d/dda to d0/ddd 0 2024-06-03T01:21:10.788 INFO:tasks.workunit.client.0.smithi032.stdout:9/665: write d0/d11/d1d/d38/d4f/fa1 [297400,35455] 0 2024-06-03T01:21:10.793 INFO:tasks.workunit.client.0.smithi032.stdout:4/665: rmdir d4/d57 39 2024-06-03T01:21:10.797 INFO:tasks.workunit.client.0.smithi032.stdout:2/744: fdatasync d5/da/d1a/f26 0 2024-06-03T01:21:10.800 INFO:tasks.workunit.client.0.smithi032.stdout:3/526: creat de/d29/fa7 x:0 0 0 2024-06-03T01:21:10.802 INFO:tasks.workunit.client.0.smithi032.stdout:5/738: symlink da/dd/lea 0 2024-06-03T01:21:10.803 INFO:tasks.workunit.client.0.smithi032.stdout:0/687: rmdir d1/da/dd2 39 2024-06-03T01:21:10.803 INFO:tasks.workunit.client.0.smithi032.stdout:0/688: stat d1/c75 0 2024-06-03T01:21:10.806 INFO:tasks.workunit.client.0.smithi032.stdout:9/666: dwrite d0/d11/d1d/f99 [0,4194304] 0 2024-06-03T01:21:10.806 INFO:tasks.workunit.client.0.smithi032.stdout:9/667: read - d0/d11/d1d/d38/d4f/fc3 zero size 2024-06-03T01:21:10.807 INFO:tasks.workunit.client.0.smithi032.stdout:5/739: dread da/dd/f92 [0,4194304] 0 2024-06-03T01:21:10.810 INFO:tasks.workunit.client.0.smithi032.stdout:4/666: creat d4/db/da/d1d/d6b/feb x:0 0 0 2024-06-03T01:21:10.811 INFO:tasks.workunit.client.0.smithi032.stdout:2/745: write d5/d38/d46/d5c/d6c/fdf [348616,64319] 0 2024-06-03T01:21:10.812 INFO:tasks.workunit.client.0.smithi032.stdout:0/689: dread d1/d46/f48 [0,4194304] 0 2024-06-03T01:21:10.813 INFO:tasks.workunit.client.0.smithi032.stdout:5/740: dread da/f17 [0,4194304] 0 2024-06-03T01:21:10.813 INFO:tasks.workunit.client.0.smithi032.stdout:5/741: chown da/dd/le0 3450723 1 2024-06-03T01:21:10.813 INFO:tasks.workunit.client.0.smithi032.stdout:3/527: rename de/d29/d2e/d31/d24/l6a to de/d29/d2e/d31/d3c/d50/d5e/la8 0 2024-06-03T01:21:10.813 INFO:tasks.workunit.client.0.smithi032.stdout:5/742: write da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fbe [370353,113603] 0 2024-06-03T01:21:10.813 INFO:tasks.workunit.client.0.smithi032.stdout:5/743: truncate da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fd2 4283019 0 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:3/528: creat de/d29/d2e/d31/d23/d53/d69/d8c/fa9 x:0 0 0 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:5/744: write da/dd/df/d10/dbc/d18/f1d [3570262,69315] 0 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:5/745: dread - da/dd/df/d10/dbc/d18/fda zero size 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:0/690: dread d1/da/d10/d29/fd0 [0,4194304] 0 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/668: symlink d0/d11/d76/lde 0 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/669: read - d0/d11/fa5 zero size 2024-06-03T01:21:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/670: readlink d0/d11/d1d/d38/d4f/db0/l95 0 2024-06-03T01:21:10.817 INFO:tasks.workunit.client.0.smithi032.stdout:4/667: mkdir d4/db/da/d37/dab/dec 0 2024-06-03T01:21:10.817 INFO:tasks.workunit.client.0.smithi032.stdout:4/668: chown d4/db/da/f3f 27928 1 2024-06-03T01:21:10.818 INFO:tasks.workunit.client.0.smithi032.stdout:2/746: truncate d5/da/f39 709047 0 2024-06-03T01:21:10.818 INFO:tasks.workunit.client.0.smithi032.stdout:2/747: fsync d5/d38/d46/d53/d57/dac/de0/fe1 0 2024-06-03T01:21:10.818 INFO:tasks.workunit.client.0.smithi032.stdout:2/748: write d5/f23 [5539586,87600] 0 2024-06-03T01:21:10.819 INFO:tasks.workunit.client.0.smithi032.stdout:3/529: rmdir de/d29/d2e/d31/d3c/d50/d62 39 2024-06-03T01:21:10.819 INFO:tasks.workunit.client.0.smithi032.stdout:3/530: fsync de/d29/d2e/d31/d23/d53/d69/d8c/f97 0 2024-06-03T01:21:10.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/691: mkdir d1/da/d28/d3b/d68/ddd 0 2024-06-03T01:21:10.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/692: fdatasync d1/da/d10/d80/d9f/dac/fbd 0 2024-06-03T01:21:10.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/693: write d1/da/d28/d3b/d5d/d8d/f91 [46954,1636] 0 2024-06-03T01:21:10.824 INFO:tasks.workunit.client.0.smithi032.stdout:4/669: mknod d4/db/da/d37/dab/dec/ced 0 2024-06-03T01:21:10.824 INFO:tasks.workunit.client.0.smithi032.stdout:4/670: readlink d4/db/da/d2d/d27/ld9 0 2024-06-03T01:21:10.828 INFO:tasks.workunit.client.0.smithi032.stdout:2/749: rename d5/da/d1a/d69/d83/fa1 to d5/d38/d46/d53/da9/ff7 0 2024-06-03T01:21:10.830 INFO:tasks.workunit.client.0.smithi032.stdout:2/750: truncate d5/d38/d46/f81 2831971 0 2024-06-03T01:21:10.834 INFO:tasks.workunit.client.0.smithi032.stdout:3/531: rename de/d29/d2e/fa0 to de/d29/d2e/d31/d8d/faa 0 2024-06-03T01:21:10.838 INFO:tasks.workunit.client.0.smithi032.stdout:2/751: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/df8 0 2024-06-03T01:21:10.840 INFO:tasks.workunit.client.0.smithi032.stdout:3/532: mkdir de/d29/d2e/d31/d23/d53/d79/dab 0 2024-06-03T01:21:10.843 INFO:tasks.workunit.client.0.smithi032.stdout:9/671: dwrite d0/d11/f63 [0,4194304] 0 2024-06-03T01:21:10.843 INFO:tasks.workunit.client.0.smithi032.stdout:3/533: unlink de/d29/c58 0 2024-06-03T01:21:10.845 INFO:tasks.workunit.client.0.smithi032.stdout:9/672: creat d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/fdf x:0 0 0 2024-06-03T01:21:10.845 INFO:tasks.workunit.client.0.smithi032.stdout:9/673: stat d0/d11/d1d/d38/f3e 0 2024-06-03T01:21:10.846 INFO:tasks.workunit.client.0.smithi032.stdout:5/746: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fbe [0,4194304] 0 2024-06-03T01:21:10.847 INFO:tasks.workunit.client.0.smithi032.stdout:2/752: dread d5/d38/d46/d53/d57/dac/f75 [0,4194304] 0 2024-06-03T01:21:10.848 INFO:tasks.workunit.client.0.smithi032.stdout:3/534: rename de/d29/c48 to de/d29/d2e/d31/d23/d53/d79/cac 0 2024-06-03T01:21:10.851 INFO:tasks.workunit.client.0.smithi032.stdout:0/694: dwrite d1/da/d28/f8b [0,4194304] 0 2024-06-03T01:21:10.851 INFO:tasks.workunit.client.0.smithi032.stdout:4/671: dwrite d4/db/da/f49 [0,4194304] 0 2024-06-03T01:21:10.851 INFO:tasks.workunit.client.0.smithi032.stdout:0/695: chown d1/da/d10/f6f 1660889 1 2024-06-03T01:21:10.851 INFO:tasks.workunit.client.0.smithi032.stdout:4/672: write d4/db/da/d5d/d8e/fa3 [677020,17768] 0 2024-06-03T01:21:10.855 INFO:tasks.workunit.client.0.smithi032.stdout:5/747: mkdir da/dd/de6/deb 0 2024-06-03T01:21:10.855 INFO:tasks.workunit.client.0.smithi032.stdout:5/748: write da/dd/df/d10/dbc/d18/d1a/d30/f5b [1706663,66868] 0 2024-06-03T01:21:10.855 INFO:tasks.workunit.client.0.smithi032.stdout:5/749: dread - da/dd/de6/fa2 zero size 2024-06-03T01:21:10.858 INFO:tasks.workunit.client.0.smithi032.stdout:3/535: mknod de/d29/d2e/d31/d3c/d50/d5e/d7e/cad 0 2024-06-03T01:21:10.859 INFO:tasks.workunit.client.0.smithi032.stdout:9/674: dread d0/d11/d76/fa7 [0,4194304] 0 2024-06-03T01:21:10.865 INFO:tasks.workunit.client.0.smithi032.stdout:2/753: getdents d5/d38/d46/d53/d57/d6d/dce/d8f 0 2024-06-03T01:21:10.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/675: rename d0/d11/d1d/d38/d4f/db0/d41/fc0 to d0/da4/fe0 0 2024-06-03T01:21:10.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/676: chown d0/d11/d1d/d38/d4f/db0/c2c 1268953 1 2024-06-03T01:21:10.873 INFO:tasks.workunit.client.0.smithi032.stdout:2/754: write d5/d38/f3b [8869419,19682] 0 2024-06-03T01:21:10.878 INFO:tasks.workunit.client.0.smithi032.stdout:3/536: write de/d29/d2e/d31/d23/d53/d69/f8e [5610283,96597] 0 2024-06-03T01:21:10.878 INFO:tasks.workunit.client.0.smithi032.stdout:0/696: dwrite d1/da/d10/d29/fbc [0,4194304] 0 2024-06-03T01:21:10.881 INFO:tasks.workunit.client.0.smithi032.stdout:9/677: write d0/d11/d9c/fa3 [2874191,87975] 0 2024-06-03T01:21:10.883 INFO:tasks.workunit.client.0.smithi032.stdout:0/697: rename d1/da/f9e to d1/da/d28/d3b/d68/db5/fde 0 2024-06-03T01:21:10.886 INFO:tasks.workunit.client.0.smithi032.stdout:3/537: dread de/d29/d2e/d31/d24/f39 [0,4194304] 0 2024-06-03T01:21:10.886 INFO:tasks.workunit.client.0.smithi032.stdout:3/538: creat de/d29/d2e/d31/d3c/d50/d56/fae x:0 0 0 2024-06-03T01:21:10.886 INFO:tasks.workunit.client.0.smithi032.stdout:5/750: dwrite da/dd/df/f33 [0,4194304] 0 2024-06-03T01:21:10.886 INFO:tasks.workunit.client.0.smithi032.stdout:9/678: mkdir d0/d23/d25/de1 0 2024-06-03T01:21:10.888 INFO:tasks.workunit.client.0.smithi032.stdout:5/751: dread da/dd/df/d10/dbc/d18/d35/d70/f95 [0,4194304] 0 2024-06-03T01:21:10.888 INFO:tasks.workunit.client.0.smithi032.stdout:0/698: mknod d1/da/d28/d3b/d68/db5/cdf 0 2024-06-03T01:21:10.890 INFO:tasks.workunit.client.0.smithi032.stdout:3/539: mkdir de/d29/d2e/d31/d24/daf 0 2024-06-03T01:21:10.896 INFO:tasks.workunit.client.0.smithi032.stdout:9/679: rename d0/d11/d79/f9f to d0/d23/d25/dd6/fe2 0 2024-06-03T01:21:10.896 INFO:tasks.workunit.client.0.smithi032.stdout:9/680: write d0/d23/d25/f8e [2844797,126892] 0 2024-06-03T01:21:10.896 INFO:tasks.workunit.client.0.smithi032.stdout:9/681: stat d0/d23/d25/l6f 0 2024-06-03T01:21:10.896 INFO:tasks.workunit.client.0.smithi032.stdout:2/755: dwrite d5/d38/d46/d53/d57/d6d/dce/fca [0,4194304] 0 2024-06-03T01:21:10.898 INFO:tasks.workunit.client.0.smithi032.stdout:0/699: mknod d1/da/d28/d3b/d68/db5/ce0 0 2024-06-03T01:21:10.898 INFO:tasks.workunit.client.0.smithi032.stdout:0/700: chown d1/da/d28/d3b/d5d/da9/daa/lc8 5182753 1 2024-06-03T01:21:10.899 INFO:tasks.workunit.client.0.smithi032.stdout:0/701: mknod d1/da/d10/d29/ce1 0 2024-06-03T01:21:10.901 INFO:tasks.workunit.client.0.smithi032.stdout:0/702: unlink d1/da/d28/d3b/d5d/da9/fd8 0 2024-06-03T01:21:10.902 INFO:tasks.workunit.client.0.smithi032.stdout:0/703: dread d1/da/d10/d53/f7e [0,4194304] 0 2024-06-03T01:21:10.903 INFO:tasks.workunit.client.0.smithi032.stdout:0/704: link d1/da/l42 d1/da/d28/d3b/d68/db5/le2 0 2024-06-03T01:21:10.914 INFO:tasks.workunit.client.0.smithi032.stdout:5/752: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fcb [0,4194304] 0 2024-06-03T01:21:10.915 INFO:tasks.workunit.client.0.smithi032.stdout:9/682: truncate d0/d11/d1d/f99 2511612 0 2024-06-03T01:21:10.915 INFO:tasks.workunit.client.0.smithi032.stdout:9/683: write d0/d11/d1d/d38/d4f/fc3 [298649,81674] 0 2024-06-03T01:21:10.915 INFO:tasks.workunit.client.0.smithi032.stdout:9/684: stat d0/d11/d1d/d38/d4f/db0/f39 0 2024-06-03T01:21:10.918 INFO:tasks.workunit.client.0.smithi032.stdout:9/685: rename d0/d11/d1d/d38/d4f/f60 to d0/fe3 0 2024-06-03T01:21:10.919 INFO:tasks.workunit.client.0.smithi032.stdout:9/686: mknod d0/d23/d24/ce4 0 2024-06-03T01:21:10.927 INFO:tasks.workunit.client.0.smithi032.stdout:5/753: read da/dd/df/f46 [4872747,45174] 0 2024-06-03T01:21:10.927 INFO:tasks.workunit.client.0.smithi032.stdout:5/754: symlink da/dd/df/d10/dbc/d18/lec 0 2024-06-03T01:21:10.928 INFO:tasks.workunit.client.0.smithi032.stdout:5/755: write da/f17 [757838,41966] 0 2024-06-03T01:21:10.928 INFO:tasks.workunit.client.0.smithi032.stdout:3/540: dwrite f9 [0,4194304] 0 2024-06-03T01:21:10.935 INFO:tasks.workunit.client.0.smithi032.stdout:2/756: dwrite d5/da/d1a/d2a/d36/dc3/fdb [0,4194304] 0 2024-06-03T01:21:10.935 INFO:tasks.workunit.client.0.smithi032.stdout:9/687: dread d0/ff [0,4194304] 0 2024-06-03T01:21:10.938 INFO:tasks.workunit.client.0.smithi032.stdout:9/688: getdents d0/d11/d1d/d38/d4f/db0/d41/dc6 0 2024-06-03T01:21:10.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/757: mkdir d5/da/d1a/d69/d83/df9 0 2024-06-03T01:21:10.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/758: write d5/d38/d46/d53/d57/d6d/dce/d8f/fee [358613,109317] 0 2024-06-03T01:21:10.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/759: fdatasync d5/da/d1a/d2a/d36/d3f/fd6 0 2024-06-03T01:21:10.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/689: creat d0/d23/d25/de1/fe5 x:0 0 0 2024-06-03T01:21:10.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/690: chown d0/d23/d25/f52 2 1 2024-06-03T01:21:10.942 INFO:tasks.workunit.client.0.smithi032.stdout:9/691: creat d0/d23/d25/dd6/fe6 x:0 0 0 2024-06-03T01:21:10.942 INFO:tasks.workunit.client.0.smithi032.stdout:9/692: chown d0/d23/d24/c59 230518762 1 2024-06-03T01:21:10.942 INFO:tasks.workunit.client.0.smithi032.stdout:2/760: mkdir d5/d38/dfa 0 2024-06-03T01:21:10.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/761: getdents d5/da/d1a/d2a/d36/d3f 0 2024-06-03T01:21:10.951 INFO:tasks.workunit.client.0.smithi032.stdout:5/756: dwrite da/dd/df/d10/dbc/d18/d35/d70/fb3 [4194304,4194304] 0 2024-06-03T01:21:10.951 INFO:tasks.workunit.client.0.smithi032.stdout:3/541: dwrite de/d29/d2e/d31/d23/f6e [0,4194304] 0 2024-06-03T01:21:10.958 INFO:tasks.workunit.client.0.smithi032.stdout:3/542: symlink de/d29/d2e/d31/d23/lb0 0 2024-06-03T01:21:10.961 INFO:tasks.workunit.client.0.smithi032.stdout:5/757: write da/dd/df/d10/dbc/d18/d35/d44/dc3/f7f [3555703,67895] 0 2024-06-03T01:21:10.961 INFO:tasks.workunit.client.0.smithi032.stdout:5/758: chown da/dd/df/d10/dbc/fa0 0 1 2024-06-03T01:21:10.963 INFO:tasks.workunit.client.0.smithi032.stdout:9/693: dwrite d0/d11/d1d/fbb [0,4194304] 0 2024-06-03T01:21:10.963 INFO:tasks.workunit.client.0.smithi032.stdout:3/543: creat de/d29/d2e/d31/d3c/d50/d62/fb1 x:0 0 0 2024-06-03T01:21:10.968 INFO:tasks.workunit.client.0.smithi032.stdout:5/759: creat da/dd/df/d10/dbc/d18/d35/d70/fed x:0 0 0 2024-06-03T01:21:10.968 INFO:tasks.workunit.client.0.smithi032.stdout:5/760: truncate da/dd/df/d10/dbc/d18/d35/d70/f95 1464149 0 2024-06-03T01:21:10.971 INFO:tasks.workunit.client.0.smithi032.stdout:2/762: dwrite d5/d38/d46/d53/fa7 [4194304,4194304] 0 2024-06-03T01:21:10.976 INFO:tasks.workunit.client.0.smithi032.stdout:2/763: dread d5/f3c [0,4194304] 0 2024-06-03T01:21:10.977 INFO:tasks.workunit.client.0.smithi032.stdout:2/764: creat d5/d38/d46/d53/da9/ffb x:0 0 0 2024-06-03T01:21:10.990 INFO:tasks.workunit.client.0.smithi032.stdout:2/765: dread d5/da/d1a/d2a/d36/d3f/f4f [0,4194304] 0 2024-06-03T01:21:10.991 INFO:tasks.workunit.client.0.smithi032.stdout:2/766: truncate d5/d38/d46/d53/d90/f9b 3934773 0 2024-06-03T01:21:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:2/767: symlink d5/d38/d46/d53/d57/dac/lfc 0 2024-06-03T01:21:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:2/768: chown d5/d38/d46/d5c/d6c/fdf 3 1 2024-06-03T01:21:10.994 INFO:tasks.workunit.client.0.smithi032.stdout:9/694: dwrite d0/fd [0,4194304] 0 2024-06-03T01:21:10.996 INFO:tasks.workunit.client.0.smithi032.stdout:9/695: creat d0/d11/d1d/d38/d4f/db0/ddb/fe7 x:0 0 0 2024-06-03T01:21:10.996 INFO:tasks.workunit.client.0.smithi032.stdout:9/696: write d0/d23/f3d [1493595,126063] 0 2024-06-03T01:21:10.996 INFO:tasks.workunit.client.0.smithi032.stdout:9/697: chown d0/d11/l5f 13049656 1 2024-06-03T01:21:10.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/698: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/de8 0 2024-06-03T01:21:10.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/699: creat d0/da4/fe9 x:0 0 0 2024-06-03T01:21:10.997 INFO:tasks.workunit.client.0.smithi032.stdout:3/544: dwrite de/d29/d2e/d31/d24/f73 [0,4194304] 0 2024-06-03T01:21:10.997 INFO:tasks.workunit.client.0.smithi032.stdout:3/545: write de/d29/d2e/d31/d23/f4a [908059,101463] 0 2024-06-03T01:21:11.000 INFO:tasks.workunit.client.0.smithi032.stdout:5/761: dwrite da/dd/df/d10/dbc/d18/d1a/fb0 [0,4194304] 0 2024-06-03T01:21:11.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:10 smithi032 ceph-mon[29360]: pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 233 MiB/s rd, 283 MiB/s wr, 423 op/s 2024-06-03T01:21:11.100 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:10 smithi044 ceph-mon[36956]: pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 233 MiB/s rd, 283 MiB/s wr, 423 op/s 2024-06-03T01:21:11.164 INFO:tasks.workunit.client.0.smithi032.stdout:5/762: write da/dd/df/f97 [253283,41341] 0 2024-06-03T01:21:11.164 INFO:tasks.workunit.client.0.smithi032.stdout:2/769: dwrite d5/d38/d46/d53/da9/ff7 [0,4194304] 0 2024-06-03T01:21:11.164 INFO:tasks.workunit.client.0.smithi032.stdout:2/770: creat d5/d38/d46/ffd x:0 0 0 2024-06-03T01:21:11.167 INFO:tasks.workunit.client.0.smithi032.stdout:5/763: unlink da/dd/df/d10/dbc/fa9 0 2024-06-03T01:21:11.168 INFO:tasks.workunit.client.0.smithi032.stdout:9/700: write d0/d11/d1d/da6/fa2 [3523605,74738] 0 2024-06-03T01:21:11.168 INFO:tasks.workunit.client.0.smithi032.stdout:9/701: write d0/f57 [3223928,35885] 0 2024-06-03T01:21:11.181 INFO:tasks.workunit.client.0.smithi032.stdout:2/771: write d5/fe [2739129,102327] 0 2024-06-03T01:21:11.185 INFO:tasks.workunit.client.0.smithi032.stdout:2/772: dread d5/d38/d46/d5c/f8b [0,4194304] 0 2024-06-03T01:21:11.185 INFO:tasks.workunit.client.0.smithi032.stdout:2/773: stat d5/d38/d46/d53/d57/d6d/dce/l4c 0 2024-06-03T01:21:11.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/546: dwrite de/f11 [4194304,4194304] 0 2024-06-03T01:21:11.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/547: creat de/d63/fb2 x:0 0 0 2024-06-03T01:21:11.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/764: dwrite da/f2a [0,4194304] 0 2024-06-03T01:21:11.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/765: read - da/dd/df/d10/dbc/d18/fda zero size 2024-06-03T01:21:11.195 INFO:tasks.workunit.client.0.smithi032.stdout:5/766: chown da/dd/df/d10/dbc/d25/d74/la7 13755 1 2024-06-03T01:21:11.196 INFO:tasks.workunit.client.0.smithi032.stdout:5/767: mknod da/dd/df/d10/cee 0 2024-06-03T01:21:11.198 INFO:tasks.workunit.client.0.smithi032.stdout:5/768: rmdir da/dd/df/d10/dbc/d18/d35 39 2024-06-03T01:21:11.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/769: rename da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fd2 to da/dd/df/d10/dbc/d25/fef 0 2024-06-03T01:21:11.202 INFO:tasks.workunit.client.0.smithi032.stdout:5/770: creat da/dd/df/d10/dbc/d18/d1a/ff0 x:0 0 0 2024-06-03T01:21:11.209 INFO:tasks.workunit.client.0.smithi032.stdout:3/548: dwrite de/d29/d2e/d31/d23/f59 [0,4194304] 0 2024-06-03T01:21:11.210 INFO:tasks.workunit.client.0.smithi032.stdout:2/774: dwrite d5/d38/d46/d53/d57/dac/f54 [0,4194304] 0 2024-06-03T01:21:11.214 INFO:tasks.workunit.client.0.smithi032.stdout:2/775: symlink d5/lfe 0 2024-06-03T01:21:11.220 INFO:tasks.workunit.client.0.smithi032.stdout:5/771: dwrite da/dd/df/d10/dbc/d18/d35/d70/f86 [0,4194304] 0 2024-06-03T01:21:11.220 INFO:tasks.workunit.client.0.smithi032.stdout:5/772: fsync da/dd/fd3 0 2024-06-03T01:21:11.222 INFO:tasks.workunit.client.0.smithi032.stdout:3/549: rmdir de/d29/d2e 39 2024-06-03T01:21:11.224 INFO:tasks.workunit.client.0.smithi032.stdout:2/776: symlink d5/da/d2e/dec/lff 0 2024-06-03T01:21:11.224 INFO:tasks.workunit.client.0.smithi032.stdout:2/777: chown d5/da/d1a/d2a/d36/d3f/f78 41128 1 2024-06-03T01:21:11.226 INFO:tasks.workunit.client.0.smithi032.stdout:5/773: mknod da/dab/cf1 0 2024-06-03T01:21:11.230 INFO:tasks.workunit.client.0.smithi032.stdout:5/774: rename da/dd/df/d10/dbc/d18/faa to da/dd/df/d10/dbc/d18/d35/d44/dc3/da8/ff2 0 2024-06-03T01:21:11.230 INFO:tasks.workunit.client.0.smithi032.stdout:5/775: fdatasync da/dd/df/d10/dbc/d18/d35/f98 0 2024-06-03T01:21:11.233 INFO:tasks.workunit.client.0.smithi032.stdout:5/776: symlink da/dd/df/d10/dbc/d18/d35/d44/dc3/da8/lf3 0 2024-06-03T01:21:11.233 INFO:tasks.workunit.client.0.smithi032.stdout:5/777: write da/dd/df/f97 [1305190,47738] 0 2024-06-03T01:21:11.234 INFO:tasks.workunit.client.0.smithi032.stdout:3/550: dread de/d29/d2e/d31/d24/fa5 [0,4194304] 0 2024-06-03T01:21:11.234 INFO:tasks.workunit.client.0.smithi032.stdout:3/551: write de/d29/f95 [357762,61261] 0 2024-06-03T01:21:11.235 INFO:tasks.workunit.client.0.smithi032.stdout:5/778: mknod da/dd/de6/d7a/cf4 0 2024-06-03T01:21:11.236 INFO:tasks.workunit.client.0.smithi032.stdout:5/779: symlink da/dd/df/d10/dbc/d18/d1a/lf5 0 2024-06-03T01:21:11.237 INFO:tasks.workunit.client.0.smithi032.stdout:5/780: read da/dd/df/d10/dbc/d18/d35/d44/fb7 [113645,15213] 0 2024-06-03T01:21:11.238 INFO:tasks.workunit.client.0.smithi032.stdout:3/552: dread de/d29/f3f [0,4194304] 0 2024-06-03T01:21:11.238 INFO:tasks.workunit.client.0.smithi032.stdout:3/553: chown de/d29/d2e/d31/d23/f6e 20403 1 2024-06-03T01:21:11.239 INFO:tasks.workunit.client.0.smithi032.stdout:5/781: truncate da/dd/fdf 3641655 0 2024-06-03T01:21:11.240 INFO:tasks.workunit.client.0.smithi032.stdout:2/778: dwrite d5/d38/d46/d53/d57/dac/f75 [0,4194304] 0 2024-06-03T01:21:11.240 INFO:tasks.workunit.client.0.smithi032.stdout:2/779: creat d5/d38/d46/d53/da9/f100 x:0 0 0 2024-06-03T01:21:11.241 INFO:tasks.workunit.client.0.smithi032.stdout:2/780: write d5/f35 [5148771,73893] 0 2024-06-03T01:21:11.248 INFO:tasks.workunit.client.0.smithi032.stdout:2/781: mknod d5/d38/d46/d53/d57/dac/c101 0 2024-06-03T01:21:11.253 INFO:tasks.workunit.client.0.smithi032.stdout:5/782: read da/dd/df/d10/dbc/d18/d35/d44/f72 [3843296,95142] 0 2024-06-03T01:21:11.254 INFO:tasks.workunit.client.0.smithi032.stdout:3/554: dwrite de/d29/d2e/d31/d3c/d50/d56/f60 [0,4194304] 0 2024-06-03T01:21:11.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/555: creat de/d29/d2e/d31/d3c/d50/d5e/d7e/fb3 x:0 0 0 2024-06-03T01:21:11.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/556: write de/d29/d2e/d31/d3c/f44 [5276039,51836] 0 2024-06-03T01:21:11.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/557: truncate de/d29/d2e/d31/f32 5883827 0 2024-06-03T01:21:11.259 INFO:tasks.workunit.client.0.smithi032.stdout:3/558: mkdir de/d29/d2e/d31/d23/d53/d69/d8c/db4 0 2024-06-03T01:21:11.262 INFO:tasks.workunit.client.0.smithi032.stdout:2/782: dread d5/da/f2c [0,4194304] 0 2024-06-03T01:21:11.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/783: write da/f2a [3573325,130274] 0 2024-06-03T01:21:11.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/784: write da/dd/df/d10/dbc/d18/d1a/f77 [1398179,83933] 0 2024-06-03T01:21:11.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/785: write da/dd/df/d10/dbc/d18/f50 [1297495,127093] 0 2024-06-03T01:21:11.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/786: readlink da/dd/lc7 0 2024-06-03T01:21:11.267 INFO:tasks.workunit.client.0.smithi032.stdout:2/783: rmdir d5/d38/d46/d53/d90/def 0 2024-06-03T01:21:11.267 INFO:tasks.workunit.client.0.smithi032.stdout:2/784: fdatasync d5/d38/f3b 0 2024-06-03T01:21:11.273 INFO:tasks.workunit.client.0.smithi032.stdout:2/785: stat d5/da/d1a/f7b 0 2024-06-03T01:21:11.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/786: symlink d5/d38/d46/dbe/l102 0 2024-06-03T01:21:11.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/787: fsync d5/da/d1a/fa5 0 2024-06-03T01:21:11.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/788: chown d5/d38/d46/d5c/d6c/d80/cd4 193070326 1 2024-06-03T01:21:11.279 INFO:tasks.workunit.client.0.smithi032.stdout:2/789: symlink d5/d38/d46/d53/d57/d6d/dce/d8f/l103 0 2024-06-03T01:21:11.279 INFO:tasks.workunit.client.0.smithi032.stdout:2/790: read d5/d38/d46/d53/d57/d6d/f98 [24583,52938] 0 2024-06-03T01:21:11.286 INFO:tasks.workunit.client.0.smithi032.stdout:3/559: dwrite de/d29/d2e/d31/d3c/d50/d62/fa2 [0,4194304] 0 2024-06-03T01:21:11.286 INFO:tasks.workunit.client.0.smithi032.stdout:3/560: creat de/d29/fb5 x:0 0 0 2024-06-03T01:21:11.287 INFO:tasks.workunit.client.0.smithi032.stdout:2/791: rename d5/da/d1a/d69 to d5/d38/d46/d53/d57/d6d/dce/d67/d104 0 2024-06-03T01:21:11.294 INFO:tasks.workunit.client.0.smithi032.stdout:2/792: unlink d5/d38/db5/fcf 0 2024-06-03T01:21:11.296 INFO:tasks.workunit.client.0.smithi032.stdout:2/793: mknod d5/da/d1a/d2a/c105 0 2024-06-03T01:21:11.298 INFO:tasks.workunit.client.0.smithi032.stdout:2/794: chown d5/da/d1a/c47 24631832 1 2024-06-03T01:21:11.298 INFO:tasks.workunit.client.0.smithi032.stdout:2/795: stat d5/da/d1a/d2a/da8/db6/cd8 0 2024-06-03T01:21:11.301 INFO:tasks.workunit.client.0.smithi032.stdout:3/561: dread de/d29/d2e/d31/d23/d53/d69/f8e [4194304,4194304] 0 2024-06-03T01:21:11.302 INFO:tasks.workunit.client.0.smithi032.stdout:3/562: chown de/d29/d2e/d31/d3c/f61 170 1 2024-06-03T01:21:11.303 INFO:tasks.workunit.client.0.smithi032.stdout:3/563: creat de/d29/d2e/d31/d3c/fb6 x:0 0 0 2024-06-03T01:21:11.303 INFO:tasks.workunit.client.0.smithi032.stdout:3/564: chown de/d29/d2e/d31/d3c/f61 3990 1 2024-06-03T01:21:11.303 INFO:tasks.workunit.client.0.smithi032.stdout:5/787: dwrite da/dd/df/d10/dbc/d25/f8b [0,4194304] 0 2024-06-03T01:21:11.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/788: write da/dd/df/d10/dbc/d25/d74/f8e [1501515,79241] 0 2024-06-03T01:21:11.304 INFO:tasks.workunit.client.0.smithi032.stdout:3/565: symlink de/d29/d2e/d31/d3c/d50/d62/lb7 0 2024-06-03T01:21:11.304 INFO:tasks.workunit.client.0.smithi032.stdout:3/566: read de/f5b [267710,88183] 0 2024-06-03T01:21:11.306 INFO:tasks.workunit.client.0.smithi032.stdout:5/789: chown da/dd/df/lb4 8 1 2024-06-03T01:21:11.308 INFO:tasks.workunit.client.0.smithi032.stdout:3/567: creat de/d29/d2e/d31/d3c/d50/d62/fb8 x:0 0 0 2024-06-03T01:21:11.308 INFO:tasks.workunit.client.0.smithi032.stdout:3/568: chown de/d29/d2e/c7b 2984 1 2024-06-03T01:21:11.308 INFO:tasks.workunit.client.0.smithi032.stdout:3/569: fdatasync de/d29/d2e/d31/d23/d53/d69/d8c/fa9 0 2024-06-03T01:21:11.309 INFO:tasks.workunit.client.0.smithi032.stdout:5/790: symlink da/dab/de2/lf6 0 2024-06-03T01:21:11.309 INFO:tasks.workunit.client.0.smithi032.stdout:3/570: symlink de/d63/lb9 0 2024-06-03T01:21:11.312 INFO:tasks.workunit.client.0.smithi032.stdout:5/791: link da/dd/df/d10/dbc/d18/d35/f5d da/dd/df/d10/dbc/d18/d1a/d30/ff7 0 2024-06-03T01:21:11.313 INFO:tasks.workunit.client.0.smithi032.stdout:5/792: mknod da/dd/de6/d7a/cf8 0 2024-06-03T01:21:11.316 INFO:tasks.workunit.client.0.smithi032.stdout:2/796: dwrite d5/da/d1a/d2a/d36/d3f/f78 [0,4194304] 0 2024-06-03T01:21:11.317 INFO:tasks.workunit.client.0.smithi032.stdout:2/797: readlink d5/d38/d46/d53/da9/lc0 0 2024-06-03T01:21:11.323 INFO:tasks.workunit.client.0.smithi032.stdout:3/571: dread de/d29/d2e/f4d [0,4194304] 0 2024-06-03T01:21:11.326 INFO:tasks.workunit.client.0.smithi032.stdout:3/572: rename de/d29/d2e/d31/f96 to de/d29/d2e/d31/d3c/d50/d5e/fba 0 2024-06-03T01:21:11.326 INFO:tasks.workunit.client.0.smithi032.stdout:3/573: rename de/d29/d2e/d31/d24/daf to de/d29/d2e/d31/d24/daf/dbb 22 2024-06-03T01:21:11.326 INFO:tasks.workunit.client.0.smithi032.stdout:3/574: rename de to de/d29/d2e/d31/d3c/d50/d5e/dbc 22 2024-06-03T01:21:11.329 INFO:tasks.workunit.client.0.smithi032.stdout:3/575: link de/d29/d2e/d31/d23/f6e de/d29/d2e/d31/d3c/d50/d5e/fbd 0 2024-06-03T01:21:11.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/576: creat de/d29/d2e/d31/d78/fbe x:0 0 0 2024-06-03T01:21:11.333 INFO:tasks.workunit.client.0.smithi032.stdout:3/577: unlink de/d29/d2e/d31/d3c/d50/d66/f68 0 2024-06-03T01:21:11.334 INFO:tasks.workunit.client.0.smithi032.stdout:3/578: stat de/d29/d2e/d31/d8d/f9d 0 2024-06-03T01:21:11.338 INFO:tasks.workunit.client.0.smithi032.stdout:3/579: mkdir de/d29/d2e/d31/d23/d53/dbf 0 2024-06-03T01:21:11.338 INFO:tasks.workunit.client.0.smithi032.stdout:3/580: link de/d29/f65 de/d29/d2e/d31/d3c/fc0 0 2024-06-03T01:21:11.338 INFO:tasks.workunit.client.0.smithi032.stdout:3/581: write de/d29/d2e/d31/d24/f2f [3365451,5896] 0 2024-06-03T01:21:11.342 INFO:tasks.workunit.client.0.smithi032.stdout:2/798: dwrite d5/d38/d46/d53/d57/fe8 [0,4194304] 0 2024-06-03T01:21:11.343 INFO:tasks.workunit.client.0.smithi032.stdout:5/793: dwrite da/dd/df/d10/dbc/d25/fa4 [0,4194304] 0 2024-06-03T01:21:11.345 INFO:tasks.workunit.client.0.smithi032.stdout:2/799: creat d5/d38/d46/d53/d57/d6d/dce/d67/f106 x:0 0 0 2024-06-03T01:21:11.349 INFO:tasks.workunit.client.0.smithi032.stdout:2/800: creat d5/da/f107 x:0 0 0 2024-06-03T01:21:11.349 INFO:tasks.workunit.client.0.smithi032.stdout:2/801: chown d5/d38/d46/dbe/l102 25 1 2024-06-03T01:21:11.351 INFO:tasks.workunit.client.0.smithi032.stdout:2/802: unlink d5/d38/d46/d53/d57/dac/f75 0 2024-06-03T01:21:11.352 INFO:tasks.workunit.client.0.smithi032.stdout:2/803: rename d5/da/d1a/d2a/d36/l8c to d5/da/d2e/l108 0 2024-06-03T01:21:11.367 INFO:tasks.workunit.client.0.smithi032.stdout:5/794: dread da/dd/df/d10/dbc/d18/d35/d70/fd8 [0,4194304] 0 2024-06-03T01:21:11.371 INFO:tasks.workunit.client.0.smithi032.stdout:5/795: unlink da/dd/df/d10/dbc/d18/f50 0 2024-06-03T01:21:11.374 INFO:tasks.workunit.client.0.smithi032.stdout:5/796: rename da/dd/df/d10/dbc/l65 to da/dd/df/d10/dbc/d18/db9/lf9 0 2024-06-03T01:21:11.374 INFO:tasks.workunit.client.0.smithi032.stdout:5/797: dread - da/dd/df/d10/dbc/d18/d1a/d30/fd4 zero size 2024-06-03T01:21:11.377 INFO:tasks.workunit.client.0.smithi032.stdout:5/798: truncate da/dd/df/d10/dbc/d18/d35/d70/f86 1041012 0 2024-06-03T01:21:11.377 INFO:tasks.workunit.client.0.smithi032.stdout:3/582: dwrite de/f10 [0,4194304] 0 2024-06-03T01:21:11.377 INFO:tasks.workunit.client.0.smithi032.stdout:5/799: creat da/dd/de6/ffa x:0 0 0 2024-06-03T01:21:11.379 INFO:tasks.workunit.client.0.smithi032.stdout:5/800: creat da/dab/ffb x:0 0 0 2024-06-03T01:21:11.379 INFO:tasks.workunit.client.0.smithi032.stdout:5/801: truncate da/dd/df/d10/dbc/d25/d74/f88 1188270 0 2024-06-03T01:21:11.379 INFO:tasks.workunit.client.0.smithi032.stdout:5/802: creat da/dd/df/d10/dbc/d18/d35/d70/ffc x:0 0 0 2024-06-03T01:21:11.392 INFO:tasks.workunit.client.0.smithi032.stdout:2/804: dwrite d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f99 [0,4194304] 0 2024-06-03T01:21:11.392 INFO:tasks.workunit.client.0.smithi032.stdout:3/583: dread de/f1a [4194304,4194304] 0 2024-06-03T01:21:11.392 INFO:tasks.workunit.client.0.smithi032.stdout:3/584: readlink de/d29/d2e/d31/d3c/d50/d62/lb7 0 2024-06-03T01:21:11.392 INFO:tasks.workunit.client.0.smithi032.stdout:3/585: write de/d29/d2e/d31/d23/d53/d69/d8c/f97 [979827,16338] 0 2024-06-03T01:21:11.392 INFO:tasks.workunit.client.0.smithi032.stdout:3/586: fdatasync de/d29/d2e/d31/d3c/f44 0 2024-06-03T01:21:11.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/587: creat de/d29/d2e/d31/d3c/d50/d66/fc1 x:0 0 0 2024-06-03T01:21:11.397 INFO:tasks.workunit.client.0.smithi032.stdout:3/588: symlink de/d29/d2e/lc2 0 2024-06-03T01:21:11.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/805: dread d5/d38/d46/d53/d57/dac/f54 [0,4194304] 0 2024-06-03T01:21:11.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/806: readlink d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/lab 0 2024-06-03T01:21:11.402 INFO:tasks.workunit.client.0.smithi032.stdout:2/807: fsync d5/da/d1a/d2a/d36/f66 0 2024-06-03T01:21:11.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/589: rename de/d29/d2e/d31/d3c/d50/d5e to de/d29/d2e/d31/d8d/dc3 0 2024-06-03T01:21:11.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/590: chown de/d29/l51 7006 1 2024-06-03T01:21:11.405 INFO:tasks.workunit.client.0.smithi032.stdout:2/808: creat d5/d38/d46/d53/d57/dac/f109 x:0 0 0 2024-06-03T01:21:11.409 INFO:tasks.workunit.client.0.smithi032.stdout:3/591: creat de/d29/d2e/d31/d23/d53/dbf/fc4 x:0 0 0 2024-06-03T01:21:11.411 INFO:tasks.workunit.client.0.smithi032.stdout:2/809: mkdir d5/d38/d46/d53/d57/dac/d10a 0 2024-06-03T01:21:11.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/592: symlink de/d29/d2e/d31/d3c/d50/lc5 0 2024-06-03T01:21:11.413 INFO:tasks.workunit.client.0.smithi032.stdout:3/593: chown de/d29/d2e/d31/d23/f9f 57 1 2024-06-03T01:21:11.413 INFO:tasks.workunit.client.0.smithi032.stdout:3/594: write de/d29/d2e/d31/d3c/d50/d62/fb1 [440560,7809] 0 2024-06-03T01:21:11.413 INFO:tasks.workunit.client.0.smithi032.stdout:3/595: dread - de/d29/d2e/d31/d78/fbe zero size 2024-06-03T01:21:11.413 INFO:tasks.workunit.client.0.smithi032.stdout:3/596: write de/d29/d2e/d31/d23/d53/d69/d8c/f97 [759470,119184] 0 2024-06-03T01:21:11.413 INFO:tasks.workunit.client.0.smithi032.stdout:2/810: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/d104/d10b 0 2024-06-03T01:21:11.415 INFO:tasks.workunit.client.0.smithi032.stdout:2/811: getdents d5 0 2024-06-03T01:21:11.422 INFO:tasks.workunit.client.0.smithi032.stdout:2/812: rename d5/d38/d46/d53/d90 to d5/d38/d46/d5c/d6c/d89/d10c 0 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:2/813: write d5/da/f2f [5783192,69868] 0 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:2/814: dread - d5/d38/d46/d53/d57/dac/de0/fe1 zero size 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:3/597: dwrite de/d29/fb5 [0,4194304] 0 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:3/598: creat de/d29/fc6 x:0 0 0 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:3/599: write de/d29/d2e/d31/d8d/f9d [483065,19552] 0 2024-06-03T01:21:11.427 INFO:tasks.workunit.client.0.smithi032.stdout:3/600: chown de/d63/fb2 374923 1 2024-06-03T01:21:11.430 INFO:tasks.workunit.client.0.smithi032.stdout:2/815: rename d5/da/d2e/c62 to d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/c10d 0 2024-06-03T01:21:11.433 INFO:tasks.workunit.client.0.smithi032.stdout:3/601: truncate de/d29/d2e/f45 223781 0 2024-06-03T01:21:11.434 INFO:tasks.workunit.client.0.smithi032.stdout:2/816: mkdir d5/da/d1a/d2a/da8/d10e 0 2024-06-03T01:21:11.435 INFO:tasks.workunit.client.0.smithi032.stdout:2/817: readlink d5/d38/d46/d5c/d6c/d80/l85 0 2024-06-03T01:21:11.435 INFO:tasks.workunit.client.0.smithi032.stdout:2/818: truncate d5/d38/d46/fe5 954050 0 2024-06-03T01:21:11.436 INFO:tasks.workunit.client.0.smithi032.stdout:3/602: mkdir de/d29/d2e/d31/d3c/d50/d56/dc7 0 2024-06-03T01:21:11.442 INFO:tasks.workunit.client.0.smithi032.stdout:2/819: dread d5/d38/d46/d5c/d6c/d89/d10c/f9b [0,4194304] 0 2024-06-03T01:21:11.444 INFO:tasks.workunit.client.0.smithi032.stdout:3/603: dread de/d29/d2e/d31/d23/f9f [0,4194304] 0 2024-06-03T01:21:11.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/604: truncate de/d29/d2e/d31/d24/f39 2033212 0 2024-06-03T01:21:11.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/605: creat de/d29/d2e/d31/d24/fc8 x:0 0 0 2024-06-03T01:21:11.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/606: dread - de/d29/d2e/d31/d8d/dc3/d7e/fb3 zero size 2024-06-03T01:21:11.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/607: truncate de/d29/d2e/d31/d23/d53/d69/d8c/f9c 502219 0 2024-06-03T01:21:11.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/608: readlink de/d29/d2e/d31/d23/l2c 0 2024-06-03T01:21:11.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/609: mknod de/cc9 0 2024-06-03T01:21:11.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/610: read - de/d29/fc6 zero size 2024-06-03T01:21:11.449 INFO:tasks.workunit.client.0.smithi032.stdout:3/611: chown de/d29/d2e/d31/d23/d53/d69/d8c/f9c 349890 1 2024-06-03T01:21:11.451 INFO:tasks.workunit.client.0.smithi032.stdout:3/612: unlink de/d29/d2e/d31/d3c/d50/d62/fa2 0 2024-06-03T01:21:11.455 INFO:tasks.workunit.client.0.smithi032.stdout:3/613: unlink de/d29/d2e/d31/d23/d53/d69/f84 0 2024-06-03T01:21:11.455 INFO:tasks.workunit.client.0.smithi032.stdout:3/614: dread - de/d63/f7a zero size 2024-06-03T01:21:11.466 INFO:tasks.workunit.client.0.smithi032.stdout:2/820: dwrite d5/da/f2f [0,4194304] 0 2024-06-03T01:21:11.467 INFO:tasks.workunit.client.0.smithi032.stdout:2/821: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/d10f 0 2024-06-03T01:21:11.467 INFO:tasks.workunit.client.0.smithi032.stdout:2/822: write d5/da/f39 [897974,50380] 0 2024-06-03T01:21:11.483 INFO:tasks.workunit.client.0.smithi032.stdout:3/615: dwrite de/d29/f95 [0,4194304] 0 2024-06-03T01:21:11.484 INFO:tasks.workunit.client.0.smithi032.stdout:2/823: dwrite d5/d38/d46/d53/d57/dac/f54 [4194304,4194304] 0 2024-06-03T01:21:11.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/616: mkdir de/d29/d2e/d31/d23/d53/d69/dca 0 2024-06-03T01:21:11.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/617: dread de/d29/d2e/d31/d23/f9f [0,4194304] 0 2024-06-03T01:21:11.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/618: fdatasync de/d29/d2e/d31/d8d/dc3/fa6 0 2024-06-03T01:21:11.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/619: dread - de/d29/d2e/d31/d3c/d50/d56/fae zero size 2024-06-03T01:21:11.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/620: write de/d29/f7f [814613,117171] 0 2024-06-03T01:21:11.496 INFO:tasks.workunit.client.0.smithi032.stdout:2/824: dread d5/da/d1a/fa5 [0,4194304] 0 2024-06-03T01:21:11.499 INFO:tasks.workunit.client.0.smithi032.stdout:3/621: dread de/d29/d2e/d31/f1c [0,4194304] 0 2024-06-03T01:21:11.504 INFO:tasks.workunit.client.0.smithi032.stdout:2/825: dread d5/da/d1a/d2a/d36/d3f/f63 [0,4194304] 0 2024-06-03T01:21:11.504 INFO:tasks.workunit.client.0.smithi032.stdout:2/826: read d5/da/f20 [6628232,21120] 0 2024-06-03T01:21:11.506 INFO:tasks.workunit.client.0.smithi032.stdout:2/827: rename d5/da/d2e/c7d to d5/d38/d46/d53/d57/d6d/dce/d8f/c110 0 2024-06-03T01:21:11.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/828: write d5/d38/d46/d53/da9/ff7 [2829690,77484] 0 2024-06-03T01:21:11.516 INFO:tasks.workunit.client.0.smithi032.stdout:2/829: mknod d5/da/d1a/d2a/da8/c111 0 2024-06-03T01:21:11.524 INFO:tasks.workunit.client.0.smithi032.stdout:2/830: dread d5/da/f20 [4194304,4194304] 0 2024-06-03T01:21:11.524 INFO:tasks.workunit.client.0.smithi032.stdout:3/622: dwrite de/d29/d2e/d31/d3c/d50/d62/fb1 [0,4194304] 0 2024-06-03T01:21:11.525 INFO:tasks.workunit.client.0.smithi032.stdout:3/623: truncate de/d29/d2e/d31/d23/f9f 4991526 0 2024-06-03T01:21:11.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/831: mknod d5/d38/d46/d5c/d6c/d80/c112 0 2024-06-03T01:21:11.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/832: read - d5/da/d1a/d2a/d36/d3f/fd6 zero size 2024-06-03T01:21:11.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/833: chown d5/d38/d46/d5c/d6c/fdf 556260 1 2024-06-03T01:21:11.528 INFO:tasks.workunit.client.0.smithi032.stdout:3/624: creat de/d29/d2e/d31/d23/d53/d69/fcb x:0 0 0 2024-06-03T01:21:11.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/625: read de/d29/d2e/d31/d23/f9f [4360042,32385] 0 2024-06-03T01:21:11.531 INFO:tasks.workunit.client.0.smithi032.stdout:3/626: getdents de/d63 0 2024-06-03T01:21:11.531 INFO:tasks.workunit.client.0.smithi032.stdout:3/627: chown de/d29 126 1 2024-06-03T01:21:11.537 INFO:tasks.workunit.client.0.smithi032.stdout:2/834: truncate d5/d38/d46/d53/da9/ff7 1703039 0 2024-06-03T01:21:11.538 INFO:tasks.workunit.client.0.smithi032.stdout:2/835: fsync d5/da/f2c 0 2024-06-03T01:21:11.544 INFO:tasks.workunit.client.0.smithi032.stdout:3/628: dread de/d29/d2e/d31/d23/f6e [0,4194304] 0 2024-06-03T01:21:11.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/836: dread d5/da/f39 [0,4194304] 0 2024-06-03T01:21:11.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/837: truncate d5/d38/d46/d5c/d6c/d89/f93 1801707 0 2024-06-03T01:21:11.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/838: chown d5/d38/d46/d5c/d6c/d89/d10c/fa3 24 1 2024-06-03T01:21:11.565 INFO:tasks.workunit.client.0.smithi032.stdout:3/629: creat de/d29/d2e/d31/d23/d53/d69/dca/fcc x:0 0 0 2024-06-03T01:21:11.567 INFO:tasks.workunit.client.0.smithi032.stdout:2/839: creat d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f113 x:0 0 0 2024-06-03T01:21:11.570 INFO:tasks.workunit.client.0.smithi032.stdout:3/630: rename de/d29/d2e/d31/d3c/d50/d56/c71 to de/d29/d2e/d31/da1/ccd 0 2024-06-03T01:21:11.577 INFO:tasks.workunit.client.0.smithi032.stdout:2/840: dread d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/fb2 [0,4194304] 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/841: mkdir d5/d38/d46/d114 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/842: write d5/f1c [1281207,125259] 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/843: write d5/da/f107 [786810,32846] 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/844: readlink d5/d38/d46/d53/d57/d6d/dce/d8f/l103 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/845: write d5/da/d2e/f3d [801248,113394] 0 2024-06-03T01:21:11.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/846: write d5/da/f29 [4228639,73605] 0 2024-06-03T01:21:11.580 INFO:tasks.workunit.client.0.smithi032.stdout:2/847: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/df5/d115 0 2024-06-03T01:21:11.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/631: write f5 [3542971,74399] 0 2024-06-03T01:21:11.581 INFO:tasks.workunit.client.0.smithi032.stdout:3/632: dread - de/d29/d2e/d31/d8d/dc3/f89 zero size 2024-06-03T01:21:11.581 INFO:tasks.workunit.client.0.smithi032.stdout:3/633: write de/d29/d2e/d31/d78/f86 [5859116,58883] 0 2024-06-03T01:21:11.581 INFO:tasks.workunit.client.0.smithi032.stdout:2/848: mkdir d5/da/d1a/d2a/da8/d10e/d116 0 2024-06-03T01:21:11.582 INFO:tasks.workunit.client.0.smithi032.stdout:2/849: dread - d5/f72 zero size 2024-06-03T01:21:11.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/850: unlink d5/d38/d46/d5c/d6c/d80/c112 0 2024-06-03T01:21:11.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/851: chown d5/d38/d46/d53/da9/fd9 45553175 1 2024-06-03T01:21:11.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/852: dread - d5/d38/d46/d53/d57/dac/fe9 zero size 2024-06-03T01:21:11.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/853: rename d5 to d5/da/d1a/d2a/d117 22 2024-06-03T01:21:11.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/854: chown d5/d38/d46/d53/d57/d6d/dce/d8f/l103 962939791 1 2024-06-03T01:21:11.585 INFO:tasks.workunit.client.0.smithi032.stdout:2/855: mknod d5/d38/d46/d53/d57/d6d/c118 0 2024-06-03T01:21:11.585 INFO:tasks.workunit.client.0.smithi032.stdout:2/856: truncate d5/da/d1a/f6f 1100321 0 2024-06-03T01:21:11.585 INFO:tasks.workunit.client.0.smithi032.stdout:2/857: symlink d5/d38/l119 0 2024-06-03T01:21:11.586 INFO:tasks.workunit.client.0.smithi032.stdout:2/858: symlink d5/d38/d46/d5c/d6c/l11a 0 2024-06-03T01:21:11.587 INFO:tasks.workunit.client.0.smithi032.stdout:2/859: chown d5/l7 40388 1 2024-06-03T01:21:11.597 INFO:tasks.workunit.client.0.smithi032.stdout:3/634: dwrite f5 [8388608,4194304] 0 2024-06-03T01:21:11.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/635: write de/f15 [3549378,22993] 0 2024-06-03T01:21:11.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/636: fsync de/d29/d2e/d31/d24/f39 0 2024-06-03T01:21:11.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/637: readlink de/d29/d2e/d31/d3c/d50/d62/lb7 0 2024-06-03T01:21:11.607 INFO:tasks.workunit.client.0.smithi032.stdout:3/638: mknod de/d29/d2e/d31/d23/cce 0 2024-06-03T01:21:11.609 INFO:tasks.workunit.client.0.smithi032.stdout:3/639: creat de/d29/d2e/d31/d23/d53/d69/d8c/fcf x:0 0 0 2024-06-03T01:21:11.633 INFO:tasks.workunit.client.0.smithi032.stdout:3/640: dwrite de/d29/d2e/f45 [0,4194304] 0 2024-06-03T01:21:11.636 INFO:tasks.workunit.client.0.smithi032.stdout:3/641: truncate de/d29/d2e/d31/d3c/f44 2811222 0 2024-06-03T01:21:11.652 INFO:tasks.workunit.client.0.smithi032.stdout:3/642: dwrite de/d29/d2e/f4d [4194304,4194304] 0 2024-06-03T01:21:11.669 INFO:tasks.workunit.client.0.smithi032.stdout:3/643: dwrite de/d29/f70 [0,4194304] 0 2024-06-03T01:21:11.769 INFO:tasks.workunit.client.0.smithi032.stdout:8/536: sync 2024-06-03T01:21:11.771 INFO:tasks.workunit.client.0.smithi032.stdout:8/537: symlink dc/d25/d28/d51/d5d/d5e/laa 0 2024-06-03T01:21:11.773 INFO:tasks.workunit.client.0.smithi032.stdout:8/538: link dc/d25/d64/f89 dc/de/d1e/d43/d9b/d76/fab 0 2024-06-03T01:21:11.774 INFO:tasks.workunit.client.0.smithi032.stdout:8/539: mkdir dc/de/d1e/d43/d9b/d76/dac 0 2024-06-03T01:21:11.782 INFO:tasks.workunit.client.0.smithi032.stdout:8/540: write dc/f15 [41344,97271] 0 2024-06-03T01:21:11.784 INFO:tasks.workunit.client.0.smithi032.stdout:8/541: rmdir dc/d25/d28 39 2024-06-03T01:21:11.791 INFO:tasks.workunit.client.0.smithi032.stdout:8/542: dread dc/d25/d28/d3c/f77 [0,4194304] 0 2024-06-03T01:21:11.793 INFO:tasks.workunit.client.0.smithi032.stdout:8/543: mkdir dc/d25/d28/d58/dad 0 2024-06-03T01:21:11.802 INFO:tasks.workunit.client.0.smithi032.stdout:8/544: dread dc/d25/d64/fa4 [0,4194304] 0 2024-06-03T01:21:11.803 INFO:tasks.workunit.client.0.smithi032.stdout:8/545: mkdir dc/de/d1e/d43/d9b/d76/dae 0 2024-06-03T01:21:11.806 INFO:tasks.workunit.client.0.smithi032.stdout:8/546: rename dc/de/f11 to dc/d25/d28/d51/d5d/d5e/faf 0 2024-06-03T01:21:11.806 INFO:tasks.workunit.client.0.smithi032.stdout:8/547: fdatasync dc/d25/d28/d32/f50 0 2024-06-03T01:21:11.807 INFO:tasks.workunit.client.0.smithi032.stdout:8/548: link dc/de/d1e/d43/d9b/f9c dc/de/d1e/d43/d9b/fb0 0 2024-06-03T01:21:11.807 INFO:tasks.workunit.client.0.smithi032.stdout:8/549: write dc/d25/d28/d3c/f77 [5103914,62792] 0 2024-06-03T01:21:11.807 INFO:tasks.workunit.client.0.smithi032.stdout:8/550: readlink dc/de/d1e/l5b 0 2024-06-03T01:21:11.822 INFO:tasks.workunit.client.0.smithi032.stdout:8/551: dwrite dc/d25/d28/d3c/f5a [0,4194304] 0 2024-06-03T01:21:11.825 INFO:tasks.workunit.client.0.smithi032.stdout:8/552: mknod dc/d25/d28/d33/d78/cb1 0 2024-06-03T01:21:11.825 INFO:tasks.workunit.client.0.smithi032.stdout:8/553: getdents dc/de/d1e/d43/d9b/d76/dae 0 2024-06-03T01:21:11.825 INFO:tasks.workunit.client.0.smithi032.stdout:8/554: truncate dc/de/d1e/d43/d9b/f9c 373479 0 2024-06-03T01:21:11.825 INFO:tasks.workunit.client.0.smithi032.stdout:8/555: write dc/de/d1e/d43/d9b/fb0 [918502,79660] 0 2024-06-03T01:21:11.825 INFO:tasks.workunit.client.0.smithi032.stdout:8/556: rmdir dc/de/d1e/d43/d9b/d76 39 2024-06-03T01:21:11.828 INFO:tasks.workunit.client.0.smithi032.stdout:8/557: unlink dc/d25/d28/d51/d5d/d5e/laa 0 2024-06-03T01:21:11.830 INFO:tasks.workunit.client.0.smithi032.stdout:8/558: dread dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:11.830 INFO:tasks.workunit.client.0.smithi032.stdout:8/559: fdatasync dc/d25/d28/d51/d5d/d5e/faf 0 2024-06-03T01:21:11.836 INFO:tasks.workunit.client.0.smithi032.stdout:8/560: getdents dc/de/d1e/d43/d9b 0 2024-06-03T01:21:11.836 INFO:tasks.workunit.client.0.smithi032.stdout:8/561: write dc/de/f95 [1133330,65921] 0 2024-06-03T01:21:11.850 INFO:tasks.workunit.client.0.smithi032.stdout:8/562: dwrite dc/f13 [0,4194304] 0 2024-06-03T01:21:11.857 INFO:tasks.workunit.client.0.smithi032.stdout:8/563: dread dc/d25/d28/d33/f48 [0,4194304] 0 2024-06-03T01:21:11.859 INFO:tasks.workunit.client.0.smithi032.stdout:8/564: creat dc/d25/d28/d33/fb2 x:0 0 0 2024-06-03T01:21:11.861 INFO:tasks.workunit.client.0.smithi032.stdout:8/565: link dc/de/d1e/d43/d9b/f9c dc/de/d1e/d43/d9b/d76/dae/fb3 0 2024-06-03T01:21:11.882 INFO:tasks.workunit.client.0.smithi032.stdout:8/566: dwrite dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:11.882 INFO:tasks.workunit.client.0.smithi032.stdout:8/567: truncate f1 1869693 0 2024-06-03T01:21:11.883 INFO:tasks.workunit.client.0.smithi032.stdout:8/568: link dc/de/l19 dc/de/d1e/d43/lb4 0 2024-06-03T01:21:11.884 INFO:tasks.workunit.client.0.smithi032.stdout:8/569: mkdir dc/d25/d64/db5 0 2024-06-03T01:21:11.901 INFO:tasks.workunit.client.0.smithi032.stdout:8/570: dwrite dc/d25/d28/d51/d5d/d5e/faf [4194304,4194304] 0 2024-06-03T01:21:11.901 INFO:tasks.workunit.client.0.smithi032.stdout:8/571: truncate dc/d25/f55 194222 0 2024-06-03T01:21:11.901 INFO:tasks.workunit.client.0.smithi032.stdout:8/572: stat dc/de/d1e/d43/d9b/fb0 0 2024-06-03T01:21:11.901 INFO:tasks.workunit.client.0.smithi032.stdout:8/573: write f8 [1038432,16758] 0 2024-06-03T01:21:11.903 INFO:tasks.workunit.client.0.smithi032.stdout:8/574: rename dc/de/d1e/d43/d9b/fb0 to dc/de/d1e/d43/d9b/d76/fb6 0 2024-06-03T01:21:11.904 INFO:tasks.workunit.client.0.smithi032.stdout:8/575: creat dc/de/d1e/d43/d9b/d76/fb7 x:0 0 0 2024-06-03T01:21:11.914 INFO:tasks.workunit.client.0.smithi032.stdout:8/576: read dc/d25/d28/d3c/f42 [953558,66785] 0 2024-06-03T01:21:11.915 INFO:tasks.workunit.client.0.smithi032.stdout:8/577: creat dc/d25/d28/fb8 x:0 0 0 2024-06-03T01:21:11.915 INFO:tasks.workunit.client.0.smithi032.stdout:8/578: write dc/d14/d31/f7b [193990,102925] 0 2024-06-03T01:21:11.916 INFO:tasks.workunit.client.0.smithi032.stdout:8/579: mknod dc/d25/d28/d58/dad/cb9 0 2024-06-03T01:21:11.922 INFO:tasks.workunit.client.0.smithi032.stdout:8/580: dread dc/d25/d28/d58/f79 [0,4194304] 0 2024-06-03T01:21:11.922 INFO:tasks.workunit.client.0.smithi032.stdout:8/581: truncate dc/d25/d28/d33/fb2 646837 0 2024-06-03T01:21:11.923 INFO:tasks.workunit.client.0.smithi032.stdout:8/582: symlink dc/d25/d28/d32/lba 0 2024-06-03T01:21:11.923 INFO:tasks.workunit.client.0.smithi032.stdout:8/583: fsync dc/d25/d28/d32/f36 0 2024-06-03T01:21:11.924 INFO:tasks.workunit.client.0.smithi032.stdout:8/584: mkdir dc/d25/d28/d32/dbb 0 2024-06-03T01:21:11.932 INFO:tasks.workunit.client.0.smithi032.stdout:8/585: dread dc/d25/d28/d3c/f3f [0,4194304] 0 2024-06-03T01:21:11.938 INFO:tasks.workunit.client.0.smithi032.stdout:8/586: dread dc/d25/d28/d32/f50 [0,4194304] 0 2024-06-03T01:21:11.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/587: dread - dc/de/d1e/fa0 zero size 2024-06-03T01:21:11.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/588: chown dc/d25/d64 5217 1 2024-06-03T01:21:11.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/589: readlink dc/d14/d31/d3b/l6b 0 2024-06-03T01:21:11.948 INFO:tasks.workunit.client.0.smithi032.stdout:8/590: dread dc/d25/d28/d51/d5d/d5e/faf [4194304,4194304] 0 2024-06-03T01:21:11.949 INFO:tasks.workunit.client.0.smithi032.stdout:8/591: truncate dc/d14/f2d 2699482 0 2024-06-03T01:21:11.954 INFO:tasks.workunit.client.0.smithi032.stdout:8/592: symlink dc/d25/d28/d33/lbc 0 2024-06-03T01:21:11.954 INFO:tasks.workunit.client.0.smithi032.stdout:8/593: chown dc/d25/d28/d58/dad 0 1 2024-06-03T01:21:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:11 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:12.194 INFO:tasks.workunit.client.0.smithi032.stdout:1/557: sync 2024-06-03T01:21:12.194 INFO:tasks.workunit.client.0.smithi032.stdout:1/558: dread - d18/d32/d39/d60/f72 zero size 2024-06-03T01:21:12.194 INFO:tasks.workunit.client.0.smithi032.stdout:1/559: fdatasync d18/d3f/d5f/d94/f96 0 2024-06-03T01:21:12.202 INFO:tasks.workunit.client.0.smithi032.stdout:1/560: dread d18/d3f/d5f/d94/f96 [0,4194304] 0 2024-06-03T01:21:12.212 INFO:tasks.workunit.client.0.smithi032.stdout:1/561: dread d18/d3f/d5f/d6b/d9d/f8b [0,4194304] 0 2024-06-03T01:21:12.212 INFO:tasks.workunit.client.0.smithi032.stdout:1/562: truncate d18/f80 927035 0 2024-06-03T01:21:12.215 INFO:tasks.workunit.client.0.smithi032.stdout:1/563: dread d18/d2d/d5e/f64 [0,4194304] 0 2024-06-03T01:21:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:11 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:12.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/564: dwrite d18/d32/d39/d60/f72 [0,4194304] 0 2024-06-03T01:21:12.234 INFO:tasks.workunit.client.0.smithi032.stdout:1/565: creat d18/d3f/d5f/d6b/d78/f9f x:0 0 0 2024-06-03T01:21:12.241 INFO:tasks.workunit.client.0.smithi032.stdout:1/566: dread d18/d3f/d5f/d6b/d78/d3d/f8c [0,4194304] 0 2024-06-03T01:21:12.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/567: dwrite d18/d3f/d5f/d6b/d78/d3d/f8c [0,4194304] 0 2024-06-03T01:21:12.290 INFO:tasks.workunit.client.0.smithi032.stdout:1/568: link d18/d2d/d5e/c68 d18/d32/d39/d60/ca0 0 2024-06-03T01:21:12.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/569: rename d18/d3f/d5f/d6b/d78/f2e to d18/d3f/d5f/d6b/d78/d3d/d6a/fa1 0 2024-06-03T01:21:12.293 INFO:tasks.workunit.client.0.smithi032.stdout:1/570: rename d18/f34 to d18/d3f/d5f/d98/fa2 0 2024-06-03T01:21:12.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/571: mknod d18/d3f/d5f/d6b/d78/d3d/d47/d6e/ca3 0 2024-06-03T01:21:12.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/572: creat d18/d3f/d5f/d6b/d78/d3d/d6a/fa4 x:0 0 0 2024-06-03T01:21:12.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/573: chown d18/d32/l8d 5 1 2024-06-03T01:21:12.300 INFO:tasks.workunit.client.0.smithi032.stdout:1/574: dread d18/f1a [0,4194304] 0 2024-06-03T01:21:12.307 INFO:tasks.workunit.client.0.smithi032.stdout:1/575: dread d18/d3f/d5f/d6b/d78/d3d/f8c [0,4194304] 0 2024-06-03T01:21:12.308 INFO:tasks.workunit.client.0.smithi032.stdout:1/576: getdents d18/d3f/d5f/d6b/d9d 0 2024-06-03T01:21:12.308 INFO:tasks.workunit.client.0.smithi032.stdout:1/577: creat d18/d3f/d5f/d94/fa5 x:0 0 0 2024-06-03T01:21:12.309 INFO:tasks.workunit.client.0.smithi032.stdout:1/578: mkdir d18/d3f/d5f/da6 0 2024-06-03T01:21:12.309 INFO:tasks.workunit.client.0.smithi032.stdout:1/579: fdatasync d18/d3f/d5f/d6b/d78/d3d/f8c 0 2024-06-03T01:21:12.310 INFO:tasks.workunit.client.0.smithi032.stdout:1/580: mkdir d18/d27/da7 0 2024-06-03T01:21:12.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/581: mkdir d18/d3f/d5f/d7b/da8 0 2024-06-03T01:21:12.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/582: dread - d18/d3f/d5f/d94/f97 zero size 2024-06-03T01:21:12.311 INFO:tasks.workunit.client.0.smithi032.stdout:1/583: readlink d18/l1d 0 2024-06-03T01:21:12.312 INFO:tasks.workunit.client.0.smithi032.stdout:1/584: link d18/d2d/d5e/l76 d18/d3f/d5f/d6b/d9d/la9 0 2024-06-03T01:21:12.317 INFO:tasks.workunit.client.0.smithi032.stdout:1/585: rmdir d18/d3f/d5f/d6b/d78 39 2024-06-03T01:21:12.319 INFO:tasks.workunit.client.0.smithi032.stdout:1/586: mkdir d18/d32/daa 0 2024-06-03T01:21:12.319 INFO:tasks.workunit.client.0.smithi032.stdout:1/587: readlink lf 0 2024-06-03T01:21:12.320 INFO:tasks.workunit.client.0.smithi032.stdout:1/588: mknod d18/d32/cab 0 2024-06-03T01:21:12.321 INFO:tasks.workunit.client.0.smithi032.stdout:1/589: creat d18/d3f/d5f/d6b/d9d/fac x:0 0 0 2024-06-03T01:21:12.322 INFO:tasks.workunit.client.0.smithi032.stdout:1/590: unlink d18/c20 0 2024-06-03T01:21:12.323 INFO:tasks.workunit.client.0.smithi032.stdout:1/591: symlink d18/d3f/d5f/d7b/lad 0 2024-06-03T01:21:12.324 INFO:tasks.workunit.client.0.smithi032.stdout:1/592: creat d18/d3f/d5f/da6/fae x:0 0 0 2024-06-03T01:21:12.324 INFO:tasks.workunit.client.0.smithi032.stdout:1/593: chown d18/d32/d39 10430156 1 2024-06-03T01:21:12.324 INFO:tasks.workunit.client.0.smithi032.stdout:1/594: dread - d18/d3f/d5f/d6b/d78/d3d/d6a/f99 zero size 2024-06-03T01:21:12.325 INFO:tasks.workunit.client.0.smithi032.stdout:1/595: getdents d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91 0 2024-06-03T01:21:12.325 INFO:tasks.workunit.client.0.smithi032.stdout:1/596: truncate d18/d3f/d5f/d94/f97 147423 0 2024-06-03T01:21:12.328 INFO:tasks.workunit.client.0.smithi032.stdout:1/597: dread d18/d32/f52 [0,4194304] 0 2024-06-03T01:21:12.329 INFO:tasks.workunit.client.0.smithi032.stdout:1/598: read d18/f80 [182104,13650] 0 2024-06-03T01:21:12.331 INFO:tasks.workunit.client.0.smithi032.stdout:1/599: link d18/f1a d18/faf 0 2024-06-03T01:21:12.331 INFO:tasks.workunit.client.0.smithi032.stdout:1/600: creat d18/d32/d3c/fb0 x:0 0 0 2024-06-03T01:21:12.336 INFO:tasks.workunit.client.0.smithi032.stdout:1/601: read d18/d3f/d5f/d6b/d9d/f8b [1440368,103314] 0 2024-06-03T01:21:12.336 INFO:tasks.workunit.client.0.smithi032.stdout:1/602: creat d18/d27/d2b/fb1 x:0 0 0 2024-06-03T01:21:12.343 INFO:tasks.workunit.client.0.smithi032.stdout:1/603: write fa [3560453,93251] 0 2024-06-03T01:21:12.343 INFO:tasks.workunit.client.0.smithi032.stdout:1/604: truncate d18/d27/d2b/f7f 5201559 0 2024-06-03T01:21:12.343 INFO:tasks.workunit.client.0.smithi032.stdout:1/605: chown d18/d32/l4e 0 1 2024-06-03T01:21:12.343 INFO:tasks.workunit.client.0.smithi032.stdout:1/606: fsync d18/f8e 0 2024-06-03T01:21:12.361 INFO:tasks.workunit.client.0.smithi032.stdout:1/607: dwrite d18/d32/f4b [0,4194304] 0 2024-06-03T01:21:12.376 INFO:tasks.workunit.client.0.smithi032.stdout:1/608: dwrite d18/d3f/d5f/d98/fa2 [0,4194304] 0 2024-06-03T01:21:12.379 INFO:tasks.workunit.client.0.smithi032.stdout:1/609: creat d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 x:0 0 0 2024-06-03T01:21:12.380 INFO:tasks.workunit.client.0.smithi032.stdout:1/610: mknod d18/d3f/d5f/d6b/d78/d3d/cb3 0 2024-06-03T01:21:12.381 INFO:tasks.workunit.client.0.smithi032.stdout:1/611: creat d18/d3f/d5f/d6b/d78/d3d/d47/fb4 x:0 0 0 2024-06-03T01:21:12.381 INFO:tasks.workunit.client.0.smithi032.stdout:1/612: fdatasync d18/d3f/d5f/d6b/d78/d3d/d47/f77 0 2024-06-03T01:21:12.381 INFO:tasks.workunit.client.0.smithi032.stdout:1/613: mknod d18/d3f/cb5 0 2024-06-03T01:21:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:1/614: write d18/f5d [3889512,69018] 0 2024-06-03T01:21:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:1/615: stat d18/d32/l4e 0 2024-06-03T01:21:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:1/616: rmdir d18/d32 39 2024-06-03T01:21:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:1/617: dread - d18/d3f/d5f/d6b/d78/d3d/d47/f77 zero size 2024-06-03T01:21:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:1/618: fdatasync d18/d32/d39/d60/f72 0 2024-06-03T01:21:12.389 INFO:tasks.workunit.client.0.smithi032.stdout:1/619: dread d18/d3f/f53 [0,4194304] 0 2024-06-03T01:21:12.390 INFO:tasks.workunit.client.0.smithi032.stdout:1/620: mknod d18/d2d/d5e/cb6 0 2024-06-03T01:21:12.390 INFO:tasks.workunit.client.0.smithi032.stdout:1/621: fdatasync d18/d3f/d5f/d98/fa2 0 2024-06-03T01:21:12.392 INFO:tasks.workunit.client.0.smithi032.stdout:1/622: creat d18/d27/fb7 x:0 0 0 2024-06-03T01:21:12.393 INFO:tasks.workunit.client.0.smithi032.stdout:1/623: creat d18/d3f/d5f/d6b/d78/fb8 x:0 0 0 2024-06-03T01:21:12.395 INFO:tasks.workunit.client.0.smithi032.stdout:1/624: link d18/d32/d39/f4a d18/d3f/fb9 0 2024-06-03T01:21:12.395 INFO:tasks.workunit.client.0.smithi032.stdout:1/625: chown d18/d27/d2b/c65 10 1 2024-06-03T01:21:12.407 INFO:tasks.workunit.client.0.smithi032.stdout:6/525: sync 2024-06-03T01:21:12.418 INFO:tasks.workunit.client.0.smithi032.stdout:1/626: dwrite d18/d3f/d5f/d6b/d78/d3d/d6a/fa4 [0,4194304] 0 2024-06-03T01:21:12.423 INFO:tasks.workunit.client.0.smithi032.stdout:1/627: dread d18/d3f/d5f/d6b/d78/d3d/f8c [0,4194304] 0 2024-06-03T01:21:12.423 INFO:tasks.workunit.client.0.smithi032.stdout:6/526: dwrite d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:12.426 INFO:tasks.workunit.client.0.smithi032.stdout:1/628: unlink d18/d3f/d5f/d6b/d78/d3d/d47/f51 0 2024-06-03T01:21:12.429 INFO:tasks.workunit.client.0.smithi032.stdout:6/527: unlink d8/d55/l71 0 2024-06-03T01:21:12.431 INFO:tasks.workunit.client.0.smithi032.stdout:1/629: mknod d18/d3f/d5f/d6b/d78/cba 0 2024-06-03T01:21:12.434 INFO:tasks.workunit.client.0.smithi032.stdout:1/630: rename d18/d3f/d5f/d6b/d78/cba to d18/d32/d39/cbb 0 2024-06-03T01:21:12.434 INFO:tasks.workunit.client.0.smithi032.stdout:1/631: truncate d18/d3f/d5f/d94/fa5 315404 0 2024-06-03T01:21:12.435 INFO:tasks.workunit.client.0.smithi032.stdout:1/632: creat d18/d2d/d5e/fbc x:0 0 0 2024-06-03T01:21:12.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/528: dread d8/d11/d20/d2b/d39/f1f [0,4194304] 0 2024-06-03T01:21:12.438 INFO:tasks.workunit.client.0.smithi032.stdout:1/633: unlink d18/d2d/d5e/fbc 0 2024-06-03T01:21:12.441 INFO:tasks.workunit.client.0.smithi032.stdout:1/634: symlink d18/d2d/lbd 0 2024-06-03T01:21:12.441 INFO:tasks.workunit.client.0.smithi032.stdout:1/635: creat d18/d27/fbe x:0 0 0 2024-06-03T01:21:12.443 INFO:tasks.workunit.client.0.smithi032.stdout:1/636: creat d18/d27/d89/fbf x:0 0 0 2024-06-03T01:21:12.443 INFO:tasks.workunit.client.0.smithi032.stdout:1/637: write d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [986171,12828] 0 2024-06-03T01:21:12.444 INFO:tasks.workunit.client.0.smithi032.stdout:1/638: mkdir d18/dc0 0 2024-06-03T01:21:12.448 INFO:tasks.workunit.client.0.smithi032.stdout:0/705: sync 2024-06-03T01:21:12.448 INFO:tasks.workunit.client.0.smithi032.stdout:4/673: sync 2024-06-03T01:21:12.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/529: dread d8/ff [0,4194304] 0 2024-06-03T01:21:12.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/530: truncate d8/d11/d20/d7f/f78 813594 0 2024-06-03T01:21:12.449 INFO:tasks.workunit.client.0.smithi032.stdout:0/706: rmdir d1/da/d28/d3b/d50 39 2024-06-03T01:21:12.451 INFO:tasks.workunit.client.0.smithi032.stdout:4/674: mknod d4/db/da/d2d/de1/cee 0 2024-06-03T01:21:12.455 INFO:tasks.workunit.client.0.smithi032.stdout:0/707: link d1/da/d28/d3b/d50/l5e d1/da/dd2/le3 0 2024-06-03T01:21:12.455 INFO:tasks.workunit.client.0.smithi032.stdout:0/708: stat d1/da/d28/cab 0 2024-06-03T01:21:12.455 INFO:tasks.workunit.client.0.smithi032.stdout:0/709: read d1/da/d28/d3b/f76 [1187245,6012] 0 2024-06-03T01:21:12.457 INFO:tasks.workunit.client.0.smithi032.stdout:6/531: dread d8/d11/f3e [0,4194304] 0 2024-06-03T01:21:12.461 INFO:tasks.workunit.client.0.smithi032.stdout:6/532: symlink d8/d11/d20/lab 0 2024-06-03T01:21:12.471 INFO:tasks.workunit.client.0.smithi032.stdout:7/652: sync 2024-06-03T01:21:12.471 INFO:tasks.workunit.client.0.smithi032.stdout:5/803: sync 2024-06-03T01:21:12.471 INFO:tasks.workunit.client.0.smithi032.stdout:2/860: sync 2024-06-03T01:21:12.471 INFO:tasks.workunit.client.0.smithi032.stdout:2/861: readlink d5/d38/d46/d53/l7a 0 2024-06-03T01:21:12.471 INFO:tasks.workunit.client.0.smithi032.stdout:8/594: sync 2024-06-03T01:21:12.472 INFO:tasks.workunit.client.0.smithi032.stdout:3/644: sync 2024-06-03T01:21:12.472 INFO:tasks.workunit.client.0.smithi032.stdout:9/702: sync 2024-06-03T01:21:12.473 INFO:tasks.workunit.client.0.smithi032.stdout:0/710: dwrite d1/da/d28/d3b/d5d/fa5 [4194304,4194304] 0 2024-06-03T01:21:12.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/804: symlink da/dd/df/d10/lfd 0 2024-06-03T01:21:12.480 INFO:tasks.workunit.client.0.smithi032.stdout:7/653: link dc/d12/d22/d42/d61/f95 dc/d12/d6a/d86/d88/d8b/db8/fce 0 2024-06-03T01:21:12.480 INFO:tasks.workunit.client.0.smithi032.stdout:3/645: truncate de/d29/d2e/d31/d78/f80 714015 0 2024-06-03T01:21:12.481 INFO:tasks.workunit.client.0.smithi032.stdout:8/595: rename dc/de/d1e/d43/d9b/d76/fb6 to dc/d25/d64/fbd 0 2024-06-03T01:21:12.482 INFO:tasks.workunit.client.0.smithi032.stdout:0/711: write d1/da/d28/d3b/f76 [214935,32752] 0 2024-06-03T01:21:12.482 INFO:tasks.workunit.client.0.smithi032.stdout:0/712: fdatasync d1/da/d28/d3b/d68/d82/f9a 0 2024-06-03T01:21:12.486 INFO:tasks.workunit.client.0.smithi032.stdout:3/646: link de/d29/d2e/d31/d3c/d50/d56/f5c de/d29/d2e/fd0 0 2024-06-03T01:21:12.488 INFO:tasks.workunit.client.0.smithi032.stdout:8/596: mknod dc/d25/d28/d51/d5d/cbe 0 2024-06-03T01:21:12.489 INFO:tasks.workunit.client.0.smithi032.stdout:8/597: write fa [2647733,9255] 0 2024-06-03T01:21:12.489 INFO:tasks.workunit.client.0.smithi032.stdout:8/598: fsync dc/d25/d28/d58/f86 0 2024-06-03T01:21:12.489 INFO:tasks.workunit.client.0.smithi032.stdout:8/599: fdatasync dc/f15 0 2024-06-03T01:21:12.489 INFO:tasks.workunit.client.0.smithi032.stdout:6/533: dwrite d8/d11/d20/f8c [0,4194304] 0 2024-06-03T01:21:12.496 INFO:tasks.workunit.client.0.smithi032.stdout:8/600: read dc/d25/d28/d58/f79 [673471,96295] 0 2024-06-03T01:21:12.498 INFO:tasks.workunit.client.0.smithi032.stdout:2/862: dwrite d5/f6b [0,4194304] 0 2024-06-03T01:21:12.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/601: unlink dc/d14/f22 0 2024-06-03T01:21:12.503 INFO:tasks.workunit.client.0.smithi032.stdout:6/534: dread d8/d11/d20/d36/f38 [0,4194304] 0 2024-06-03T01:21:12.503 INFO:tasks.workunit.client.0.smithi032.stdout:7/654: dwrite dc/f30 [0,4194304] 0 2024-06-03T01:21:12.504 INFO:tasks.workunit.client.0.smithi032.stdout:2/863: creat d5/da/d1a/d2a/da8/d10e/d116/f11b x:0 0 0 2024-06-03T01:21:12.504 INFO:tasks.workunit.client.0.smithi032.stdout:0/713: dwrite d1/f17 [0,4194304] 0 2024-06-03T01:21:12.507 INFO:tasks.workunit.client.0.smithi032.stdout:9/703: rename d0/d11/d1d/da6/d71 to d0/d23/d24/d92/dea 0 2024-06-03T01:21:12.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/805: rmdir da/dd/df/d10 39 2024-06-03T01:21:12.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/806: write da/dd/df/d10/dbc/d18/d35/d70/fd8 [1046700,130201] 0 2024-06-03T01:21:12.514 INFO:tasks.workunit.client.0.smithi032.stdout:6/535: rmdir d8/d11/d20/d3b/d84 0 2024-06-03T01:21:12.520 INFO:tasks.workunit.client.0.smithi032.stdout:7/655: link dc/d12/d2a/d81/fac dc/d12/d6a/d86/d88/d8b/fcf 0 2024-06-03T01:21:12.520 INFO:tasks.workunit.client.0.smithi032.stdout:7/656: getdents dc/d12/d22/d9f 0 2024-06-03T01:21:12.520 INFO:tasks.workunit.client.0.smithi032.stdout:2/864: mknod d5/d38/d46/d5c/d6c/c11c 0 2024-06-03T01:21:12.522 INFO:tasks.workunit.client.0.smithi032.stdout:3/647: rename de/d29/d2e/d31/d23/lb0 to de/d29/d2e/d31/d8d/ld1 0 2024-06-03T01:21:12.525 INFO:tasks.workunit.client.0.smithi032.stdout:9/704: dwrite d0/d11/d1d/f99 [0,4194304] 0 2024-06-03T01:21:12.525 INFO:tasks.workunit.client.0.smithi032.stdout:9/705: dread - d0/da4/fb2 zero size 2024-06-03T01:21:12.525 INFO:tasks.workunit.client.0.smithi032.stdout:9/706: stat d0/ff 0 2024-06-03T01:21:12.525 INFO:tasks.workunit.client.0.smithi032.stdout:8/602: getdents dc/d25/d28/d58/d8d/d94/d59 0 2024-06-03T01:21:12.525 INFO:tasks.workunit.client.0.smithi032.stdout:6/536: dread d8/d11/d20/d2b/f4a [0,4194304] 0 2024-06-03T01:21:12.526 INFO:tasks.workunit.client.0.smithi032.stdout:0/714: dwrite d1/da/d28/f33 [4194304,4194304] 0 2024-06-03T01:21:12.526 INFO:tasks.workunit.client.0.smithi032.stdout:0/715: dread - d1/da/d28/d3b/d5d/f67 zero size 2024-06-03T01:21:12.526 INFO:tasks.workunit.client.0.smithi032.stdout:0/716: chown d1/da/f74 59 1 2024-06-03T01:21:12.528 INFO:tasks.workunit.client.0.smithi032.stdout:5/807: symlink da/dab/lfe 0 2024-06-03T01:21:12.528 INFO:tasks.workunit.client.0.smithi032.stdout:5/808: chown da/dd/de6/d9e 126825 1 2024-06-03T01:21:12.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/657: mknod dc/d12/d22/cd0 0 2024-06-03T01:21:12.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/658: fsync dc/d12/d24/f38 0 2024-06-03T01:21:12.532 INFO:tasks.workunit.client.0.smithi032.stdout:2/865: creat d5/d38/d46/d53/d57/dac/d10a/f11d x:0 0 0 2024-06-03T01:21:12.534 INFO:tasks.workunit.client.0.smithi032.stdout:3/648: rename de/d29/d2e/d31/d3c/l81 to de/d29/d2e/d31/d3c/d50/d62/ld2 0 2024-06-03T01:21:12.534 INFO:tasks.workunit.client.0.smithi032.stdout:3/649: stat de/d29/d2e/f45 0 2024-06-03T01:21:12.544 INFO:tasks.workunit.client.0.smithi032.stdout:8/603: symlink dc/d25/d28/d51/d5d/d5e/lbf 0 2024-06-03T01:21:12.545 INFO:tasks.workunit.client.0.smithi032.stdout:8/604: write f2 [1288763,91826] 0 2024-06-03T01:21:12.545 INFO:tasks.workunit.client.0.smithi032.stdout:8/605: write dc/de/f37 [52191,85988] 0 2024-06-03T01:21:12.545 INFO:tasks.workunit.client.0.smithi032.stdout:8/606: write dc/d25/d28/d3c/f42 [1159688,122826] 0 2024-06-03T01:21:12.545 INFO:tasks.workunit.client.0.smithi032.stdout:8/607: readlink dc/d25/d28/d33/lbc 0 2024-06-03T01:21:12.547 INFO:tasks.workunit.client.0.smithi032.stdout:5/809: creat da/dd/df/d10/dbc/d18/d35/d70/fff x:0 0 0 2024-06-03T01:21:12.548 INFO:tasks.workunit.client.0.smithi032.stdout:2/866: creat d5/d38/d46/d5c/d6c/f11e x:0 0 0 2024-06-03T01:21:12.549 INFO:tasks.workunit.client.0.smithi032.stdout:3/650: truncate de/f11 5137346 0 2024-06-03T01:21:12.549 INFO:tasks.workunit.client.0.smithi032.stdout:3/651: creat de/d29/d2e/d31/d23/d53/dbf/fd3 x:0 0 0 2024-06-03T01:21:12.551 INFO:tasks.workunit.client.0.smithi032.stdout:8/608: truncate dc/d25/d64/fbd 657603 0 2024-06-03T01:21:12.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/810: symlink da/dd/df/d10/dbc/d18/db9/l100 0 2024-06-03T01:21:12.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/811: write da/dd/df/d10/dbc/d18/d35/d70/f84 [1034209,105340] 0 2024-06-03T01:21:12.553 INFO:tasks.workunit.client.0.smithi032.stdout:2/867: creat d5/da/d1a/d2a/f11f x:0 0 0 2024-06-03T01:21:12.555 INFO:tasks.workunit.client.0.smithi032.stdout:3/652: symlink de/d29/d2e/d31/d23/d53/d79/ld4 0 2024-06-03T01:21:12.556 INFO:tasks.workunit.client.0.smithi032.stdout:2/868: mknod d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/c120 0 2024-06-03T01:21:12.556 INFO:tasks.workunit.client.0.smithi032.stdout:2/869: write d5/d38/d46/d53/d57/d6d/dce/d67/f106 [608766,81096] 0 2024-06-03T01:21:12.557 INFO:tasks.workunit.client.0.smithi032.stdout:3/653: symlink de/d29/d2e/d31/d3c/d50/d62/ld5 0 2024-06-03T01:21:12.557 INFO:tasks.workunit.client.0.smithi032.stdout:3/654: fdatasync de/d29/d2e/d31/d23/d53/d69/d8c/fcf 0 2024-06-03T01:21:12.557 INFO:tasks.workunit.client.0.smithi032.stdout:2/870: symlink d5/d38/d46/d53/d57/d6d/dce/d67/d104/l121 0 2024-06-03T01:21:12.558 INFO:tasks.workunit.client.0.smithi032.stdout:2/871: write d5/da/d2e/fb8 [557202,83713] 0 2024-06-03T01:21:12.559 INFO:tasks.workunit.client.0.smithi032.stdout:2/872: mknod d5/c122 0 2024-06-03T01:21:12.560 INFO:tasks.workunit.client.0.smithi032.stdout:3/655: write de/d29/d2e/d31/d24/f25 [1432109,60137] 0 2024-06-03T01:21:12.561 INFO:tasks.workunit.client.0.smithi032.stdout:2/873: link d5/d38/d46/lf0 d5/d38/d46/d53/d57/d6d/dce/d67/d10f/l123 0 2024-06-03T01:21:12.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/717: dwrite d1/da/d10/f5b [4194304,4194304] 0 2024-06-03T01:21:12.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/718: chown d1/da/d28/d3b/d5d/d8d 11 1 2024-06-03T01:21:12.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/719: creat d1/da/d28/fe4 x:0 0 0 2024-06-03T01:21:12.569 INFO:tasks.workunit.client.0.smithi032.stdout:2/874: write d5/da/d1a/d2a/d36/d3f/f63 [1271711,38205] 0 2024-06-03T01:21:12.570 INFO:tasks.workunit.client.0.smithi032.stdout:0/720: dread d1/da/d28/d3b/f76 [0,4194304] 0 2024-06-03T01:21:12.571 INFO:tasks.workunit.client.0.smithi032.stdout:2/875: getdents d5/d38/d46/d53/d57/d6d/dce/d67/df5 0 2024-06-03T01:21:12.571 INFO:tasks.workunit.client.0.smithi032.stdout:2/876: truncate d5/d38/d46/d53/d57/ff3 993034 0 2024-06-03T01:21:12.571 INFO:tasks.workunit.client.0.smithi032.stdout:6/537: dwrite d8/d11/d20/d7f/f91 [0,4194304] 0 2024-06-03T01:21:12.572 INFO:tasks.workunit.client.0.smithi032.stdout:0/721: truncate d1/d46/f98 835226 0 2024-06-03T01:21:12.572 INFO:tasks.workunit.client.0.smithi032.stdout:0/722: stat d1/da/d10/d80/d9f/dac/lbb 0 2024-06-03T01:21:12.572 INFO:tasks.workunit.client.0.smithi032.stdout:0/723: dread - d1/da/db2/fc5 zero size 2024-06-03T01:21:12.573 INFO:tasks.workunit.client.0.smithi032.stdout:7/659: dwrite dc/dbe/fc7 [0,4194304] 0 2024-06-03T01:21:12.574 INFO:tasks.workunit.client.0.smithi032.stdout:5/812: dwrite da/dd/df/d10/f14 [0,4194304] 0 2024-06-03T01:21:12.574 INFO:tasks.workunit.client.0.smithi032.stdout:2/877: rename d5/da/d1a/d2a/d36/d3f/f78 to d5/da/d2e/dec/f124 0 2024-06-03T01:21:12.580 INFO:tasks.workunit.client.0.smithi032.stdout:0/724: truncate d1/da/d10/d29/fbc 1454221 0 2024-06-03T01:21:12.583 INFO:tasks.workunit.client.0.smithi032.stdout:7/660: write dc/d12/d22/d42/f51 [719328,55868] 0 2024-06-03T01:21:12.592 INFO:tasks.workunit.client.0.smithi032.stdout:5/813: creat da/f101 x:0 0 0 2024-06-03T01:21:12.592 INFO:tasks.workunit.client.0.smithi032.stdout:5/814: chown da/dd/df/d10/dbc/d18/d1a/d30/f94 3323689 1 2024-06-03T01:21:12.593 INFO:tasks.workunit.client.0.smithi032.stdout:5/815: write da/dd/df/d10/f39 [4639727,110801] 0 2024-06-03T01:21:12.593 INFO:tasks.workunit.client.0.smithi032.stdout:5/816: write da/dd/df/d10/dbc/d18/d35/d70/ffc [413535,78704] 0 2024-06-03T01:21:12.593 INFO:tasks.workunit.client.0.smithi032.stdout:9/707: write d0/d23/d25/f52 [2284621,99223] 0 2024-06-03T01:21:12.594 INFO:tasks.workunit.client.0.smithi032.stdout:9/708: chown d0/d23/d25/f4c 75 1 2024-06-03T01:21:12.597 INFO:tasks.workunit.client.0.smithi032.stdout:2/878: mkdir d5/d38/d46/d53/d57/d6d/dce/d67/df5/d115/d125 0 2024-06-03T01:21:12.599 INFO:tasks.workunit.client.0.smithi032.stdout:6/538: rmdir d8/d55 39 2024-06-03T01:21:12.603 INFO:tasks.workunit.client.0.smithi032.stdout:2/879: rename d5/f58 to d5/d38/d46/d53/d57/d6d/dce/d67/d104/d10b/f126 0 2024-06-03T01:21:12.605 INFO:tasks.workunit.client.0.smithi032.stdout:5/817: dread da/dd/df/d10/dbc/d25/fef [0,4194304] 0 2024-06-03T01:21:12.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/609: dread dc/de/d1e/d43/d9b/f9c [0,4194304] 0 2024-06-03T01:21:12.605 INFO:tasks.workunit.client.0.smithi032.stdout:0/725: dwrite d1/da/d28/d3b/d68/fbf [0,4194304] 0 2024-06-03T01:21:12.605 INFO:tasks.workunit.client.0.smithi032.stdout:7/661: dwrite dc/d12/d6a/fa7 [0,4194304] 0 2024-06-03T01:21:12.606 INFO:tasks.workunit.client.0.smithi032.stdout:6/539: write d8/d50/f62 [389710,118697] 0 2024-06-03T01:21:12.606 INFO:tasks.workunit.client.0.smithi032.stdout:6/540: chown d8/d11/d20/d36/c47 12792007 1 2024-06-03T01:21:12.607 INFO:tasks.workunit.client.0.smithi032.stdout:2/880: creat d5/d38/f127 x:0 0 0 2024-06-03T01:21:12.607 INFO:tasks.workunit.client.0.smithi032.stdout:2/881: creat d5/d38/d46/d53/d57/dac/f128 x:0 0 0 2024-06-03T01:21:12.610 INFO:tasks.workunit.client.0.smithi032.stdout:5/818: rmdir da/dd/de6/d9e 39 2024-06-03T01:21:12.611 INFO:tasks.workunit.client.0.smithi032.stdout:8/610: write dc/de/d1e/d43/d9b/d76/fab [3333524,71442] 0 2024-06-03T01:21:12.611 INFO:tasks.workunit.client.0.smithi032.stdout:8/611: dread - dc/de/d1e/fa0 zero size 2024-06-03T01:21:12.611 INFO:tasks.workunit.client.0.smithi032.stdout:8/612: creat dc/de/d1e/d43/d9b/d76/fc0 x:0 0 0 2024-06-03T01:21:12.612 INFO:tasks.workunit.client.0.smithi032.stdout:2/882: dread d5/f1c [0,4194304] 0 2024-06-03T01:21:12.618 INFO:tasks.workunit.client.0.smithi032.stdout:7/662: mknod dc/d12/d22/d42/cd1 0 2024-06-03T01:21:12.618 INFO:tasks.workunit.client.0.smithi032.stdout:7/663: read - dc/d12/d2a/d2f/d63/d69/f5f zero size 2024-06-03T01:21:12.619 INFO:tasks.workunit.client.0.smithi032.stdout:6/541: unlink d8/d11/d20/d36/c90 0 2024-06-03T01:21:12.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/819: mknod da/dd/df/d10/dbc/d18/d1a/d30/c102 0 2024-06-03T01:21:12.624 INFO:tasks.workunit.client.0.smithi032.stdout:5/820: dread da/dd/df/d10/dbc/d25/d74/f8e [0,4194304] 0 2024-06-03T01:21:12.625 INFO:tasks.workunit.client.0.smithi032.stdout:0/726: dwrite d1/da/d10/f7a [0,4194304] 0 2024-06-03T01:21:12.625 INFO:tasks.workunit.client.0.smithi032.stdout:9/709: dwrite d0/d11/f22 [4194304,4194304] 0 2024-06-03T01:21:12.626 INFO:tasks.workunit.client.0.smithi032.stdout:8/613: unlink dc/d14/d31/d3b/l40 0 2024-06-03T01:21:12.628 INFO:tasks.workunit.client.0.smithi032.stdout:2/883: rmdir d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83 39 2024-06-03T01:21:12.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/664: creat dc/d12/d2a/d2f/fd2 x:0 0 0 2024-06-03T01:21:12.632 INFO:tasks.workunit.client.0.smithi032.stdout:6/542: truncate d8/d11/d20/d36/f38 2970330 0 2024-06-03T01:21:12.632 INFO:tasks.workunit.client.0.smithi032.stdout:6/543: creat d8/d50/fac x:0 0 0 2024-06-03T01:21:12.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/821: creat da/dd/df/f103 x:0 0 0 2024-06-03T01:21:12.635 INFO:tasks.workunit.client.0.smithi032.stdout:0/727: stat d1/da/d10/c25 0 2024-06-03T01:21:12.635 INFO:tasks.workunit.client.0.smithi032.stdout:0/728: chown d1/da/d10/l2e 501 1 2024-06-03T01:21:12.638 INFO:tasks.workunit.client.0.smithi032.stdout:9/710: truncate d0/f15 2285711 0 2024-06-03T01:21:12.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/822: mknod da/dd/df/d10/dbc/d18/c104 0 2024-06-03T01:21:12.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/823: chown da/dd/df/d10/dbc/d18/d35/d70/fb3 852 1 2024-06-03T01:21:12.645 INFO:tasks.workunit.client.0.smithi032.stdout:5/824: stat da/c59 0 2024-06-03T01:21:12.648 INFO:tasks.workunit.client.0.smithi032.stdout:8/614: getdents dc/d25/d28/d58 0 2024-06-03T01:21:12.652 INFO:tasks.workunit.client.0.smithi032.stdout:2/884: link d5/da/d1a/c55 d5/d38/d46/d53/da9/c129 0 2024-06-03T01:21:12.652 INFO:tasks.workunit.client.0.smithi032.stdout:2/885: truncate d5/d38/d46/d5c/d6c/d89/d10c/fa3 899562 0 2024-06-03T01:21:12.654 INFO:tasks.workunit.client.0.smithi032.stdout:5/825: symlink da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/l105 0 2024-06-03T01:21:12.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/826: dread da/dd/df/d10/dbc/d25/d74/f8e [0,4194304] 0 2024-06-03T01:21:12.657 INFO:tasks.workunit.client.0.smithi032.stdout:5/827: write da/dd/df/d10/f14 [7002115,22493] 0 2024-06-03T01:21:12.657 INFO:tasks.workunit.client.0.smithi032.stdout:0/729: unlink d1/c5 0 2024-06-03T01:21:12.657 INFO:tasks.workunit.client.0.smithi032.stdout:0/730: chown d1/da/d10/d53/c4e 852 1 2024-06-03T01:21:12.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/886: creat d5/d38/d46/d5c/f12a x:0 0 0 2024-06-03T01:21:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:5/828: creat da/dd/df/d10/dbc/f106 x:0 0 0 2024-06-03T01:21:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:5/829: stat da/dd/df/d10/dbc/d18/d35/d44/c4a 0 2024-06-03T01:21:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:6/544: dwrite d8/d11/d20/f9f [0,4194304] 0 2024-06-03T01:21:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:5/830: creat da/dab/f107 x:0 0 0 2024-06-03T01:21:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:8/615: dread dc/d25/f88 [0,4194304] 0 2024-06-03T01:21:12.662 INFO:tasks.workunit.client.0.smithi032.stdout:8/616: fdatasync dc/d14/f2d 0 2024-06-03T01:21:12.662 INFO:tasks.workunit.client.0.smithi032.stdout:7/665: dwrite dc/d12/d22/d42/f50 [0,4194304] 0 2024-06-03T01:21:12.662 INFO:tasks.workunit.client.0.smithi032.stdout:9/711: dwrite d0/d11/d1d/d38/d4f/db0/f39 [4194304,4194304] 0 2024-06-03T01:21:12.662 INFO:tasks.workunit.client.0.smithi032.stdout:9/712: dread - d0/f9a zero size 2024-06-03T01:21:12.664 INFO:tasks.workunit.client.0.smithi032.stdout:2/887: creat d5/f12b x:0 0 0 2024-06-03T01:21:12.664 INFO:tasks.workunit.client.0.smithi032.stdout:2/888: write d5/d38/d46/d53/d57/dac/de0/fe1 [437429,118281] 0 2024-06-03T01:21:12.664 INFO:tasks.workunit.client.0.smithi032.stdout:2/889: truncate d5/da/d1a/d2a/d36/f66 1573494 0 2024-06-03T01:21:12.665 INFO:tasks.workunit.client.0.smithi032.stdout:9/713: dread d0/d23/d25/f8e [0,4194304] 0 2024-06-03T01:21:12.673 INFO:tasks.workunit.client.0.smithi032.stdout:6/545: read d8/d11/d20/d2b/f3c [746849,106136] 0 2024-06-03T01:21:12.673 INFO:tasks.workunit.client.0.smithi032.stdout:6/546: chown d8/ff 0 1 2024-06-03T01:21:12.676 INFO:tasks.workunit.client.0.smithi032.stdout:7/666: mkdir dc/d12/d24/dd3 0 2024-06-03T01:21:12.676 INFO:tasks.workunit.client.0.smithi032.stdout:7/667: chown dc/d12/d22/d42/cd1 102360 1 2024-06-03T01:21:12.677 INFO:tasks.workunit.client.0.smithi032.stdout:0/731: getdents d1/da/d28/d3b/d5d/da9 0 2024-06-03T01:21:12.677 INFO:tasks.workunit.client.0.smithi032.stdout:0/732: stat d1/da/d10/f6f 0 2024-06-03T01:21:12.677 INFO:tasks.workunit.client.0.smithi032.stdout:0/733: fsync d1/da/d28/d3b/d5d/fa5 0 2024-06-03T01:21:12.679 INFO:tasks.workunit.client.0.smithi032.stdout:9/714: rename d0/d23/d25/dd6/fe6 to d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/feb 0 2024-06-03T01:21:12.680 INFO:tasks.workunit.client.0.smithi032.stdout:2/890: dwrite d5/fe [0,4194304] 0 2024-06-03T01:21:12.684 INFO:tasks.workunit.client.0.smithi032.stdout:8/617: link f1 dc/d14/fc1 0 2024-06-03T01:21:12.684 INFO:tasks.workunit.client.0.smithi032.stdout:8/618: fsync dc/d25/d28/d58/f86 0 2024-06-03T01:21:12.684 INFO:tasks.workunit.client.0.smithi032.stdout:8/619: dread - dc/d25/d28/d58/f86 zero size 2024-06-03T01:21:12.685 INFO:tasks.workunit.client.0.smithi032.stdout:2/891: dread d5/da/d1a/fa5 [0,4194304] 0 2024-06-03T01:21:12.685 INFO:tasks.workunit.client.0.smithi032.stdout:6/547: getdents d8 0 2024-06-03T01:21:12.687 INFO:tasks.workunit.client.0.smithi032.stdout:8/620: dread dc/de/f95 [0,4194304] 0 2024-06-03T01:21:12.688 INFO:tasks.workunit.client.0.smithi032.stdout:7/668: mkdir dc/d12/d24/dd4 0 2024-06-03T01:21:12.690 INFO:tasks.workunit.client.0.smithi032.stdout:0/734: mkdir d1/da/d10/d80/de5 0 2024-06-03T01:21:12.691 INFO:tasks.workunit.client.0.smithi032.stdout:5/831: rmdir da/dd/df/d10/dbc 39 2024-06-03T01:21:12.693 INFO:tasks.workunit.client.0.smithi032.stdout:9/715: mknod d0/d11/d1d/d38/d4f/db0/ddb/cec 0 2024-06-03T01:21:12.693 INFO:tasks.workunit.client.0.smithi032.stdout:9/716: creat d0/d11/d1d/d38/d4f/db0/ddb/fed x:0 0 0 2024-06-03T01:21:12.696 INFO:tasks.workunit.client.0.smithi032.stdout:2/892: rename d5/d38/d46/d53/d57/ff3 to d5/d38/d46/d5c/d6c/d80/f12c 0 2024-06-03T01:21:12.697 INFO:tasks.workunit.client.0.smithi032.stdout:6/548: creat d8/d11/d20/d3b/fad x:0 0 0 2024-06-03T01:21:12.699 INFO:tasks.workunit.client.0.smithi032.stdout:8/621: truncate dc/d25/d64/fa4 1002055 0 2024-06-03T01:21:12.701 INFO:tasks.workunit.client.0.smithi032.stdout:7/669: truncate dc/d12/d1b/f2c 1429020 0 2024-06-03T01:21:12.701 INFO:tasks.workunit.client.0.smithi032.stdout:7/670: chown dc/d12/d1b/fa5 567383 1 2024-06-03T01:21:12.701 INFO:tasks.workunit.client.0.smithi032.stdout:7/671: rename dc/d12/d6a to dc/d12/d6a/d86/d89/dd5 22 2024-06-03T01:21:12.703 INFO:tasks.workunit.client.0.smithi032.stdout:0/735: creat d1/da/d28/d3b/d5d/fe6 x:0 0 0 2024-06-03T01:21:12.703 INFO:tasks.workunit.client.0.smithi032.stdout:0/736: chown d1/da/d28/d3b/d5d/d8d/dbe/dc4 599225 1 2024-06-03T01:21:12.705 INFO:tasks.workunit.client.0.smithi032.stdout:5/832: mknod da/dd/df/d10/dbc/d18/c108 0 2024-06-03T01:21:12.710 INFO:tasks.workunit.client.0.smithi032.stdout:9/717: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/dee 0 2024-06-03T01:21:12.710 INFO:tasks.workunit.client.0.smithi032.stdout:9/718: truncate d0/d11/d1d/d38/d4f/fa1 2449454 0 2024-06-03T01:21:12.710 INFO:tasks.workunit.client.0.smithi032.stdout:7/672: write dc/d12/d6a/fa7 [93100,88250] 0 2024-06-03T01:21:12.711 INFO:tasks.workunit.client.0.smithi032.stdout:2/893: rmdir d5/da/d1a/d2a/d36/d3f 39 2024-06-03T01:21:12.712 INFO:tasks.workunit.client.0.smithi032.stdout:6/549: mkdir d8/d11/d20/d36/dae 0 2024-06-03T01:21:12.713 INFO:tasks.workunit.client.0.smithi032.stdout:8/622: dwrite dc/d25/d28/d51/f71 [0,4194304] 0 2024-06-03T01:21:12.714 INFO:tasks.workunit.client.0.smithi032.stdout:0/737: unlink d1/c75 0 2024-06-03T01:21:12.716 INFO:tasks.workunit.client.0.smithi032.stdout:9/719: mknod d0/d23/d25/dd6/cef 0 2024-06-03T01:21:12.716 INFO:tasks.workunit.client.0.smithi032.stdout:9/720: fdatasync d0/d11/d1d/d38/d4f/db0/fa0 0 2024-06-03T01:21:12.722 INFO:tasks.workunit.client.0.smithi032.stdout:2/894: mknod d5/d38/d46/d53/d57/d6d/dce/d67/df5/d115/d125/c12d 0 2024-06-03T01:21:12.725 INFO:tasks.workunit.client.0.smithi032.stdout:6/550: link d8/d55/l5c d8/d4c/laf 0 2024-06-03T01:21:12.725 INFO:tasks.workunit.client.0.smithi032.stdout:5/833: dread da/dd/df/d10/f42 [0,4194304] 0 2024-06-03T01:21:12.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/738: dread d1/da/d10/f4f [0,4194304] 0 2024-06-03T01:21:12.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/739: dread - d1/da/d10/d80/d9f/dac/fbd zero size 2024-06-03T01:21:12.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/740: fsync d1/da/db2/fdc 0 2024-06-03T01:21:12.728 INFO:tasks.workunit.client.0.smithi032.stdout:2/895: dread d5/da/d1a/f4b [0,4194304] 0 2024-06-03T01:21:12.729 INFO:tasks.workunit.client.0.smithi032.stdout:8/623: dwrite f2 [0,4194304] 0 2024-06-03T01:21:12.731 INFO:tasks.workunit.client.0.smithi032.stdout:9/721: truncate d0/f5 1900726 0 2024-06-03T01:21:12.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/624: dread dc/d25/d28/d33/fb2 [0,4194304] 0 2024-06-03T01:21:12.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/625: fdatasync dc/d25/d28/d3c/f42 0 2024-06-03T01:21:12.733 INFO:tasks.workunit.client.0.smithi032.stdout:7/673: rmdir dc/d12/d2a/d2f 39 2024-06-03T01:21:12.735 INFO:tasks.workunit.client.0.smithi032.stdout:5/834: mknod da/dd/df/d10/dbc/d25/c109 0 2024-06-03T01:21:12.735 INFO:tasks.workunit.client.0.smithi032.stdout:5/835: write da/f3e [907703,21706] 0 2024-06-03T01:21:12.736 INFO:tasks.workunit.client.0.smithi032.stdout:0/741: mknod d1/da/d28/d3b/d5d/da9/ce7 0 2024-06-03T01:21:12.736 INFO:tasks.workunit.client.0.smithi032.stdout:0/742: chown d1/da/d28/d3b/d50/f58 133 1 2024-06-03T01:21:12.737 INFO:tasks.workunit.client.0.smithi032.stdout:2/896: mknod d5/da/d1a/d2a/d36/dc3/c12e 0 2024-06-03T01:21:12.738 INFO:tasks.workunit.client.0.smithi032.stdout:2/897: truncate d5/d38/d46/d5c/d6c/d89/d10c/fa3 1501852 0 2024-06-03T01:21:12.738 INFO:tasks.workunit.client.0.smithi032.stdout:2/898: readlink d5/d38/d46/d5c/d6c/l11a 0 2024-06-03T01:21:12.741 INFO:tasks.workunit.client.0.smithi032.stdout:9/722: mkdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/df0 0 2024-06-03T01:21:12.741 INFO:tasks.workunit.client.0.smithi032.stdout:9/723: chown d0/fe3 6391904 1 2024-06-03T01:21:12.743 INFO:tasks.workunit.client.0.smithi032.stdout:7/674: creat dc/d12/d2a/fd6 x:0 0 0 2024-06-03T01:21:12.745 INFO:tasks.workunit.client.0.smithi032.stdout:8/626: write dc/d25/d28/d32/f7d [2192156,32929] 0 2024-06-03T01:21:12.747 INFO:tasks.workunit.client.0.smithi032.stdout:5/836: creat da/dd/df/d10/f10a x:0 0 0 2024-06-03T01:21:12.751 INFO:tasks.workunit.client.0.smithi032.stdout:0/743: creat d1/da/d28/d3b/d5d/d8d/dbe/dc4/fe8 x:0 0 0 2024-06-03T01:21:12.751 INFO:tasks.workunit.client.0.smithi032.stdout:0/744: creat d1/da/db2/fe9 x:0 0 0 2024-06-03T01:21:12.751 INFO:tasks.workunit.client.0.smithi032.stdout:0/745: chown d1/faf 30424 1 2024-06-03T01:21:12.752 INFO:tasks.workunit.client.0.smithi032.stdout:6/551: dwrite d8/d11/d20/d2b/d39/d19/f40 [0,4194304] 0 2024-06-03T01:21:12.752 INFO:tasks.workunit.client.0.smithi032.stdout:6/552: write d8/d11/d20/d3b/fad [794526,29889] 0 2024-06-03T01:21:12.752 INFO:tasks.workunit.client.0.smithi032.stdout:6/553: truncate d8/d11/f17 5091853 0 2024-06-03T01:21:12.756 INFO:tasks.workunit.client.0.smithi032.stdout:9/724: rename d0/d23/d24/d92 to d0/d23/d25/de1/df1 0 2024-06-03T01:21:12.758 INFO:tasks.workunit.client.0.smithi032.stdout:7/675: mknod dc/cd7 0 2024-06-03T01:21:12.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/746: mknod d1/d46/cea 0 2024-06-03T01:21:12.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/747: dread - d1/da/d28/d3b/d68/db5/fd7 zero size 2024-06-03T01:21:12.766 INFO:tasks.workunit.client.0.smithi032.stdout:0/748: truncate d1/d46/dc3/fda 832421 0 2024-06-03T01:21:12.779 INFO:tasks.workunit.client.0.smithi032.stdout:2/899: link d5/d38/db4/lcc d5/d38/d46/d53/da9/l12f 0 2024-06-03T01:21:12.779 INFO:tasks.workunit.client.0.smithi032.stdout:2/900: creat d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f130 x:0 0 0 2024-06-03T01:21:12.780 INFO:tasks.workunit.client.0.smithi032.stdout:2/901: write d5/d38/d46/d5c/f8b [1048489,118397] 0 2024-06-03T01:21:12.780 INFO:tasks.workunit.client.0.smithi032.stdout:7/676: stat dc/d12/d24/d7b/ca3 0 2024-06-03T01:21:12.780 INFO:tasks.workunit.client.0.smithi032.stdout:0/749: mknod d1/da/d28/d3b/d5d/da9/daa/ceb 0 2024-06-03T01:21:12.781 INFO:tasks.workunit.client.0.smithi032.stdout:0/750: dread - d1/da/d28/d3b/d5d/f67 zero size 2024-06-03T01:21:12.781 INFO:tasks.workunit.client.0.smithi032.stdout:0/751: chown d1/da/d10/d29/c49 3049 1 2024-06-03T01:21:12.782 INFO:tasks.workunit.client.0.smithi032.stdout:2/902: truncate d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/fb2 1839997 0 2024-06-03T01:21:12.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/677: symlink dc/d12/d2a/d2f/ld8 0 2024-06-03T01:21:12.784 INFO:tasks.workunit.client.0.smithi032.stdout:0/752: mkdir d1/da/d28/d3b/d50/dec 0 2024-06-03T01:21:12.785 INFO:tasks.workunit.client.0.smithi032.stdout:2/903: rename d5/da/d1a/d2a/fcb to d5/d38/d46/d53/d57/dac/d10a/f131 0 2024-06-03T01:21:12.785 INFO:tasks.workunit.client.0.smithi032.stdout:2/904: creat d5/d38/db5/f132 x:0 0 0 2024-06-03T01:21:12.786 INFO:tasks.workunit.client.0.smithi032.stdout:7/678: dread dc/d12/d6a/d86/d88/d8b/db8/fce [0,4194304] 0 2024-06-03T01:21:12.786 INFO:tasks.workunit.client.0.smithi032.stdout:7/679: creat dc/d12/d6a/d86/d88/d8b/db8/fd9 x:0 0 0 2024-06-03T01:21:12.786 INFO:tasks.workunit.client.0.smithi032.stdout:0/753: truncate d1/f43 3939010 0 2024-06-03T01:21:12.787 INFO:tasks.workunit.client.0.smithi032.stdout:2/905: getdents d5/d38/d46/d53/d57/d6d 0 2024-06-03T01:21:12.787 INFO:tasks.workunit.client.0.smithi032.stdout:2/906: read - d5/d38/d46/d5c/d6c/d89/fdd zero size 2024-06-03T01:21:12.787 INFO:tasks.workunit.client.0.smithi032.stdout:2/907: chown d5/d38/d46/d53/d57/d6d/dce/d67/df8 0 1 2024-06-03T01:21:12.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/627: dwrite dc/de/d1e/f61 [0,4194304] 0 2024-06-03T01:21:12.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/680: truncate dc/d12/f25 500001 0 2024-06-03T01:21:12.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/681: dread - dc/d12/d2a/d2f/d9a/fc1 zero size 2024-06-03T01:21:12.790 INFO:tasks.workunit.client.0.smithi032.stdout:2/908: truncate d5/ff 433070 0 2024-06-03T01:21:12.791 INFO:tasks.workunit.client.0.smithi032.stdout:8/628: link dc/d25/d28/d58/d8d/d94/f6d dc/d25/d28/d51/fc2 0 2024-06-03T01:21:12.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/909: fsync d5/d38/f97 0 2024-06-03T01:21:12.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/910: readlink d5/d38/d46/d53/d57/d6d/dce/d67/lb3 0 2024-06-03T01:21:12.794 INFO:tasks.workunit.client.0.smithi032.stdout:5/837: dwrite da/dd/de6/f91 [0,4194304] 0 2024-06-03T01:21:12.794 INFO:tasks.workunit.client.0.smithi032.stdout:8/629: dread dc/d14/fc1 [0,4194304] 0 2024-06-03T01:21:12.794 INFO:tasks.workunit.client.0.smithi032.stdout:8/630: chown dc/d25/d28/d51/d5d/l87 131568873 1 2024-06-03T01:21:12.794 INFO:tasks.workunit.client.0.smithi032.stdout:6/554: dwrite d8/ff [0,4194304] 0 2024-06-03T01:21:12.794 INFO:tasks.workunit.client.0.smithi032.stdout:6/555: write d8/d11/d20/d36/f38 [2035150,31074] 0 2024-06-03T01:21:12.795 INFO:tasks.workunit.client.0.smithi032.stdout:9/725: dwrite d0/da4/faf [0,4194304] 0 2024-06-03T01:21:12.795 INFO:tasks.workunit.client.0.smithi032.stdout:9/726: fdatasync d0/d11/d1d/d38/f3e 0 2024-06-03T01:21:12.796 INFO:tasks.workunit.client.0.smithi032.stdout:9/727: write d0/d11/d1d/f49 [3724814,79784] 0 2024-06-03T01:21:12.796 INFO:tasks.workunit.client.0.smithi032.stdout:9/728: getdents d0/d23 0 2024-06-03T01:21:12.796 INFO:tasks.workunit.client.0.smithi032.stdout:2/911: mknod d5/d38/d46/d5c/d6c/d89/d10c/c133 0 2024-06-03T01:21:12.806 INFO:tasks.workunit.client.0.smithi032.stdout:2/912: read d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f99 [3422478,65146] 0 2024-06-03T01:21:12.815 INFO:tasks.workunit.client.0.smithi032.stdout:0/754: dwrite d1/da/d10/d29/fbc [0,4194304] 0 2024-06-03T01:21:12.816 INFO:tasks.workunit.client.0.smithi032.stdout:7/682: dwrite dc/d12/d1b/f21 [0,4194304] 0 2024-06-03T01:21:12.816 INFO:tasks.workunit.client.0.smithi032.stdout:7/683: fdatasync dc/d12/d24/d7b/d9d/fb7 0 2024-06-03T01:21:12.821 INFO:tasks.workunit.client.0.smithi032.stdout:8/631: rename dc/d25/d28/d32/c46 to dc/d25/d28/d58/d8d/cc3 0 2024-06-03T01:21:12.821 INFO:tasks.workunit.client.0.smithi032.stdout:8/632: fdatasync dc/d25/d28/d33/d78/f82 0 2024-06-03T01:21:12.826 INFO:tasks.workunit.client.0.smithi032.stdout:9/729: link d0/d11/d79/c7b d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/cf2 0 2024-06-03T01:21:12.826 INFO:tasks.workunit.client.0.smithi032.stdout:9/730: read d0/fe [3048294,66278] 0 2024-06-03T01:21:12.827 INFO:tasks.workunit.client.0.smithi032.stdout:0/755: read d1/da/d10/d29/f73 [7934953,34250] 0 2024-06-03T01:21:12.827 INFO:tasks.workunit.client.0.smithi032.stdout:0/756: chown d1/da/d28/d3b/d5d/da9/daa 230896559 1 2024-06-03T01:21:12.827 INFO:tasks.workunit.client.0.smithi032.stdout:0/757: chown d1/d46/c62 828537604 1 2024-06-03T01:21:12.829 INFO:tasks.workunit.client.0.smithi032.stdout:7/684: write dc/d12/d6a/d86/d88/d8b/db8/fba [1276201,48281] 0 2024-06-03T01:21:12.829 INFO:tasks.workunit.client.0.smithi032.stdout:2/913: mkdir d5/da/d1a/d2a/da8/db6/d134 0 2024-06-03T01:21:12.829 INFO:tasks.workunit.client.0.smithi032.stdout:2/914: write d5/f72 [866162,124835] 0 2024-06-03T01:21:12.838 INFO:tasks.workunit.client.0.smithi032.stdout:0/758: read d1/da/d28/d3b/d50/f58 [716709,62986] 0 2024-06-03T01:21:12.841 INFO:tasks.workunit.client.0.smithi032.stdout:9/731: dwrite d0/da4/fbe [0,4194304] 0 2024-06-03T01:21:12.841 INFO:tasks.workunit.client.0.smithi032.stdout:9/732: readlink d0/d11/l5f 0 2024-06-03T01:21:12.845 INFO:tasks.workunit.client.0.smithi032.stdout:8/633: creat dc/d25/d28/d58/d8d/d94/da6/fc4 x:0 0 0 2024-06-03T01:21:12.846 INFO:tasks.workunit.client.0.smithi032.stdout:7/685: dwrite dc/d12/d24/f57 [4194304,4194304] 0 2024-06-03T01:21:12.848 INFO:tasks.workunit.client.0.smithi032.stdout:6/556: rename d8/d11/d20/d2b/d39/f1f to d8/d11/d20/d36/dae/fb0 0 2024-06-03T01:21:12.852 INFO:tasks.workunit.client.0.smithi032.stdout:2/915: link d5/da/d1a/d2a/d36/d3f/fd6 d5/d38/d46/d53/d57/d6d/dce/d67/d104/d10b/f135 0 2024-06-03T01:21:12.854 INFO:tasks.workunit.client.0.smithi032.stdout:0/759: creat d1/da/d28/d3b/dce/fed x:0 0 0 2024-06-03T01:21:12.858 INFO:tasks.workunit.client.0.smithi032.stdout:6/557: link d8/d11/c14 d8/d11/d20/d31/cb1 0 2024-06-03T01:21:12.860 INFO:tasks.workunit.client.0.smithi032.stdout:2/916: symlink d5/d38/d46/d53/d57/d6d/dce/d67/df8/l136 0 2024-06-03T01:21:12.864 INFO:tasks.workunit.client.0.smithi032.stdout:0/760: dread d1/da/d28/f33 [4194304,4194304] 0 2024-06-03T01:21:12.865 INFO:tasks.workunit.client.0.smithi032.stdout:0/761: symlink d1/da/dd2/lee 0 2024-06-03T01:21:12.865 INFO:tasks.workunit.client.0.smithi032.stdout:0/762: chown d1/da/d28/d3b/d5d/lb6 18246876 1 2024-06-03T01:21:12.866 INFO:tasks.workunit.client.0.smithi032.stdout:0/763: mknod d1/da/d28/d3b/d5d/da9/daa/cef 0 2024-06-03T01:21:12.866 INFO:tasks.workunit.client.0.smithi032.stdout:0/764: chown d1/da/d10/f6f 13816117 1 2024-06-03T01:21:12.869 INFO:tasks.workunit.client.0.smithi032.stdout:0/765: rmdir d1/da/dd2/dd6 0 2024-06-03T01:21:12.870 INFO:tasks.workunit.client.0.smithi032.stdout:0/766: link d1/da/d28/d3b/d68/db5/cdf d1/da/d10/d29/d6c/cf0 0 2024-06-03T01:21:12.871 INFO:tasks.workunit.client.0.smithi032.stdout:0/767: symlink d1/da/d28/d3b/d68/d82/lf1 0 2024-06-03T01:21:12.873 INFO:tasks.workunit.client.0.smithi032.stdout:0/768: write d1/da/d10/f2d [4604259,56288] 0 2024-06-03T01:21:12.874 INFO:tasks.workunit.client.0.smithi032.stdout:6/558: dread d8/d11/d20/f8c [0,4194304] 0 2024-06-03T01:21:12.874 INFO:tasks.workunit.client.0.smithi032.stdout:6/559: dread - d8/d11/d20/d2b/d39/fa4 zero size 2024-06-03T01:21:12.874 INFO:tasks.workunit.client.0.smithi032.stdout:6/560: fsync d8/d11/d20/d2b/d39/f82 0 2024-06-03T01:21:12.876 INFO:tasks.workunit.client.0.smithi032.stdout:0/769: unlink d1/da/d10/d53/f7e 0 2024-06-03T01:21:12.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/733: dwrite d0/d11/d9c/fa3 [0,4194304] 0 2024-06-03T01:21:12.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/734: creat d0/d11/d76/ff3 x:0 0 0 2024-06-03T01:21:12.881 INFO:tasks.workunit.client.0.smithi032.stdout:0/770: rename d1/da/db2/lb8 to d1/d46/d3a/lf2 0 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/771: creat d1/da/d28/d3b/d50/ff3 x:0 0 0 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/772: creat d1/da/d10/d29/ff4 x:0 0 0 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/773: write d1/da/d28/d3b/d68/fa3 [6479705,803] 0 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/774: dread - d1/da/d28/d3b/d50/f8a zero size 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/775: chown d1/da/d28/f56 1562 1 2024-06-03T01:21:12.885 INFO:tasks.workunit.client.0.smithi032.stdout:0/776: creat d1/da/d28/d3b/d5d/d8d/dbe/dc4/ff5 x:0 0 0 2024-06-03T01:21:12.888 INFO:tasks.workunit.client.0.smithi032.stdout:8/634: dwrite dc/de/d1e/d43/d9b/f9c [0,4194304] 0 2024-06-03T01:21:12.891 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:12 smithi044 ceph-mon[36956]: pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 233 MiB/s rd, 283 MiB/s wr, 423 op/s 2024-06-03T01:21:12.892 INFO:tasks.workunit.client.0.smithi032.stdout:7/686: dwrite dc/f2d [0,4194304] 0 2024-06-03T01:21:12.892 INFO:tasks.workunit.client.0.smithi032.stdout:7/687: fdatasync dc/d12/d2a/d2f/d76/fae 0 2024-06-03T01:21:12.892 INFO:tasks.workunit.client.0.smithi032.stdout:6/561: dwrite d8/d11/d20/d2b/d39/f67 [0,4194304] 0 2024-06-03T01:21:12.894 INFO:tasks.workunit.client.0.smithi032.stdout:2/917: dwrite d5/da/d1a/f41 [0,4194304] 0 2024-06-03T01:21:12.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/735: dread d0/d23/f85 [0,4194304] 0 2024-06-03T01:21:12.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/736: read - d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/fd3 zero size 2024-06-03T01:21:12.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/635: mknod dc/de/d1e/d43/d9b/d76/cc5 0 2024-06-03T01:21:12.897 INFO:tasks.workunit.client.0.smithi032.stdout:7/688: unlink dc/d12/d6a/d86/d88/d8b/db8/fba 0 2024-06-03T01:21:12.903 INFO:tasks.workunit.client.0.smithi032.stdout:0/777: dwrite d1/f17 [0,4194304] 0 2024-06-03T01:21:12.903 INFO:tasks.workunit.client.0.smithi032.stdout:6/562: mkdir d8/d11/db2 0 2024-06-03T01:21:12.903 INFO:tasks.workunit.client.0.smithi032.stdout:6/563: creat d8/d11/d20/d2b/fb3 x:0 0 0 2024-06-03T01:21:12.905 INFO:tasks.workunit.client.0.smithi032.stdout:0/778: dread d1/da/d28/d3b/d5d/f94 [0,4194304] 0 2024-06-03T01:21:12.905 INFO:tasks.workunit.client.0.smithi032.stdout:2/918: truncate d5/f1c 1402888 0 2024-06-03T01:21:12.906 INFO:tasks.workunit.client.0.smithi032.stdout:9/737: mknod d0/d23/d25/cf4 0 2024-06-03T01:21:12.907 INFO:tasks.workunit.client.0.smithi032.stdout:9/738: dread - d0/da4/fb2 zero size 2024-06-03T01:21:12.908 INFO:tasks.workunit.client.0.smithi032.stdout:8/636: mknod dc/d25/d28/d58/cc6 0 2024-06-03T01:21:12.908 INFO:tasks.workunit.client.0.smithi032.stdout:6/564: dread d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:12.908 INFO:tasks.workunit.client.0.smithi032.stdout:6/565: stat d8/fb 0 2024-06-03T01:21:12.913 INFO:tasks.workunit.client.0.smithi032.stdout:0/779: creat d1/da/d28/d3b/d68/d83/ff6 x:0 0 0 2024-06-03T01:21:12.914 INFO:tasks.workunit.client.0.smithi032.stdout:8/637: creat dc/d25/d64/db5/fc7 x:0 0 0 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:8/638: write dc/de/f91 [5139000,74667] 0 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:3/656: sync 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/675: sync 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:1/639: sync 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/676: stat d4/d78 0 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:1/640: stat d18/d32/d39/c82 0 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:1/641: chown d18/l2c 5 1 2024-06-03T01:21:12.915 INFO:tasks.workunit.client.0.smithi032.stdout:3/657: creat de/d29/d2e/d31/d23/d53/d69/fd6 x:0 0 0 2024-06-03T01:21:12.917 INFO:tasks.workunit.client.0.smithi032.stdout:6/566: symlink d8/d11/d20/d36/da5/lb4 0 2024-06-03T01:21:12.917 INFO:tasks.workunit.client.0.smithi032.stdout:6/567: write d8/d11/d20/d36/f7d [4675957,84540] 0 2024-06-03T01:21:12.918 INFO:tasks.workunit.client.0.smithi032.stdout:2/919: dread d5/da/f2f [0,4194304] 0 2024-06-03T01:21:12.919 INFO:tasks.workunit.client.0.smithi032.stdout:0/780: unlink d1/da/ld1 0 2024-06-03T01:21:12.923 INFO:tasks.workunit.client.0.smithi032.stdout:9/739: dwrite d0/da4/fbe [0,4194304] 0 2024-06-03T01:21:12.924 INFO:tasks.workunit.client.0.smithi032.stdout:1/642: truncate d18/d3f/d5f/d6b/f73 1056048 0 2024-06-03T01:21:12.926 INFO:tasks.workunit.client.0.smithi032.stdout:8/639: write dc/d25/d28/d32/f50 [2724385,81739] 0 2024-06-03T01:21:12.928 INFO:tasks.workunit.client.0.smithi032.stdout:3/658: rename de/d29/d2e/d31/d23/d53/c7c to de/d29/d2e/d31/d8d/dc3/d7e/cd7 0 2024-06-03T01:21:12.928 INFO:tasks.workunit.client.0.smithi032.stdout:3/659: chown de 368248 1 2024-06-03T01:21:12.931 INFO:tasks.workunit.client.0.smithi032.stdout:0/781: mknod d1/da/d10/cf7 0 2024-06-03T01:21:12.934 INFO:tasks.workunit.client.0.smithi032.stdout:4/677: dwrite d4/db/da/d1d/d6b/fba [0,4194304] 0 2024-06-03T01:21:12.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/782: dread d1/f18 [0,4194304] 0 2024-06-03T01:21:12.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/783: creat d1/da/d28/d3b/d68/d82/ff8 x:0 0 0 2024-06-03T01:21:12.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/784: chown d1/da/d28/d3b/d5d/c7c 281568 1 2024-06-03T01:21:12.938 INFO:tasks.workunit.client.0.smithi032.stdout:1/643: truncate d18/f1a 1290013 0 2024-06-03T01:21:12.938 INFO:tasks.workunit.client.0.smithi032.stdout:1/644: fsync d18/d3f/d5f/d94/f96 0 2024-06-03T01:21:12.939 INFO:tasks.workunit.client.0.smithi032.stdout:6/568: rename d8/d11/d20/d21/d9a/la6 to d8/d50/lb5 0 2024-06-03T01:21:12.941 INFO:tasks.workunit.client.0.smithi032.stdout:3/660: mknod de/d29/d2e/d31/d3c/d50/d56/cd8 0 2024-06-03T01:21:12.944 INFO:tasks.workunit.client.0.smithi032.stdout:4/678: dread d4/db/da/d37/f48 [0,4194304] 0 2024-06-03T01:21:12.945 INFO:tasks.workunit.client.0.smithi032.stdout:1/645: symlink d18/d3f/d5f/d7b/da8/lc1 0 2024-06-03T01:21:12.947 INFO:tasks.workunit.client.0.smithi032.stdout:6/569: mknod d8/d11/d20/d3b/d66/cb6 0 2024-06-03T01:21:12.951 INFO:tasks.workunit.client.0.smithi032.stdout:0/785: getdents d1/da/d10/d80 0 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:6/570: unlink d8/d11/d20/d2b/d39/c1b 0 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:8/640: dwrite dc/de/d1e/d43/d9b/d76/fb7 [0,4194304] 0 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:8/641: chown dc/d25/d28/d32/f50 13017 1 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:9/740: dwrite d0/d11/d1d/d38/d4f/fa1 [0,4194304] 0 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:9/741: fdatasync d0/d23/d25/f52 0 2024-06-03T01:21:12.955 INFO:tasks.workunit.client.0.smithi032.stdout:9/742: write d0/d11/fd2 [648982,119454] 0 2024-06-03T01:21:12.956 INFO:tasks.workunit.client.0.smithi032.stdout:0/786: mknod d1/da/d10/d53/cf9 0 2024-06-03T01:21:12.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/787: dread d1/f18 [0,4194304] 0 2024-06-03T01:21:12.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/788: chown d1/da/db2/fe9 15774 1 2024-06-03T01:21:12.962 INFO:tasks.workunit.client.0.smithi032.stdout:0/789: truncate d1/da/d28/d3b/d68/db5/fd7 1045865 0 2024-06-03T01:21:12.963 INFO:tasks.workunit.client.0.smithi032.stdout:9/743: mknod d0/ddd/cf5 0 2024-06-03T01:21:12.966 INFO:tasks.workunit.client.0.smithi032.stdout:0/790: creat d1/da/d28/d3b/d5d/da9/ffa x:0 0 0 2024-06-03T01:21:12.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/571: write d8/d11/d20/d2b/d39/d19/f40 [1707667,109525] 0 2024-06-03T01:21:12.971 INFO:tasks.workunit.client.0.smithi032.stdout:0/791: mknod d1/da/d28/d3b/d5d/d8d/dbe/dc4/cfb 0 2024-06-03T01:21:12.972 INFO:tasks.workunit.client.0.smithi032.stdout:0/792: read - d1/da/d28/d3b/d50/ff3 zero size 2024-06-03T01:21:12.972 INFO:tasks.workunit.client.0.smithi032.stdout:0/793: truncate d1/da/d28/d3b/d68/d82/ff8 562553 0 2024-06-03T01:21:12.984 INFO:tasks.workunit.client.0.smithi032.stdout:0/794: dread d1/f2b [0,4194304] 0 2024-06-03T01:21:12.984 INFO:tasks.workunit.client.0.smithi032.stdout:6/572: write d8/d11/d20/d2b/d39/f67 [1725370,24207] 0 2024-06-03T01:21:12.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/795: link d1/da/d28/d3b/d68/d83/lb7 d1/da/d10/d80/lfc 0 2024-06-03T01:21:12.989 INFO:tasks.workunit.client.0.smithi032.stdout:0/796: rename d1/da/f1a to d1/da/d28/ffd 0 2024-06-03T01:21:12.993 INFO:tasks.workunit.client.0.smithi032.stdout:4/679: dwrite d4/db/da/d1d/faf [0,4194304] 0 2024-06-03T01:21:12.996 INFO:tasks.workunit.client.0.smithi032.stdout:1/646: dwrite d18/d3f/fb9 [4194304,4194304] 0 2024-06-03T01:21:12.996 INFO:tasks.workunit.client.0.smithi032.stdout:9/744: dwrite d0/d11/d1d/fbb [0,4194304] 0 2024-06-03T01:21:12.998 INFO:tasks.workunit.client.0.smithi032.stdout:6/573: dread f4 [0,4194304] 0 2024-06-03T01:21:13.001 INFO:tasks.workunit.client.0.smithi032.stdout:0/797: dread d1/da/d28/d3b/d5d/d8d/dbe/dc4/fd3 [0,4194304] 0 2024-06-03T01:21:13.002 INFO:tasks.workunit.client.0.smithi032.stdout:0/798: readlink d1/da/d28/d3b/d5d/d8d/lc9 0 2024-06-03T01:21:13.003 INFO:tasks.workunit.client.0.smithi032.stdout:9/745: rmdir d0/d23/d25/de1/df1/dea 39 2024-06-03T01:21:13.005 INFO:tasks.workunit.client.0.smithi032.stdout:6/574: creat d8/d4c/daa/fb7 x:0 0 0 2024-06-03T01:21:13.007 INFO:tasks.workunit.client.0.smithi032.stdout:4/680: write d4/db/da/d5d/dc5/fd3 [320018,68902] 0 2024-06-03T01:21:13.007 INFO:tasks.workunit.client.0.smithi032.stdout:4/681: chown d4/db/da/d2d/d27/d8c/db5/fb0 847336076 1 2024-06-03T01:21:13.008 INFO:tasks.workunit.client.0.smithi032.stdout:0/799: link d1/da/d28/cd9 d1/da/d28/d3b/d5d/d8d/d9c/dc2/cfe 0 2024-06-03T01:21:13.008 INFO:tasks.workunit.client.0.smithi032.stdout:0/800: readlink d1/d46/d3a/l8c 0 2024-06-03T01:21:13.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/575: dread d8/d50/f62 [0,4194304] 0 2024-06-03T01:21:13.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/746: unlink d0/d23/d25/de1/df1/dea/f82 0 2024-06-03T01:21:13.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/747: fsync d0/d11/d1d/d38/d4f/db0/ddb/fed 0 2024-06-03T01:21:13.012 INFO:tasks.workunit.client.0.smithi032.stdout:8/642: truncate dc/de/d1e/f61 3539789 0 2024-06-03T01:21:13.014 INFO:tasks.workunit.client.0.smithi032.stdout:0/801: creat d1/da/d28/d3b/d68/db5/fff x:0 0 0 2024-06-03T01:21:13.016 INFO:tasks.workunit.client.0.smithi032.stdout:6/576: unlink d8/d5f/l99 0 2024-06-03T01:21:13.016 INFO:tasks.workunit.client.0.smithi032.stdout:6/577: dread - d8/d50/fac zero size 2024-06-03T01:21:13.016 INFO:tasks.workunit.client.0.smithi032.stdout:1/647: dwrite d18/d32/d3c/f67 [0,4194304] 0 2024-06-03T01:21:13.023 INFO:tasks.workunit.client.0.smithi032.stdout:0/802: dread d1/da/d10/f6f [0,4194304] 0 2024-06-03T01:21:13.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/682: dwrite d4/db/da/d1d/d6b/feb [0,4194304] 0 2024-06-03T01:21:13.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/683: chown d4/db/da/d37/l76 218 1 2024-06-03T01:21:13.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/684: read d4/db/da/fb6 [419425,105450] 0 2024-06-03T01:21:13.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/685: write d4/d57/f9c [625643,47634] 0 2024-06-03T01:21:13.025 INFO:tasks.workunit.client.0.smithi032.stdout:6/578: symlink d8/d11/d20/d3b/lb8 0 2024-06-03T01:21:13.026 INFO:tasks.workunit.client.0.smithi032.stdout:1/648: creat d18/d3f/d5f/d7b/da8/fc2 x:0 0 0 2024-06-03T01:21:13.026 INFO:tasks.workunit.client.0.smithi032.stdout:9/748: rmdir d0/d11/d1d/d38/d4f/db0/ddb 39 2024-06-03T01:21:13.030 INFO:tasks.workunit.client.0.smithi032.stdout:8/643: getdents dc/de/d1e 0 2024-06-03T01:21:13.030 INFO:tasks.workunit.client.0.smithi032.stdout:8/644: getdents dc/de/d1e/d43/d9b/d76/dac 0 2024-06-03T01:21:13.030 INFO:tasks.workunit.client.0.smithi032.stdout:8/645: truncate dc/de/d1e/fa0 156604 0 2024-06-03T01:21:13.030 INFO:tasks.workunit.client.0.smithi032.stdout:8/646: chown dc/de/d1e/d43/lb4 41381 1 2024-06-03T01:21:13.032 INFO:tasks.workunit.client.0.smithi032.stdout:1/649: dread d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [0,4194304] 0 2024-06-03T01:21:13.035 INFO:tasks.workunit.client.0.smithi032.stdout:0/803: unlink d1/l3 0 2024-06-03T01:21:13.035 INFO:tasks.workunit.client.0.smithi032.stdout:0/804: write d1/d46/f48 [5235288,102316] 0 2024-06-03T01:21:13.041 INFO:tasks.workunit.client.0.smithi032.stdout:6/579: rename d8/d55/c3a to d8/d11/d20/d2b/d39/cb9 0 2024-06-03T01:21:13.049 INFO:tasks.workunit.client.0.smithi032.stdout:1/650: mknod d18/d27/da7/cc3 0 2024-06-03T01:21:13.052 INFO:tasks.workunit.client.0.smithi032.stdout:0/805: dwrite d1/da/d28/d3b/d68/db5/fde [0,4194304] 0 2024-06-03T01:21:13.053 INFO:tasks.workunit.client.0.smithi032.stdout:8/647: rename dc/d14/c5f to dc/d25/d28/d58/d8d/cc8 0 2024-06-03T01:21:13.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/749: rmdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9 39 2024-06-03T01:21:13.056 INFO:tasks.workunit.client.0.smithi032.stdout:6/580: rmdir d8/d11/d20/d36/da5 39 2024-06-03T01:21:13.065 INFO:tasks.workunit.client.0.smithi032.stdout:1/651: dread d18/d2d/f69 [0,4194304] 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:8/648: dread dc/d14/f2d [0,4194304] 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/806: mkdir d1/da/d10/d100 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/807: truncate d1/da/d10/d29/f45 136508 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/808: write d1/da/d28/d3b/f76 [2225330,66276] 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/809: stat d1/d46/c62 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/810: fdatasync d1/da/d10/d29/ff4 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/811: stat d1/da/db2/fe9 0 2024-06-03T01:21:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:0/812: write d1/da/d28/fe4 [577834,82161] 0 2024-06-03T01:21:13.067 INFO:tasks.workunit.client.0.smithi032.stdout:9/750: rename d0/d23/l2d to d0/d11/d1d/lf6 0 2024-06-03T01:21:13.068 INFO:tasks.workunit.client.0.smithi032.stdout:9/751: write d0/d11/f48 [4211573,62402] 0 2024-06-03T01:21:13.074 INFO:tasks.workunit.client.0.smithi032.stdout:8/649: dread dc/d25/f88 [0,4194304] 0 2024-06-03T01:21:13.075 INFO:tasks.workunit.client.0.smithi032.stdout:8/650: write dc/de/d1e/d43/d9b/d76/f8a [797737,116387] 0 2024-06-03T01:21:13.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/651: creat dc/d25/d28/d58/d8d/d94/d59/fc9 x:0 0 0 2024-06-03T01:21:13.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/652: dread - dc/d25/f6c zero size 2024-06-03T01:21:13.077 INFO:tasks.workunit.client.0.smithi032.stdout:8/653: mknod dc/d25/cca 0 2024-06-03T01:21:13.078 INFO:tasks.workunit.client.0.smithi032.stdout:9/752: write d0/d11/d1d/f99 [1981418,28658] 0 2024-06-03T01:21:13.080 INFO:tasks.workunit.client.0.smithi032.stdout:9/753: rename d0/d23/f3d to d0/d23/d24/ff7 0 2024-06-03T01:21:13.080 INFO:tasks.workunit.client.0.smithi032.stdout:9/754: stat d0/l7 0 2024-06-03T01:21:13.081 INFO:tasks.workunit.client.0.smithi032.stdout:9/755: rename d0/d11/d1d/d38/l6a to d0/d11/d76/lf8 0 2024-06-03T01:21:13.084 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:12 smithi032 ceph-mon[29360]: pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 233 MiB/s rd, 283 MiB/s wr, 423 op/s 2024-06-03T01:21:13.151 INFO:tasks.workunit.client.0.smithi032.stdout:1/652: read d18/d2d/d5e/f64 [3728188,94303] 0 2024-06-03T01:21:13.152 INFO:tasks.workunit.client.0.smithi032.stdout:0/813: dread d1/da/d10/f4f [0,4194304] 0 2024-06-03T01:21:13.154 INFO:tasks.workunit.client.0.smithi032.stdout:9/756: dwrite d0/f19 [4194304,4194304] 0 2024-06-03T01:21:13.156 INFO:tasks.workunit.client.0.smithi032.stdout:0/814: creat d1/da/d28/d3b/d68/f101 x:0 0 0 2024-06-03T01:21:13.156 INFO:tasks.workunit.client.0.smithi032.stdout:0/815: creat d1/da/d28/d3b/d5d/d8d/d9c/f102 x:0 0 0 2024-06-03T01:21:13.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/581: dwrite d8/d11/f17 [4194304,4194304] 0 2024-06-03T01:21:13.159 INFO:tasks.workunit.client.0.smithi032.stdout:0/816: mkdir d1/da/db2/d103 0 2024-06-03T01:21:13.159 INFO:tasks.workunit.client.0.smithi032.stdout:0/817: creat d1/da/d28/d3b/dce/f104 x:0 0 0 2024-06-03T01:21:13.162 INFO:tasks.workunit.client.0.smithi032.stdout:8/654: dwrite dc/d25/d28/d58/d8d/d94/d59/f9e [0,4194304] 0 2024-06-03T01:21:13.162 INFO:tasks.workunit.client.0.smithi032.stdout:8/655: truncate dc/d25/d28/fb8 513294 0 2024-06-03T01:21:13.162 INFO:tasks.workunit.client.0.smithi032.stdout:1/653: write d18/d3f/d5f/d98/fa2 [1190924,107073] 0 2024-06-03T01:21:13.162 INFO:tasks.workunit.client.0.smithi032.stdout:1/654: write d18/d3f/d5f/d98/fa2 [4819657,65619] 0 2024-06-03T01:21:13.163 INFO:tasks.workunit.client.0.smithi032.stdout:9/757: dread d0/d23/d24/ff7 [0,4194304] 0 2024-06-03T01:21:13.163 INFO:tasks.workunit.client.0.smithi032.stdout:6/582: rename d8/d11/d20/d31/f6c to d8/d11/d20/d21/d9a/fba 0 2024-06-03T01:21:13.166 INFO:tasks.workunit.client.0.smithi032.stdout:0/818: mkdir d1/da/d10/d53/d105 0 2024-06-03T01:21:13.166 INFO:tasks.workunit.client.0.smithi032.stdout:0/819: write d1/da/d28/d3b/d5d/f67 [524971,121891] 0 2024-06-03T01:21:13.166 INFO:tasks.workunit.client.0.smithi032.stdout:0/820: creat d1/da/d28/d3b/dce/f106 x:0 0 0 2024-06-03T01:21:13.173 INFO:tasks.workunit.client.0.smithi032.stdout:8/656: truncate dc/de/d1e/f61 1174322 0 2024-06-03T01:21:13.173 INFO:tasks.workunit.client.0.smithi032.stdout:8/657: fsync dc/de/d1e/d43/d9b/d76/fab 0 2024-06-03T01:21:13.173 INFO:tasks.workunit.client.0.smithi032.stdout:8/658: chown dc/d25/d28/d3c/f5a 42 1 2024-06-03T01:21:13.177 INFO:tasks.workunit.client.0.smithi032.stdout:1/655: link d18/d3f/d5f/d6b/d78/c5a d18/d27/d2b/cc4 0 2024-06-03T01:21:13.179 INFO:tasks.workunit.client.0.smithi032.stdout:9/758: rmdir d0/d11/d1d/d38/d4f/db0 39 2024-06-03T01:21:13.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/583: rename d8/d11/c1e to d8/d11/db2/cbb 0 2024-06-03T01:21:13.186 INFO:tasks.workunit.client.0.smithi032.stdout:1/656: link d18/d3f/d5f/d6b/d9d/fac d18/d32/d39/fc5 0 2024-06-03T01:21:13.186 INFO:tasks.workunit.client.0.smithi032.stdout:1/657: write d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 [790834,67874] 0 2024-06-03T01:21:13.186 INFO:tasks.workunit.client.0.smithi032.stdout:1/658: truncate d18/d3f/d5f/d6b/d78/f83 630391 0 2024-06-03T01:21:13.187 INFO:tasks.workunit.client.0.smithi032.stdout:9/759: creat d0/d11/d1d/d38/ff9 x:0 0 0 2024-06-03T01:21:13.189 INFO:tasks.workunit.client.0.smithi032.stdout:0/821: rename d1/d46/cea to d1/da/d28/d3b/d50/c107 0 2024-06-03T01:21:13.189 INFO:tasks.workunit.client.0.smithi032.stdout:0/822: creat d1/da/d28/d3b/d5d/f108 x:0 0 0 2024-06-03T01:21:13.190 INFO:tasks.workunit.client.0.smithi032.stdout:8/659: symlink dc/d14/lcb 0 2024-06-03T01:21:13.191 INFO:tasks.workunit.client.0.smithi032.stdout:8/660: write dc/d25/d28/d58/f79 [3573593,120284] 0 2024-06-03T01:21:13.192 INFO:tasks.workunit.client.0.smithi032.stdout:9/760: rmdir d0/d23/d25/de1/df1 39 2024-06-03T01:21:13.196 INFO:tasks.workunit.client.0.smithi032.stdout:8/661: rename dc/d25/d28/d58/dad/cb9 to dc/d25/d28/d32/ccc 0 2024-06-03T01:21:13.196 INFO:tasks.workunit.client.0.smithi032.stdout:8/662: fsync dc/d25/d28/d3c/f42 0 2024-06-03T01:21:13.197 INFO:tasks.workunit.client.0.smithi032.stdout:1/659: dread d18/d3f/d5f/d6b/d78/d3d/d6a/fa4 [0,4194304] 0 2024-06-03T01:21:13.197 INFO:tasks.workunit.client.0.smithi032.stdout:9/761: unlink d0/da4/faf 0 2024-06-03T01:21:13.197 INFO:tasks.workunit.client.0.smithi032.stdout:9/762: fdatasync d0/d11/d1d/d38/d4f/f7a 0 2024-06-03T01:21:13.198 INFO:tasks.workunit.client.0.smithi032.stdout:1/660: symlink d18/d32/lc6 0 2024-06-03T01:21:13.200 INFO:tasks.workunit.client.0.smithi032.stdout:0/823: dread d1/da/d28/d3b/d5d/fa5 [0,4194304] 0 2024-06-03T01:21:13.200 INFO:tasks.workunit.client.0.smithi032.stdout:0/824: creat d1/da/db2/f109 x:0 0 0 2024-06-03T01:21:13.200 INFO:tasks.workunit.client.0.smithi032.stdout:1/661: link d18/d2d/d5e/c85 d18/d3f/d5f/d7b/da8/cc7 0 2024-06-03T01:21:13.204 INFO:tasks.workunit.client.0.smithi032.stdout:0/825: rename d1/f2b to d1/da/d28/d3b/d50/dec/f10a 0 2024-06-03T01:21:13.204 INFO:tasks.workunit.client.0.smithi032.stdout:0/826: write d1/da/d28/d3b/d5d/d8d/dbe/dc4/fe8 [950522,96459] 0 2024-06-03T01:21:13.205 INFO:tasks.workunit.client.0.smithi032.stdout:0/827: chown d1/da/d10/d80 98043483 1 2024-06-03T01:21:13.207 INFO:tasks.workunit.client.0.smithi032.stdout:6/584: dwrite d8/d11/d20/d3b/f80 [0,4194304] 0 2024-06-03T01:21:13.207 INFO:tasks.workunit.client.0.smithi032.stdout:1/662: getdents d18 0 2024-06-03T01:21:13.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/828: unlink d1/lf 0 2024-06-03T01:21:13.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/829: fdatasync d1/da/db2/fe9 0 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:0/830: dread d1/da/d10/f19 [4194304,4194304] 0 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:0/831: truncate d1/da/d28/d3b/dce/fed 278106 0 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:6/585: mknod d8/d11/d20/d21/d9a/cbc 0 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:0/832: dread - d1/da/d28/d3b/d5d/d8d/dbe/dc4/ff5 zero size 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:0/833: stat d1/da/d10/l97 0 2024-06-03T01:21:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:8/663: dwrite dc/d25/d28/d51/d5d/d5e/f84 [0,4194304] 0 2024-06-03T01:21:13.218 INFO:tasks.workunit.client.0.smithi032.stdout:1/663: rename d18/d32/d3c/l49 to d18/d32/daa/lc8 0 2024-06-03T01:21:13.223 INFO:tasks.workunit.client.0.smithi032.stdout:6/586: dread f3 [0,4194304] 0 2024-06-03T01:21:13.225 INFO:tasks.workunit.client.0.smithi032.stdout:0/834: mknod d1/da/d28/d3b/d5d/da9/daa/c10b 0 2024-06-03T01:21:13.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/664: creat d18/dc0/fc9 x:0 0 0 2024-06-03T01:21:13.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/665: readlink d18/d27/l79 0 2024-06-03T01:21:13.231 INFO:tasks.workunit.client.0.smithi032.stdout:6/587: mknod d8/d11/d20/d31/cbd 0 2024-06-03T01:21:13.231 INFO:tasks.workunit.client.0.smithi032.stdout:6/588: stat d8/d11 0 2024-06-03T01:21:13.231 INFO:tasks.workunit.client.0.smithi032.stdout:6/589: readlink d8/d11/l79 0 2024-06-03T01:21:13.233 INFO:tasks.workunit.client.0.smithi032.stdout:8/664: dwrite dc/de/d1e/d43/d9b/f9c [0,4194304] 0 2024-06-03T01:21:13.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/835: rename d1/da/d28/d3b/d5d/d8d/d9c/dc2/cfe to d1/da/dd2/c10c 0 2024-06-03T01:21:13.235 INFO:tasks.workunit.client.0.smithi032.stdout:1/666: mknod d18/d27/d89/cca 0 2024-06-03T01:21:13.235 INFO:tasks.workunit.client.0.smithi032.stdout:1/667: write d18/d3f/d5f/d6b/d78/f83 [1256869,55546] 0 2024-06-03T01:21:13.235 INFO:tasks.workunit.client.0.smithi032.stdout:1/668: fsync d18/d32/d39/f4a 0 2024-06-03T01:21:13.239 INFO:tasks.workunit.client.0.smithi032.stdout:8/665: rename dc/l56 to dc/d25/d28/d58/lcd 0 2024-06-03T01:21:13.239 INFO:tasks.workunit.client.0.smithi032.stdout:8/666: fdatasync dc/de/f95 0 2024-06-03T01:21:13.241 INFO:tasks.workunit.client.0.smithi032.stdout:6/590: write d8/d11/d20/d21/f27 [737271,79114] 0 2024-06-03T01:21:13.243 INFO:tasks.workunit.client.0.smithi032.stdout:1/669: symlink d18/d32/d3c/lcb 0 2024-06-03T01:21:13.243 INFO:tasks.workunit.client.0.smithi032.stdout:1/670: chown d18/dc0/fc9 94776275 1 2024-06-03T01:21:13.245 INFO:tasks.workunit.client.0.smithi032.stdout:8/667: symlink dc/d25/d64/lce 0 2024-06-03T01:21:13.247 INFO:tasks.workunit.client.0.smithi032.stdout:6/591: mknod d8/d11/d20/d36/cbe 0 2024-06-03T01:21:13.249 INFO:tasks.workunit.client.0.smithi032.stdout:8/668: mknod dc/de/d1e/d43/d9b/d76/ccf 0 2024-06-03T01:21:13.250 INFO:tasks.workunit.client.0.smithi032.stdout:0/836: dwrite d1/fc [0,4194304] 0 2024-06-03T01:21:13.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/592: creat d8/d11/d20/d31/fbf x:0 0 0 2024-06-03T01:21:13.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/593: write d8/d11/d20/d2b/d39/fa4 [250793,114252] 0 2024-06-03T01:21:13.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/594: chown d8/d11/d20/d21/fa2 0 1 2024-06-03T01:21:13.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/595: dread - d8/d11/fa8 zero size 2024-06-03T01:21:13.253 INFO:tasks.workunit.client.0.smithi032.stdout:0/837: creat d1/da/d28/d3b/d5d/da9/daa/f10d x:0 0 0 2024-06-03T01:21:13.255 INFO:tasks.workunit.client.0.smithi032.stdout:0/838: mkdir d1/da/d10/d29/d10e 0 2024-06-03T01:21:13.261 INFO:tasks.workunit.client.0.smithi032.stdout:1/671: dwrite d18/d32/f52 [0,4194304] 0 2024-06-03T01:21:13.261 INFO:tasks.workunit.client.0.smithi032.stdout:1/672: write d18/dc0/fc9 [444211,51951] 0 2024-06-03T01:21:13.262 INFO:tasks.workunit.client.0.smithi032.stdout:6/596: read d8/fb [863803,101313] 0 2024-06-03T01:21:13.264 INFO:tasks.workunit.client.0.smithi032.stdout:0/839: write d1/da/d28/f8b [2591402,71045] 0 2024-06-03T01:21:13.264 INFO:tasks.workunit.client.0.smithi032.stdout:0/840: chown d1/da/d28/d3b/d68/ddd 58003719 1 2024-06-03T01:21:13.266 INFO:tasks.workunit.client.0.smithi032.stdout:8/669: dwrite f2 [0,4194304] 0 2024-06-03T01:21:13.266 INFO:tasks.workunit.client.0.smithi032.stdout:8/670: write dc/d25/d28/d58/f86 [208067,122949] 0 2024-06-03T01:21:13.266 INFO:tasks.workunit.client.0.smithi032.stdout:1/673: getdents d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91 0 2024-06-03T01:21:13.266 INFO:tasks.workunit.client.0.smithi032.stdout:1/674: write d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 [1862325,90241] 0 2024-06-03T01:21:13.268 INFO:tasks.workunit.client.0.smithi032.stdout:6/597: dread d8/f6f [0,4194304] 0 2024-06-03T01:21:13.269 INFO:tasks.workunit.client.0.smithi032.stdout:0/841: mkdir d1/da/dd2/d10f 0 2024-06-03T01:21:13.271 INFO:tasks.workunit.client.0.smithi032.stdout:8/671: creat dc/d25/d28/d58/dad/fd0 x:0 0 0 2024-06-03T01:21:13.271 INFO:tasks.workunit.client.0.smithi032.stdout:8/672: write dc/d14/d31/f7b [1128312,19441] 0 2024-06-03T01:21:13.271 INFO:tasks.workunit.client.0.smithi032.stdout:8/673: creat dc/d25/d28/d58/d8d/d94/d59/fd1 x:0 0 0 2024-06-03T01:21:13.272 INFO:tasks.workunit.client.0.smithi032.stdout:8/674: readlink dc/l44 0 2024-06-03T01:21:13.272 INFO:tasks.workunit.client.0.smithi032.stdout:1/675: creat d18/d3f/fcc x:0 0 0 2024-06-03T01:21:13.273 INFO:tasks.workunit.client.0.smithi032.stdout:0/842: mkdir d1/da/db2/d110 0 2024-06-03T01:21:13.275 INFO:tasks.workunit.client.0.smithi032.stdout:6/598: dread d8/d11/f3e [0,4194304] 0 2024-06-03T01:21:13.276 INFO:tasks.workunit.client.0.smithi032.stdout:6/599: chown d8/d11/d20/f8c 0 1 2024-06-03T01:21:13.276 INFO:tasks.workunit.client.0.smithi032.stdout:8/675: truncate dc/de/d1e/d43/d9b/f9c 3009750 0 2024-06-03T01:21:13.278 INFO:tasks.workunit.client.0.smithi032.stdout:0/843: mknod d1/da/d28/d3b/c111 0 2024-06-03T01:21:13.278 INFO:tasks.workunit.client.0.smithi032.stdout:0/844: chown d1/da/d10/c66 2 1 2024-06-03T01:21:13.278 INFO:tasks.workunit.client.0.smithi032.stdout:6/600: rmdir d8/d11/d20/d2b/d39/d19 39 2024-06-03T01:21:13.278 INFO:tasks.workunit.client.0.smithi032.stdout:1/676: dread d18/d3f/d5f/d6b/d78/f45 [0,4194304] 0 2024-06-03T01:21:13.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/676: mknod dc/d25/d28/d33/d78/cd2 0 2024-06-03T01:21:13.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/677: write dc/d25/d28/fb8 [1321919,39300] 0 2024-06-03T01:21:13.282 INFO:tasks.workunit.client.0.smithi032.stdout:6/601: rename d8/d50/f62 to d8/d4c/fc0 0 2024-06-03T01:21:13.282 INFO:tasks.workunit.client.0.smithi032.stdout:6/602: dread - d8/d11/d20/d2b/f93 zero size 2024-06-03T01:21:13.283 INFO:tasks.workunit.client.0.smithi032.stdout:6/603: chown f3 1256 1 2024-06-03T01:21:13.287 INFO:tasks.workunit.client.0.smithi032.stdout:6/604: mkdir d8/d11/d20/d2b/d39/d7e/dc1 0 2024-06-03T01:21:13.287 INFO:tasks.workunit.client.0.smithi032.stdout:0/845: write d1/da/d10/f7a [3437751,88309] 0 2024-06-03T01:21:13.290 INFO:tasks.workunit.client.0.smithi032.stdout:6/605: rename d8/d11/d20/d2b/d39/f82 to d8/d11/d20/d3b/fc2 0 2024-06-03T01:21:13.293 INFO:tasks.workunit.client.0.smithi032.stdout:0/846: rmdir d1/da/d28/d3b/d5d/d8d/dbe/dc4 39 2024-06-03T01:21:13.293 INFO:tasks.workunit.client.0.smithi032.stdout:0/847: creat d1/da/d10/d53/f112 x:0 0 0 2024-06-03T01:21:13.294 INFO:tasks.workunit.client.0.smithi032.stdout:0/848: creat d1/f113 x:0 0 0 2024-06-03T01:21:13.295 INFO:tasks.workunit.client.0.smithi032.stdout:0/849: unlink d1/da/db2/fdc 0 2024-06-03T01:21:13.295 INFO:tasks.workunit.client.0.smithi032.stdout:0/850: truncate d1/faf 560188 0 2024-06-03T01:21:13.296 INFO:tasks.workunit.client.0.smithi032.stdout:0/851: fdatasync d1/da/f74 0 2024-06-03T01:21:13.296 INFO:tasks.workunit.client.0.smithi032.stdout:0/852: write d1/f18 [1124076,86394] 0 2024-06-03T01:21:13.296 INFO:tasks.workunit.client.0.smithi032.stdout:0/853: creat d1/da/d10/d80/d9f/dac/f114 x:0 0 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/678: dwrite dc/d25/d28/d51/f71 [0,4194304] 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/679: creat dc/d25/d64/db5/fd3 x:0 0 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/680: getdents dc/de/d1e 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/681: fdatasync dc/d25/d28/d58/d8d/d94/d59/fc9 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/682: fdatasync dc/d25/f6c 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/683: truncate f8 2017370 0 2024-06-03T01:21:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:0/854: mkdir d1/da/d10/d80/de5/d115 0 2024-06-03T01:21:13.300 INFO:tasks.workunit.client.0.smithi032.stdout:6/606: write d8/d11/d20/d2b/d39/d19/f46 [1351200,51669] 0 2024-06-03T01:21:13.300 INFO:tasks.workunit.client.0.smithi032.stdout:6/607: write d8/d11/d20/d3b/f41 [3939238,119435] 0 2024-06-03T01:21:13.302 INFO:tasks.workunit.client.0.smithi032.stdout:8/684: rename dc/de/d1e/d43/ca7 to dc/d25/d28/d51/cd4 0 2024-06-03T01:21:13.305 INFO:tasks.workunit.client.0.smithi032.stdout:8/685: link dc/d14/fc1 dc/d25/d28/fd5 0 2024-06-03T01:21:13.305 INFO:tasks.workunit.client.0.smithi032.stdout:8/686: fdatasync dc/d25/d28/d3c/f3f 0 2024-06-03T01:21:13.306 INFO:tasks.workunit.client.0.smithi032.stdout:8/687: read f8 [95125,35103] 0 2024-06-03T01:21:13.306 INFO:tasks.workunit.client.0.smithi032.stdout:1/677: truncate d18/d32/d3c/f67 469907 0 2024-06-03T01:21:13.307 INFO:tasks.workunit.client.0.smithi032.stdout:0/855: rmdir d1/da/d28/d3b/d68/db5 39 2024-06-03T01:21:13.307 INFO:tasks.workunit.client.0.smithi032.stdout:0/856: truncate d1/da/d10/f7a 4577959 0 2024-06-03T01:21:13.308 INFO:tasks.workunit.client.0.smithi032.stdout:6/608: write d8/d11/d20/d2b/f4a [488966,69994] 0 2024-06-03T01:21:13.308 INFO:tasks.workunit.client.0.smithi032.stdout:1/678: write d18/d3f/d5f/d6b/f73 [610793,95103] 0 2024-06-03T01:21:13.316 INFO:tasks.workunit.client.0.smithi032.stdout:6/609: creat d8/d11/d20/d3b/d66/fc3 x:0 0 0 2024-06-03T01:21:13.320 INFO:tasks.workunit.client.0.smithi032.stdout:1/679: dwrite d18/d3f/fcc [0,4194304] 0 2024-06-03T01:21:13.321 INFO:tasks.workunit.client.0.smithi032.stdout:6/610: link d8/d11/d20/d3b/d66/cb6 d8/d11/cc4 0 2024-06-03T01:21:13.321 INFO:tasks.workunit.client.0.smithi032.stdout:8/688: rmdir dc/de 39 2024-06-03T01:21:13.325 INFO:tasks.workunit.client.0.smithi032.stdout:1/680: write d18/d2d/f69 [3464129,53235] 0 2024-06-03T01:21:13.328 INFO:tasks.workunit.client.0.smithi032.stdout:6/611: rename d8/d11/d20/d31/l51 to d8/d11/d20/d2b/d39/lc5 0 2024-06-03T01:21:13.328 INFO:tasks.workunit.client.0.smithi032.stdout:6/612: chown d8/d11/d20/d2b/d39/d7e/c9c 200784 1 2024-06-03T01:21:13.328 INFO:tasks.workunit.client.0.smithi032.stdout:0/857: write d1/f43 [3250419,87392] 0 2024-06-03T01:21:13.330 INFO:tasks.workunit.client.0.smithi032.stdout:1/681: mknod d18/d3f/d5f/d6b/d78/ccd 0 2024-06-03T01:21:13.330 INFO:tasks.workunit.client.0.smithi032.stdout:1/682: write d18/d3f/d5f/d6b/d78/f83 [2349012,113705] 0 2024-06-03T01:21:13.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/613: rmdir d8/d4c 39 2024-06-03T01:21:13.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/614: creat d8/d11/d20/d31/fc6 x:0 0 0 2024-06-03T01:21:13.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/615: getdents d8/d11/d20/d89 0 2024-06-03T01:21:13.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/616: write d8/d50/fac [339818,25898] 0 2024-06-03T01:21:13.332 INFO:tasks.workunit.client.0.smithi032.stdout:0/858: creat d1/da/d10/d29/d6c/f116 x:0 0 0 2024-06-03T01:21:13.336 INFO:tasks.workunit.client.0.smithi032.stdout:8/689: getdents dc/d25/d28/d58/dad 0 2024-06-03T01:21:13.341 INFO:tasks.workunit.client.0.smithi032.stdout:1/683: dread d18/f5d [0,4194304] 0 2024-06-03T01:21:13.341 INFO:tasks.workunit.client.0.smithi032.stdout:1/684: chown d18/f5d 929 1 2024-06-03T01:21:13.342 INFO:tasks.workunit.client.0.smithi032.stdout:6/617: mkdir d8/d11/d20/d2b/dc7 0 2024-06-03T01:21:13.342 INFO:tasks.workunit.client.0.smithi032.stdout:8/690: mknod dc/d25/d28/d58/d8d/d94/da6/cd6 0 2024-06-03T01:21:13.344 INFO:tasks.workunit.client.0.smithi032.stdout:8/691: dread dc/d25/d28/d3c/f77 [4194304,4194304] 0 2024-06-03T01:21:13.347 INFO:tasks.workunit.client.0.smithi032.stdout:1/685: link d18/d3f/cb5 d18/d3f/d5f/d6b/cce 0 2024-06-03T01:21:13.352 INFO:tasks.workunit.client.0.smithi032.stdout:6/618: mkdir d8/d11/d20/d36/da5/dc8 0 2024-06-03T01:21:13.352 INFO:tasks.workunit.client.0.smithi032.stdout:6/619: chown d8/d11/d20/d2b/d39/d19/l81 1 1 2024-06-03T01:21:13.354 INFO:tasks.workunit.client.0.smithi032.stdout:8/692: truncate dc/d25/d28/fb8 688910 0 2024-06-03T01:21:13.355 INFO:tasks.workunit.client.0.smithi032.stdout:8/693: dread dc/d25/d28/d33/fb2 [0,4194304] 0 2024-06-03T01:21:13.355 INFO:tasks.workunit.client.0.smithi032.stdout:8/694: write dc/de/d1e/f61 [85966,66368] 0 2024-06-03T01:21:13.356 INFO:tasks.workunit.client.0.smithi032.stdout:1/686: mknod d18/d32/d39/ccf 0 2024-06-03T01:21:13.356 INFO:tasks.workunit.client.0.smithi032.stdout:1/687: creat d18/d27/fd0 x:0 0 0 2024-06-03T01:21:13.357 INFO:tasks.workunit.client.0.smithi032.stdout:6/620: symlink d8/d50/lc9 0 2024-06-03T01:21:13.359 INFO:tasks.workunit.client.0.smithi032.stdout:8/695: mkdir dc/de/d1e/d43/d9b/dd7 0 2024-06-03T01:21:13.359 INFO:tasks.workunit.client.0.smithi032.stdout:8/696: write dc/d25/d28/d32/f36 [398204,13527] 0 2024-06-03T01:21:13.359 INFO:tasks.workunit.client.0.smithi032.stdout:8/697: write dc/d25/d28/d58/f79 [3585777,17046] 0 2024-06-03T01:21:13.359 INFO:tasks.workunit.client.0.smithi032.stdout:8/698: chown dc/d25/d28/d3c/c85 879 1 2024-06-03T01:21:13.362 INFO:tasks.workunit.client.0.smithi032.stdout:6/621: getdents d8/d5f 0 2024-06-03T01:21:13.362 INFO:tasks.workunit.client.0.smithi032.stdout:6/622: write d8/d11/d20/d2b/d39/d19/f46 [5168384,76851] 0 2024-06-03T01:21:13.364 INFO:tasks.workunit.client.0.smithi032.stdout:8/699: link dc/de/l19 dc/d25/d28/d58/d8d/d94/d59/ld8 0 2024-06-03T01:21:13.364 INFO:tasks.workunit.client.0.smithi032.stdout:8/700: fdatasync dc/d25/d64/fa4 0 2024-06-03T01:21:13.364 INFO:tasks.workunit.client.0.smithi032.stdout:8/701: fdatasync dc/d14/fc1 0 2024-06-03T01:21:13.372 INFO:tasks.workunit.client.0.smithi032.stdout:6/623: rmdir d8/d11 39 2024-06-03T01:21:13.374 INFO:tasks.workunit.client.0.smithi032.stdout:8/702: mknod dc/d14/cd9 0 2024-06-03T01:21:13.374 INFO:tasks.workunit.client.0.smithi032.stdout:6/624: dread d8/d4c/fc0 [0,4194304] 0 2024-06-03T01:21:13.375 INFO:tasks.workunit.client.0.smithi032.stdout:6/625: rmdir d8/d11/d20/d3b 39 2024-06-03T01:21:13.376 INFO:tasks.workunit.client.0.smithi032.stdout:6/626: fsync f4 0 2024-06-03T01:21:13.413 INFO:tasks.workunit.client.0.smithi032.stdout:1/688: dwrite fa [0,4194304] 0 2024-06-03T01:21:13.413 INFO:tasks.workunit.client.0.smithi032.stdout:1/689: fdatasync d18/d3f/d5f/d6b/d78/d3d/d47/fb4 0 2024-06-03T01:21:13.414 INFO:tasks.workunit.client.0.smithi032.stdout:8/703: dwrite dc/de/fa8 [0,4194304] 0 2024-06-03T01:21:13.414 INFO:tasks.workunit.client.0.smithi032.stdout:6/627: dwrite d8/d11/f34 [0,4194304] 0 2024-06-03T01:21:13.414 INFO:tasks.workunit.client.0.smithi032.stdout:6/628: readlink d8/d11/d20/d21/l26 0 2024-06-03T01:21:13.414 INFO:tasks.workunit.client.0.smithi032.stdout:1/690: symlink d18/d3f/d5f/d7b/ld1 0 2024-06-03T01:21:13.415 INFO:tasks.workunit.client.0.smithi032.stdout:8/704: mkdir dc/de/d1e/d43/d9b/dd7/dda 0 2024-06-03T01:21:13.415 INFO:tasks.workunit.client.0.smithi032.stdout:8/705: chown dc/de/fa8 257356 1 2024-06-03T01:21:13.418 INFO:tasks.workunit.client.0.smithi032.stdout:6/629: getdents d8/d11/d20/d2b/d39 0 2024-06-03T01:21:13.421 INFO:tasks.workunit.client.0.smithi032.stdout:8/706: dread dc/de/f37 [0,4194304] 0 2024-06-03T01:21:13.421 INFO:tasks.workunit.client.0.smithi032.stdout:1/691: link f17 d18/d32/d3c/fd2 0 2024-06-03T01:21:13.422 INFO:tasks.workunit.client.0.smithi032.stdout:1/692: dread d18/d3f/d5f/d94/f97 [0,4194304] 0 2024-06-03T01:21:13.423 INFO:tasks.workunit.client.0.smithi032.stdout:1/693: truncate d18/f1a 84540 0 2024-06-03T01:21:13.432 INFO:tasks.workunit.client.0.smithi032.stdout:8/707: read dc/d25/f35 [890885,115239] 0 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/630: dwrite d8/d50/fac [0,4194304] 0 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/631: fdatasync d8/d11/d20/d2b/f3c 0 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/632: fdatasync d8/d11/d20/d3b/f80 0 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/633: write d8/d11/d20/d2b/d39/d19/f4d [2391552,126470] 0 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/634: dread - d8/d11/d20/d31/fbf zero size 2024-06-03T01:21:13.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/635: chown d8/d50/lb5 452534 1 2024-06-03T01:21:13.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/694: dwrite d18/f8e [0,4194304] 0 2024-06-03T01:21:13.444 INFO:tasks.workunit.client.0.smithi032.stdout:6/636: symlink d8/d4c/lca 0 2024-06-03T01:21:13.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/637: symlink d8/d11/d20/lcb 0 2024-06-03T01:21:13.453 INFO:tasks.workunit.client.0.smithi032.stdout:6/638: dread d8/fb [0,4194304] 0 2024-06-03T01:21:13.453 INFO:tasks.workunit.client.0.smithi032.stdout:8/708: dwrite fa [0,4194304] 0 2024-06-03T01:21:13.455 INFO:tasks.workunit.client.0.smithi032.stdout:6/639: rename d8/d11/d20/d7f/f78 to d8/d11/db2/fcc 0 2024-06-03T01:21:13.456 INFO:tasks.workunit.client.0.smithi032.stdout:1/695: dwrite d18/d2d/f50 [0,4194304] 0 2024-06-03T01:21:13.456 INFO:tasks.workunit.client.0.smithi032.stdout:1/696: truncate d18/d27/fb7 277206 0 2024-06-03T01:21:13.457 INFO:tasks.workunit.client.0.smithi032.stdout:1/697: rename fa to d18/d32/d39/d60/fd3 0 2024-06-03T01:21:13.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/698: readlink l4 0 2024-06-03T01:21:13.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/699: stat d18/d27/da7/cc3 0 2024-06-03T01:21:13.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/700: fsync d18/d3f/d5f/d6b/f73 0 2024-06-03T01:21:13.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/701: fdatasync d18/d3f/f53 0 2024-06-03T01:21:13.458 INFO:tasks.workunit.client.0.smithi032.stdout:1/702: symlink d18/d27/ld4 0 2024-06-03T01:21:13.460 INFO:tasks.workunit.client.0.smithi032.stdout:1/703: read d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [332992,23263] 0 2024-06-03T01:21:13.461 INFO:tasks.workunit.client.0.smithi032.stdout:8/709: write dc/f13 [3890441,93673] 0 2024-06-03T01:21:13.463 INFO:tasks.workunit.client.0.smithi032.stdout:6/640: write f4 [4735543,85643] 0 2024-06-03T01:21:13.464 INFO:tasks.workunit.client.0.smithi032.stdout:6/641: mknod d8/ccd 0 2024-06-03T01:21:13.466 INFO:tasks.workunit.client.0.smithi032.stdout:6/642: creat d8/d11/d20/d2b/d39/d7e/dc1/fce x:0 0 0 2024-06-03T01:21:13.466 INFO:tasks.workunit.client.0.smithi032.stdout:6/643: symlink d8/d11/d20/d31/d3f/lcf 0 2024-06-03T01:21:13.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/644: unlink d8/d11/d20/f8c 0 2024-06-03T01:21:13.471 INFO:tasks.workunit.client.0.smithi032.stdout:6/645: link d8/d11/d20/d7f/l9d d8/d4c/daa/ld0 0 2024-06-03T01:21:13.471 INFO:tasks.workunit.client.0.smithi032.stdout:6/646: write f3 [4756431,51638] 0 2024-06-03T01:21:13.474 INFO:tasks.workunit.client.0.smithi032.stdout:1/704: write d18/d3f/fb9 [5492442,37514] 0 2024-06-03T01:21:13.474 INFO:tasks.workunit.client.0.smithi032.stdout:1/705: fsync d18/d32/d39/fc5 0 2024-06-03T01:21:13.475 INFO:tasks.workunit.client.0.smithi032.stdout:6/647: dread d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:13.476 INFO:tasks.workunit.client.0.smithi032.stdout:1/706: creat d18/d3f/d5f/d6b/fd5 x:0 0 0 2024-06-03T01:21:13.476 INFO:tasks.workunit.client.0.smithi032.stdout:1/707: chown d18/faf 238 1 2024-06-03T01:21:13.477 INFO:tasks.workunit.client.0.smithi032.stdout:6/648: creat d8/d11/d20/d3b/fd1 x:0 0 0 2024-06-03T01:21:13.478 INFO:tasks.workunit.client.0.smithi032.stdout:1/708: dread d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [0,4194304] 0 2024-06-03T01:21:13.479 INFO:tasks.workunit.client.0.smithi032.stdout:1/709: truncate d18/d27/d2b/f7f 2326421 0 2024-06-03T01:21:13.479 INFO:tasks.workunit.client.0.smithi032.stdout:1/710: creat d18/d32/d39/d60/fd6 x:0 0 0 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:6/649: dread d8/d50/fac [0,4194304] 0 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:8/710: dwrite dc/de/f37 [0,4194304] 0 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:8/711: fsync dc/d25/f88 0 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:8/712: dread - dc/d25/d64/db5/fd3 zero size 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:6/650: write d8/d11/d20/d36/f38 [2139605,21264] 0 2024-06-03T01:21:13.491 INFO:tasks.workunit.client.0.smithi032.stdout:8/713: write dc/d25/d28/d58/f86 [1319916,233] 0 2024-06-03T01:21:13.493 INFO:tasks.workunit.client.0.smithi032.stdout:6/651: getdents d8/d11 0 2024-06-03T01:21:13.493 INFO:tasks.workunit.client.0.smithi032.stdout:6/652: chown d8/d11/d20/d2b/d39/f67 185 1 2024-06-03T01:21:13.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/714: dread dc/d25/d28/d58/d8d/d94/f6d [0,4194304] 0 2024-06-03T01:21:13.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/715: fsync dc/de/f37 0 2024-06-03T01:21:13.502 INFO:tasks.workunit.client.0.smithi032.stdout:1/711: dwrite d18/d3f/d5f/d6b/f81 [0,4194304] 0 2024-06-03T01:21:13.503 INFO:tasks.workunit.client.0.smithi032.stdout:1/712: dread d18/d27/fb7 [0,4194304] 0 2024-06-03T01:21:13.515 INFO:tasks.workunit.client.0.smithi032.stdout:1/713: dwrite d18/d3f/d5f/d6b/f73 [0,4194304] 0 2024-06-03T01:21:13.519 INFO:tasks.workunit.client.0.smithi032.stdout:8/716: rmdir dc/d25/d28/d58 39 2024-06-03T01:21:13.519 INFO:tasks.workunit.client.0.smithi032.stdout:8/717: fdatasync dc/de/d1e/d43/d9b/d76/fab 0 2024-06-03T01:21:13.519 INFO:tasks.workunit.client.0.smithi032.stdout:8/718: chown dc/d25/d28/d58/d8d/d94/d59/fd1 8884353 1 2024-06-03T01:21:13.519 INFO:tasks.workunit.client.0.smithi032.stdout:1/714: write d18/d32/d3c/f67 [439955,20103] 0 2024-06-03T01:21:13.522 INFO:tasks.workunit.client.0.smithi032.stdout:6/653: getdents d8/d11/d20/d2b/d39/d7e/dc1 0 2024-06-03T01:21:13.523 INFO:tasks.workunit.client.0.smithi032.stdout:8/719: mkdir dc/d25/ddb 0 2024-06-03T01:21:13.613 INFO:tasks.workunit.client.0.smithi032.stdout:1/715: truncate f17 2461279 0 2024-06-03T01:21:13.614 INFO:tasks.workunit.client.0.smithi032.stdout:8/720: rmdir dc/d25/d28/d58/d8d 39 2024-06-03T01:21:13.616 INFO:tasks.workunit.client.0.smithi032.stdout:6/654: symlink d8/d55/ld2 0 2024-06-03T01:21:13.616 INFO:tasks.workunit.client.0.smithi032.stdout:6/655: creat d8/d50/fd3 x:0 0 0 2024-06-03T01:21:13.626 INFO:tasks.workunit.client.0.smithi032.stdout:1/716: dread d18/d3f/d5f/d6b/d78/f5c [0,4194304] 0 2024-06-03T01:21:13.635 INFO:tasks.workunit.client.0.smithi032.stdout:8/721: dwrite f9 [0,4194304] 0 2024-06-03T01:21:13.653 INFO:tasks.workunit.client.0.smithi032.stdout:8/722: dwrite dc/d25/d28/d32/f36 [0,4194304] 0 2024-06-03T01:21:13.655 INFO:tasks.workunit.client.0.smithi032.stdout:8/723: creat dc/de/d1e/d43/d9b/d76/dae/fdc x:0 0 0 2024-06-03T01:21:13.656 INFO:tasks.workunit.client.0.smithi032.stdout:8/724: mknod dc/d14/d31/cdd 0 2024-06-03T01:21:13.656 INFO:tasks.workunit.client.0.smithi032.stdout:8/725: chown dc/de/d1e/d43/la9 11920011 1 2024-06-03T01:21:13.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/726: dwrite dc/de/d1e/f2e [0,4194304] 0 2024-06-03T01:21:13.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/727: fdatasync dc/d25/d28/d58/d8d/d94/d59/f9e 0 2024-06-03T01:21:13.682 INFO:tasks.workunit.client.0.smithi032.stdout:8/728: dread dc/d25/d64/fbd [0,4194304] 0 2024-06-03T01:21:13.682 INFO:tasks.workunit.client.0.smithi032.stdout:8/729: write dc/d14/f83 [5169347,51236] 0 2024-06-03T01:21:13.691 INFO:tasks.workunit.client.0.smithi032.stdout:5/838: sync 2024-06-03T01:21:13.691 INFO:tasks.workunit.client.0.smithi032.stdout:5/839: fsync da/dd/df/d10/dbc/d18/d1a/f77 0 2024-06-03T01:21:13.699 INFO:tasks.workunit.client.0.smithi032.stdout:8/730: dwrite dc/f15 [0,4194304] 0 2024-06-03T01:21:13.788 INFO:tasks.workunit.client.0.smithi032.stdout:5/840: dwrite da/f12 [0,4194304] 0 2024-06-03T01:21:13.791 INFO:tasks.workunit.client.0.smithi032.stdout:5/841: rename da/dd/df/d10/dbc/d18/d1a/f77 to da/dd/df/d10/f10b 0 2024-06-03T01:21:13.793 INFO:tasks.workunit.client.0.smithi032.stdout:5/842: mknod da/dd/df/d10/dbc/d18/d35/d44/c10c 0 2024-06-03T01:21:13.793 INFO:tasks.workunit.client.0.smithi032.stdout:5/843: chown da/dd/l7e 298801986 1 2024-06-03T01:21:13.797 INFO:tasks.workunit.client.0.smithi032.stdout:5/844: write da/dd/df/d10/dbc/d25/fef [683295,16129] 0 2024-06-03T01:21:13.811 INFO:tasks.workunit.client.0.smithi032.stdout:5/845: dwrite da/dab/ffb [0,4194304] 0 2024-06-03T01:21:13.811 INFO:tasks.workunit.client.0.smithi032.stdout:5/846: read da/f3e [185966,95436] 0 2024-06-03T01:21:13.812 INFO:tasks.workunit.client.0.smithi032.stdout:5/847: dread - da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fe3 zero size 2024-06-03T01:21:13.814 INFO:tasks.workunit.client.0.smithi032.stdout:5/848: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/f10d x:0 0 0 2024-06-03T01:21:13.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/849: dread da/dd/df/d10/dbc/d25/fa4 [0,4194304] 0 2024-06-03T01:21:13.827 INFO:tasks.workunit.client.0.smithi032.stdout:5/850: write da/dd/df/f5c [708493,66701] 0 2024-06-03T01:21:13.846 INFO:tasks.workunit.client.0.smithi032.stdout:5/851: dwrite da/dd/fdf [0,4194304] 0 2024-06-03T01:21:13.846 INFO:tasks.workunit.client.0.smithi032.stdout:5/852: chown da/dd/df/d10/dbc/d18/d35/d44/c4a 1172854748 1 2024-06-03T01:21:13.848 INFO:tasks.workunit.client.0.smithi032.stdout:5/853: rename da/cb to da/dd/df/d10/c10e 0 2024-06-03T01:21:13.849 INFO:tasks.workunit.client.0.smithi032.stdout:5/854: write da/dd/df/d10/dbc/d25/d74/f8e [899104,64638] 0 2024-06-03T01:21:13.856 INFO:tasks.workunit.client.0.smithi032.stdout:5/855: dread da/dd/df/fbf [0,4194304] 0 2024-06-03T01:21:13.857 INFO:tasks.workunit.client.0.smithi032.stdout:5/856: mkdir da/dd/df/d10/dd7/d10f 0 2024-06-03T01:21:13.859 INFO:tasks.workunit.client.0.smithi032.stdout:5/857: getdents da/dd/df/d10/dbc/d18/d35/d70 0 2024-06-03T01:21:13.859 INFO:tasks.workunit.client.0.smithi032.stdout:5/858: write da/dd/df/d10/dbc/d18/f1d [3990043,116096] 0 2024-06-03T01:21:13.859 INFO:tasks.workunit.client.0.smithi032.stdout:5/859: stat da/dd/df/f97 0 2024-06-03T01:21:13.859 INFO:tasks.workunit.client.0.smithi032.stdout:5/860: fdatasync da/dd/df/d10/f42 0 2024-06-03T01:21:13.859 INFO:tasks.workunit.client.0.smithi032.stdout:5/861: dread da/dd/df/d10/dbc/d18/d35/d44/fb7 [0,4194304] 0 2024-06-03T01:21:13.860 INFO:tasks.workunit.client.0.smithi032.stdout:5/862: creat da/dab/dcd/f110 x:0 0 0 2024-06-03T01:21:13.861 INFO:tasks.workunit.client.0.smithi032.stdout:5/863: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/f111 x:0 0 0 2024-06-03T01:21:13.864 INFO:tasks.workunit.client.0.smithi032.stdout:5/864: mknod da/dd/df/d10/dbc/c112 0 2024-06-03T01:21:13.873 INFO:tasks.workunit.client.0.smithi032.stdout:5/865: dread da/dd/df/f32 [0,4194304] 0 2024-06-03T01:21:13.875 INFO:tasks.workunit.client.0.smithi032.stdout:5/866: truncate da/f3e 1516928 0 2024-06-03T01:21:13.875 INFO:tasks.workunit.client.0.smithi032.stdout:5/867: write da/dab/f107 [284674,127378] 0 2024-06-03T01:21:13.880 INFO:tasks.workunit.client.0.smithi032.stdout:2/920: sync 2024-06-03T01:21:13.880 INFO:tasks.workunit.client.0.smithi032.stdout:7/689: sync 2024-06-03T01:21:13.885 INFO:tasks.workunit.client.0.smithi032.stdout:2/921: rename d5/da/c87 to d5/d38/d46/d53/d57/d6d/dce/d67/df5/d115/c137 0 2024-06-03T01:21:13.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/922: write d5/d38/d46/d53/d57/d6d/dce/fb1 [772208,83968] 0 2024-06-03T01:21:13.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/923: fsync d5/d38/d46/d53/d57/dac/de0/fe1 0 2024-06-03T01:21:13.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/924: dread - d5/d38/d46/d53/d57/dac/f128 zero size 2024-06-03T01:21:13.887 INFO:tasks.workunit.client.0.smithi032.stdout:2/925: mknod d5/d38/d46/d53/d57/dac/c138 0 2024-06-03T01:21:13.889 INFO:tasks.workunit.client.0.smithi032.stdout:2/926: write d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/fb2 [1214931,54905] 0 2024-06-03T01:21:13.890 INFO:tasks.workunit.client.0.smithi032.stdout:2/927: mknod d5/d38/d46/d53/d57/d6d/dce/d67/c139 0 2024-06-03T01:21:13.892 INFO:tasks.workunit.client.0.smithi032.stdout:5/868: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/f10d [0,4194304] 0 2024-06-03T01:21:13.912 INFO:tasks.workunit.client.0.smithi032.stdout:2/928: dwrite d5/da/d1a/d2a/d36/d3f/f63 [0,4194304] 0 2024-06-03T01:21:13.916 INFO:tasks.workunit.client.0.smithi032.stdout:5/869: dwrite da/dd/df/d10/dbc/d25/f66 [0,4194304] 0 2024-06-03T01:21:13.919 INFO:tasks.workunit.client.0.smithi032.stdout:5/870: symlink da/dd/df/d10/dbc/d18/d35/d44/dc3/l113 0 2024-06-03T01:21:13.919 INFO:tasks.workunit.client.0.smithi032.stdout:5/871: read da/dd/f68 [2438883,66124] 0 2024-06-03T01:21:13.919 INFO:tasks.workunit.client.0.smithi032.stdout:5/872: mkdir da/dd/de6/d9e/d114 0 2024-06-03T01:21:13.920 INFO:tasks.workunit.client.0.smithi032.stdout:5/873: symlink da/dd/de6/dc8/l115 0 2024-06-03T01:21:13.938 INFO:tasks.workunit.client.0.smithi032.stdout:2/929: dwrite d5/da/d1a/f41 [0,4194304] 0 2024-06-03T01:21:13.938 INFO:tasks.workunit.client.0.smithi032.stdout:2/930: write d5/d38/d46/ffd [357822,79925] 0 2024-06-03T01:21:13.938 INFO:tasks.workunit.client.0.smithi032.stdout:5/874: dwrite da/dd/df/d10/dbc/fa0 [0,4194304] 0 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/875: creat da/dd/df/d10/f116 x:0 0 0 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/876: truncate da/dd/df/d10/f116 734920 0 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/877: fdatasync f4 0 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/878: chown da/dd/df/d10/dbc/f8a 17 1 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/879: chown da/dd/df/d10/dbc/d18/d35/d44/fd9 3 1 2024-06-03T01:21:13.939 INFO:tasks.workunit.client.0.smithi032.stdout:2/931: link d5/f6b d5/d38/d46/df2/f13a 0 2024-06-03T01:21:13.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/932: unlink d5/da/d1a/d2a/d36/d3f/fd6 0 2024-06-03T01:21:13.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/933: read - d5/d38/d46/d53/d57/d6d/dce/f71 zero size 2024-06-03T01:21:13.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/934: readlink d5/d38/d46/d5c/d6c/l11a 0 2024-06-03T01:21:13.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/935: fsync d5/d38/d46/d53/d57/d6d/dce/d8f/fee 0 2024-06-03T01:21:13.943 INFO:tasks.workunit.client.0.smithi032.stdout:2/936: read - d5/d38/d46/d53/d57/dac/d10a/f131 zero size 2024-06-03T01:21:13.946 INFO:tasks.workunit.client.0.smithi032.stdout:2/937: rmdir d5/d38/d46/d53/d57/d6d/dce/d67 39 2024-06-03T01:21:13.950 INFO:tasks.workunit.client.0.smithi032.stdout:5/880: dread da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fbe [0,4194304] 0 2024-06-03T01:21:13.950 INFO:tasks.workunit.client.0.smithi032.stdout:5/881: creat da/dd/df/d10/f117 x:0 0 0 2024-06-03T01:21:13.950 INFO:tasks.workunit.client.0.smithi032.stdout:5/882: chown da/dd/df/d10/f4c 3 1 2024-06-03T01:21:13.955 INFO:tasks.workunit.client.0.smithi032.stdout:2/938: dread d5/da/fa6 [0,4194304] 0 2024-06-03T01:21:13.958 INFO:tasks.workunit.client.0.smithi032.stdout:5/883: dread da/dd/df/d10/dbc/d18/d1a/d30/f94 [0,4194304] 0 2024-06-03T01:21:13.960 INFO:tasks.workunit.client.0.smithi032.stdout:5/884: unlink da/l3b 0 2024-06-03T01:21:13.963 INFO:tasks.workunit.client.0.smithi032.stdout:2/939: dread d5/f35 [0,4194304] 0 2024-06-03T01:21:13.966 INFO:tasks.workunit.client.0.smithi032.stdout:2/940: dread d5/d38/d46/d53/d57/dac/de0/fe1 [0,4194304] 0 2024-06-03T01:21:13.967 INFO:tasks.workunit.client.0.smithi032.stdout:2/941: rename d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f130 to d5/da/d1a/d2a/d36/d3f/f13b 0 2024-06-03T01:21:13.970 INFO:tasks.workunit.client.0.smithi032.stdout:5/885: dread da/dd/df/fbf [4194304,4194304] 0 2024-06-03T01:21:13.970 INFO:tasks.workunit.client.0.smithi032.stdout:5/886: creat da/dab/f118 x:0 0 0 2024-06-03T01:21:13.970 INFO:tasks.workunit.client.0.smithi032.stdout:5/887: write da/dd/df/d10/dbc/d18/d1a/fb0 [4705902,56120] 0 2024-06-03T01:21:13.972 INFO:tasks.workunit.client.0.smithi032.stdout:5/888: mkdir da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/d119 0 2024-06-03T01:21:13.973 INFO:tasks.workunit.client.0.smithi032.stdout:5/889: rename da/f73 to da/dd/f11a 0 2024-06-03T01:21:13.973 INFO:tasks.workunit.client.0.smithi032.stdout:5/890: dread - da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fe3 zero size 2024-06-03T01:21:13.973 INFO:tasks.workunit.client.0.smithi032.stdout:5/891: fdatasync da/dd/df/d10/f117 0 2024-06-03T01:21:13.974 INFO:tasks.workunit.client.0.smithi032.stdout:5/892: creat da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/f11b x:0 0 0 2024-06-03T01:21:13.974 INFO:tasks.workunit.client.0.smithi032.stdout:5/893: fdatasync da/dd/df/d10/dbc/d18/d35/d44/fb7 0 2024-06-03T01:21:13.974 INFO:tasks.workunit.client.0.smithi032.stdout:5/894: chown da/dd/df/d10/dbc/lc4 1018 1 2024-06-03T01:21:13.975 INFO:tasks.workunit.client.0.smithi032.stdout:5/895: read da/dd/df/d10/dbc/d18/d35/d70/f95 [1436100,30289] 0 2024-06-03T01:21:13.975 INFO:tasks.workunit.client.0.smithi032.stdout:5/896: stat da/dd/df/d10/dbc/d18/d1a/d30/l6e 0 2024-06-03T01:21:13.975 INFO:tasks.workunit.client.0.smithi032.stdout:5/897: mkdir da/dd/de6/d9e/d11c 0 2024-06-03T01:21:13.976 INFO:tasks.workunit.client.0.smithi032.stdout:5/898: truncate da/dab/ffb 3692578 0 2024-06-03T01:21:13.980 INFO:tasks.workunit.client.0.smithi032.stdout:5/899: rename da/dd/df/d10/dbc/l2e to da/dd/df/d10/dbc/d18/l11d 0 2024-06-03T01:21:13.988 INFO:tasks.workunit.client.0.smithi032.stdout:2/942: dwrite d5/da/f59 [4194304,4194304] 0 2024-06-03T01:21:13.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/943: dread d5/d38/d46/f81 [0,4194304] 0 2024-06-03T01:21:13.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/944: read d5/d38/d46/f81 [1209835,99017] 0 2024-06-03T01:21:13.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/945: dread - d5/d38/d46/d53/d57/dac/f109 zero size 2024-06-03T01:21:13.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/946: fdatasync d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/f99 0 2024-06-03T01:21:13.996 INFO:tasks.workunit.client.0.smithi032.stdout:5/900: dwrite da/dd/df/f97 [0,4194304] 0 2024-06-03T01:21:13.999 INFO:tasks.workunit.client.0.smithi032.stdout:5/901: symlink da/dd/df/d10/l11e 0 2024-06-03T01:21:14.000 INFO:tasks.workunit.client.0.smithi032.stdout:2/947: mkdir d5/d38/d13c 0 2024-06-03T01:21:14.002 INFO:tasks.workunit.client.0.smithi032.stdout:5/902: symlink da/dd/df/d10/dbc/d18/d1a/l11f 0 2024-06-03T01:21:14.002 INFO:tasks.workunit.client.0.smithi032.stdout:5/903: truncate da/dd/df/d10/dbc/d25/fde 820456 0 2024-06-03T01:21:14.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/904: truncate da/dd/df/d10/dbc/d18/d1a/d30/f5b 1088661 0 2024-06-03T01:21:14.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/905: stat da/dd/df/d10/dbc/d25/d74/f88 0 2024-06-03T01:21:14.028 INFO:tasks.workunit.client.0.smithi032.stdout:2/948: dwrite d5/f23 [0,4194304] 0 2024-06-03T01:21:14.030 INFO:tasks.workunit.client.0.smithi032.stdout:5/906: dwrite da/dd/df/d10/dbc/d25/fa4 [0,4194304] 0 2024-06-03T01:21:14.030 INFO:tasks.workunit.client.0.smithi032.stdout:5/907: truncate da/dd/df/d10/dbc/d18/d35/d44/f72 4305485 0 2024-06-03T01:21:14.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/908: rmdir da/dd/df/d10/dbc/d18/d1a/d30 39 2024-06-03T01:21:14.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/909: readlink l5 0 2024-06-03T01:21:14.034 INFO:tasks.workunit.client.0.smithi032.stdout:5/910: unlink da/dd/df/d10/dbc/d25/c34 0 2024-06-03T01:21:14.036 INFO:tasks.workunit.client.0.smithi032.stdout:5/911: symlink da/dd/df/d10/dbc/d25/l120 0 2024-06-03T01:21:14.042 INFO:tasks.workunit.client.0.smithi032.stdout:2/949: write d5/d38/d46/d53/f5e [82800,17364] 0 2024-06-03T01:21:14.044 INFO:tasks.workunit.client.0.smithi032.stdout:2/950: rename d5/d38/d46/d53/d57/d6d/dce/f71 to d5/d38/db5/f13d 0 2024-06-03T01:21:14.053 INFO:tasks.workunit.client.0.smithi032.stdout:5/912: dwrite da/dd/df/f32 [0,4194304] 0 2024-06-03T01:21:14.056 INFO:tasks.workunit.client.0.smithi032.stdout:5/913: truncate da/dd/de6/f91 2839079 0 2024-06-03T01:21:14.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/951: dwrite d5/d38/d46/d53/d57/d6d/dce/f88 [0,4194304] 0 2024-06-03T01:21:14.070 INFO:tasks.workunit.client.0.smithi032.stdout:5/914: write da/dd/df/d10/dbc/d18/d35/d44/dc3/f10d [544149,105151] 0 2024-06-03T01:21:14.071 INFO:tasks.workunit.client.0.smithi032.stdout:2/952: mkdir d5/d13e 0 2024-06-03T01:21:14.071 INFO:tasks.workunit.client.0.smithi032.stdout:2/953: dread - d5/f12b zero size 2024-06-03T01:21:14.073 INFO:tasks.workunit.client.0.smithi032.stdout:2/954: creat d5/d38/d46/d53/d57/d6d/dce/d67/f13f x:0 0 0 2024-06-03T01:21:14.074 INFO:tasks.workunit.client.0.smithi032.stdout:2/955: mkdir d5/d38/db4/d140 0 2024-06-03T01:21:14.076 INFO:tasks.workunit.client.0.smithi032.stdout:2/956: truncate d5/da/f107 381642 0 2024-06-03T01:21:14.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/915: dread da/dab/ffb [0,4194304] 0 2024-06-03T01:21:14.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/916: write da/dd/df/d10/dbc/d18/d35/d70/fff [519330,83252] 0 2024-06-03T01:21:14.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/917: write da/dd/df/f103 [25001,100951] 0 2024-06-03T01:21:14.082 INFO:tasks.workunit.client.0.smithi032.stdout:2/957: mkdir d5/d38/d46/dbe/d141 0 2024-06-03T01:21:14.083 INFO:tasks.workunit.client.0.smithi032.stdout:5/918: creat da/dd/df/d10/dbc/d18/d35/d70/f121 x:0 0 0 2024-06-03T01:21:14.085 INFO:tasks.workunit.client.0.smithi032.stdout:5/919: creat da/dd/df/d10/dbc/d25/f122 x:0 0 0 2024-06-03T01:21:14.085 INFO:tasks.workunit.client.0.smithi032.stdout:5/920: chown da/dd/df/d10/dbc/d25/l45 57156286 1 2024-06-03T01:21:14.085 INFO:tasks.workunit.client.0.smithi032.stdout:5/921: creat da/f123 x:0 0 0 2024-06-03T01:21:14.089 INFO:tasks.workunit.client.0.smithi032.stdout:5/922: creat da/dd/df/d10/dbc/f124 x:0 0 0 2024-06-03T01:21:14.093 INFO:tasks.workunit.client.0.smithi032.stdout:5/923: unlink da/dd/de6/f91 0 2024-06-03T01:21:14.093 INFO:tasks.workunit.client.0.smithi032.stdout:5/924: chown da/dd/df/d10/dbc/d18/d35/d44/ldc 0 1 2024-06-03T01:21:14.096 INFO:tasks.workunit.client.0.smithi032.stdout:5/925: rename da/dd/df/d10/dbc/lc4 to da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/d119/l125 0 2024-06-03T01:21:14.098 INFO:tasks.workunit.client.0.smithi032.stdout:7/690: sync 2024-06-03T01:21:14.098 INFO:tasks.workunit.client.0.smithi032.stdout:4/686: sync 2024-06-03T01:21:14.098 INFO:tasks.workunit.client.0.smithi032.stdout:4/687: chown d4/db/da/d2d/d27/d53 2187 1 2024-06-03T01:21:14.099 INFO:tasks.workunit.client.0.smithi032.stdout:4/688: stat d4/db/da/fc3 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/763: sync 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:3/661: sync 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:3/662: creat de/d29/d2e/d31/d3c/d50/d62/fd9 x:0 0 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/764: write d0/da4/fe9 [824922,116208] 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/765: fsync d0/d11/d1d/d38/ff9 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:3/663: write de/d29/d2e/d31/f1c [2156697,125161] 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:1/717: sync 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:1/718: dread - d18/d27/fbe zero size 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:1/719: creat d18/d32/d39/d60/fd7 x:0 0 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:0/859: sync 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:0/860: read d1/da/d28/ffd [2367775,127812] 0 2024-06-03T01:21:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:6/656: sync 2024-06-03T01:21:14.101 INFO:tasks.workunit.client.0.smithi032.stdout:8/731: sync 2024-06-03T01:21:14.101 INFO:tasks.workunit.client.0.smithi032.stdout:8/732: truncate dc/d25/d28/d58/d8d/d94/d59/fc9 142135 0 2024-06-03T01:21:14.101 INFO:tasks.workunit.client.0.smithi032.stdout:5/926: mknod da/dab/de2/c126 0 2024-06-03T01:21:14.101 INFO:tasks.workunit.client.0.smithi032.stdout:2/958: dwrite d5/f72 [0,4194304] 0 2024-06-03T01:21:14.104 INFO:tasks.workunit.client.0.smithi032.stdout:7/691: mkdir dc/d12/d24/d33/daa/dda 0 2024-06-03T01:21:14.104 INFO:tasks.workunit.client.0.smithi032.stdout:7/692: chown dc/f14 489484 1 2024-06-03T01:21:14.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/720: mknod d18/d2d/d5e/cd8 0 2024-06-03T01:21:14.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/721: chown d18/d32/f4b 700510221 1 2024-06-03T01:21:14.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/722: write d18/d3f/f55 [4041661,77545] 0 2024-06-03T01:21:14.110 INFO:tasks.workunit.client.0.smithi032.stdout:1/723: write d18/d2d/d5e/f64 [4239058,118622] 0 2024-06-03T01:21:14.112 INFO:tasks.workunit.client.0.smithi032.stdout:0/861: mkdir d1/da/d10/d80/d9f/d117 0 2024-06-03T01:21:14.112 INFO:tasks.workunit.client.0.smithi032.stdout:0/862: chown d1/da/d28/d3b/d50/f5c 345 1 2024-06-03T01:21:14.114 INFO:tasks.workunit.client.0.smithi032.stdout:8/733: rename dc/f13 to dc/d93/fde 0 2024-06-03T01:21:14.115 INFO:tasks.workunit.client.0.smithi032.stdout:4/689: dread d4/db/ff [0,4194304] 0 2024-06-03T01:21:14.118 INFO:tasks.workunit.client.0.smithi032.stdout:6/657: dread d8/d11/d20/d3b/f80 [0,4194304] 0 2024-06-03T01:21:14.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/664: write de/d29/d2e/d31/d8d/dc3/fba [3259539,56333] 0 2024-06-03T01:21:14.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/665: read - de/d29/d2e/d31/d23/d53/dbf/fd3 zero size 2024-06-03T01:21:14.119 INFO:tasks.workunit.client.0.smithi032.stdout:5/927: symlink da/dd/de6/l127 0 2024-06-03T01:21:14.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/693: mkdir dc/d12/ddb 0 2024-06-03T01:21:14.124 INFO:tasks.workunit.client.0.smithi032.stdout:2/959: dwrite d5/d38/f3b [4194304,4194304] 0 2024-06-03T01:21:14.124 INFO:tasks.workunit.client.0.smithi032.stdout:2/960: fdatasync d5/d38/d46/f81 0 2024-06-03T01:21:14.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/724: symlink d18/d3f/d5f/d94/ld9 0 2024-06-03T01:21:14.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/725: getdents d18 0 2024-06-03T01:21:14.129 INFO:tasks.workunit.client.0.smithi032.stdout:1/726: dread d18/d3f/d5f/d94/f97 [0,4194304] 0 2024-06-03T01:21:14.135 INFO:tasks.workunit.client.0.smithi032.stdout:0/863: truncate d1/da/d28/d3b/d5d/f94 1149623 0 2024-06-03T01:21:14.135 INFO:tasks.workunit.client.0.smithi032.stdout:0/864: read - d1/da/d10/d53/f112 zero size 2024-06-03T01:21:14.138 INFO:tasks.workunit.client.0.smithi032.stdout:4/690: getdents d4/db/da/d5d/dc5 0 2024-06-03T01:21:14.138 INFO:tasks.workunit.client.0.smithi032.stdout:8/734: rename dc/de/fa8 to dc/d25/ddb/fdf 0 2024-06-03T01:21:14.138 INFO:tasks.workunit.client.0.smithi032.stdout:6/658: getdents d8/d4c 0 2024-06-03T01:21:14.141 INFO:tasks.workunit.client.0.smithi032.stdout:5/928: symlink da/dd/de6/d9e/d114/l128 0 2024-06-03T01:21:14.143 INFO:tasks.workunit.client.0.smithi032.stdout:5/929: dread da/dd/df/d10/f4c [0,4194304] 0 2024-06-03T01:21:14.145 INFO:tasks.workunit.client.0.smithi032.stdout:2/961: dwrite d5/da/d1a/f3e [0,4194304] 0 2024-06-03T01:21:14.146 INFO:tasks.workunit.client.0.smithi032.stdout:2/962: dread - d5/d38/d46/d53/d57/d6d/dce/d67/f13f zero size 2024-06-03T01:21:14.146 INFO:tasks.workunit.client.0.smithi032.stdout:2/963: stat d5/d38/d46/d53/d57/d6d/fbd 0 2024-06-03T01:21:14.149 INFO:tasks.workunit.client.0.smithi032.stdout:1/727: getdents d18/d3f/d5f/d6b/d78/d3d/d47/d6e 0 2024-06-03T01:21:14.149 INFO:tasks.workunit.client.0.smithi032.stdout:1/728: stat d18/d3f/d5f/d7b 0 2024-06-03T01:21:14.150 INFO:tasks.workunit.client.0.smithi032.stdout:0/865: stat d1/da/d28/d3b/d68/d83/lb7 0 2024-06-03T01:21:14.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/666: rename de/d29/d2e/d31/d24/l99 to de/d29/d2e/d31/d23/d53/d69/lda 0 2024-06-03T01:21:14.154 INFO:tasks.workunit.client.0.smithi032.stdout:8/735: creat dc/d14/d31/d3b/fe0 x:0 0 0 2024-06-03T01:21:14.154 INFO:tasks.workunit.client.0.smithi032.stdout:6/659: creat d8/d11/d20/d2b/d39/d7e/dc1/fd4 x:0 0 0 2024-06-03T01:21:14.155 INFO:tasks.workunit.client.0.smithi032.stdout:6/660: write d8/d11/d20/d31/d3f/f69 [446314,125384] 0 2024-06-03T01:21:14.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/964: creat d5/da/d1a/d2a/da8/db6/d134/f142 x:0 0 0 2024-06-03T01:21:14.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/965: truncate d5/d38/d46/d53/d57/d6d/fbd 608069 0 2024-06-03T01:21:14.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/966: creat d5/d38/d46/f143 x:0 0 0 2024-06-03T01:21:14.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/967: write d5/d38/d46/df2/f13a [4956848,41198] 0 2024-06-03T01:21:14.159 INFO:tasks.workunit.client.0.smithi032.stdout:4/691: dwrite d4/d57/d6c/fcf [0,4194304] 0 2024-06-03T01:21:14.159 INFO:tasks.workunit.client.0.smithi032.stdout:4/692: read d4/db/da/d37/f84 [2210001,88472] 0 2024-06-03T01:21:14.160 INFO:tasks.workunit.client.0.smithi032.stdout:4/693: read d4/db/da/f15 [246689,94951] 0 2024-06-03T01:21:14.162 INFO:tasks.workunit.client.0.smithi032.stdout:2/968: dread d5/da/d1a/f4b [0,4194304] 0 2024-06-03T01:21:14.165 INFO:tasks.workunit.client.0.smithi032.stdout:7/694: rmdir dc/d12/d22 39 2024-06-03T01:21:14.166 INFO:tasks.workunit.client.0.smithi032.stdout:7/695: chown dc/d12/d6a/d86/dc8 3618116 1 2024-06-03T01:21:14.167 INFO:tasks.workunit.client.0.smithi032.stdout:1/729: dread d18/d32/f4b [0,4194304] 0 2024-06-03T01:21:14.167 INFO:tasks.workunit.client.0.smithi032.stdout:5/930: dwrite f4 [8388608,4194304] 0 2024-06-03T01:21:14.167 INFO:tasks.workunit.client.0.smithi032.stdout:5/931: write da/dab/dcd/f110 [10277,72692] 0 2024-06-03T01:21:14.167 INFO:tasks.workunit.client.0.smithi032.stdout:0/866: truncate d1/da/d10/d29/f96 2171896 0 2024-06-03T01:21:14.177 INFO:tasks.workunit.client.0.smithi032.stdout:1/730: dread d18/d2d/f69 [0,4194304] 0 2024-06-03T01:21:14.180 INFO:tasks.workunit.client.0.smithi032.stdout:2/969: mknod d5/d38/d46/dbe/d141/c144 0 2024-06-03T01:21:14.180 INFO:tasks.workunit.client.0.smithi032.stdout:2/970: chown d5/lfe 16055 1 2024-06-03T01:21:14.181 INFO:tasks.workunit.client.0.smithi032.stdout:2/971: chown d5/d38/d46/d53/d57/dac/fe9 0 1 2024-06-03T01:21:14.182 INFO:tasks.workunit.client.0.smithi032.stdout:7/696: symlink dc/d12/d24/d33/ldc 0 2024-06-03T01:21:14.183 INFO:tasks.workunit.client.0.smithi032.stdout:5/932: mkdir da/dd/de6/d129 0 2024-06-03T01:21:14.184 INFO:tasks.workunit.client.0.smithi032.stdout:3/667: rename de/d29/d2e/d31/d78/f80 to de/fdb 0 2024-06-03T01:21:14.184 INFO:tasks.workunit.client.0.smithi032.stdout:3/668: chown de/d29/d2e/d31/f32 69985 1 2024-06-03T01:21:14.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/669: dread - de/d29/d2e/d31/d23/d53/d69/dca/fcc zero size 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/661: unlink d8/d11/d20/c5d 0 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/662: truncate d8/d11/d20/d2b/f93 709677 0 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/663: readlink d8/d55/l5c 0 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/664: chown d8/d11/d20/d2b/d39/d19/l1a 43712 1 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/665: chown d8/d11/d20/d2b 1 1 2024-06-03T01:21:14.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/666: chown d8/d11/d20/d36/da5/fa9 14805 1 2024-06-03T01:21:14.187 INFO:tasks.workunit.client.0.smithi032.stdout:1/731: mknod d18/d2d/cda 0 2024-06-03T01:21:14.187 INFO:tasks.workunit.client.0.smithi032.stdout:2/972: creat d5/d38/dfa/f145 x:0 0 0 2024-06-03T01:21:14.187 INFO:tasks.workunit.client.0.smithi032.stdout:5/933: truncate da/dd/df/d10/f99 1881655 0 2024-06-03T01:21:14.187 INFO:tasks.workunit.client.0.smithi032.stdout:5/934: truncate da/dab/dcd/f110 522822 0 2024-06-03T01:21:14.189 INFO:tasks.workunit.client.0.smithi032.stdout:4/694: rename d4/d57/d6c/fc9 to d4/db/da/d2d/d27/d8c/db5/fef 0 2024-06-03T01:21:14.189 INFO:tasks.workunit.client.0.smithi032.stdout:4/695: fdatasync d4/d57/d6c/fcf 0 2024-06-03T01:21:14.189 INFO:tasks.workunit.client.0.smithi032.stdout:3/670: mknod de/d29/d2e/cdc 0 2024-06-03T01:21:14.189 INFO:tasks.workunit.client.0.smithi032.stdout:3/671: dread - de/d29/d2e/d31/d3c/d50/d66/fc1 zero size 2024-06-03T01:21:14.190 INFO:tasks.workunit.client.0.smithi032.stdout:1/732: mkdir d18/d27/da7/ddb 0 2024-06-03T01:21:14.191 INFO:tasks.workunit.client.0.smithi032.stdout:5/935: mknod da/dd/de6/d129/c12a 0 2024-06-03T01:21:14.191 INFO:tasks.workunit.client.0.smithi032.stdout:5/936: creat da/dd/de6/f12b x:0 0 0 2024-06-03T01:21:14.192 INFO:tasks.workunit.client.0.smithi032.stdout:6/667: rename d8/d11/d20/d36 to d8/dd5 0 2024-06-03T01:21:14.192 INFO:tasks.workunit.client.0.smithi032.stdout:6/668: fsync d8/d55/f86 0 2024-06-03T01:21:14.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/696: mkdir d4/db/da/d37/df0 0 2024-06-03T01:21:14.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/697: write d4/db/fa0 [4496703,23830] 0 2024-06-03T01:21:14.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/698: fdatasync d4/d78/fbe 0 2024-06-03T01:21:14.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/699: truncate d4/d7f/fad 1511521 0 2024-06-03T01:21:14.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/937: truncate da/f2a 2749077 0 2024-06-03T01:21:14.195 INFO:tasks.workunit.client.0.smithi032.stdout:3/672: rename de/d29/d2e/d31/d23/d53/d79/ld4 to de/d29/d2e/d31/d3c/d50/d56/dc7/ldd 0 2024-06-03T01:21:14.196 INFO:tasks.workunit.client.0.smithi032.stdout:6/669: symlink d8/dd5/da5/ld6 0 2024-06-03T01:21:14.196 INFO:tasks.workunit.client.0.smithi032.stdout:4/700: rmdir d4/d7f 39 2024-06-03T01:21:14.196 INFO:tasks.workunit.client.0.smithi032.stdout:4/701: chown d4/d57 1397 1 2024-06-03T01:21:14.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/938: mknod da/dd/df/d10/dbc/d25/d74/c12c 0 2024-06-03T01:21:14.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/939: dread - da/f123 zero size 2024-06-03T01:21:14.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/940: readlink da/dd/df/d10/dbc/d18/d35/d44/dc3/l67 0 2024-06-03T01:21:14.198 INFO:tasks.workunit.client.0.smithi032.stdout:1/733: rename d18/d32/d39/d60/fd7 to d18/d3f/d5f/d6b/d78/d3d/d6a/fdc 0 2024-06-03T01:21:14.199 INFO:tasks.workunit.client.0.smithi032.stdout:6/670: truncate d8/d11/d20/d21/f27 3082137 0 2024-06-03T01:21:14.199 INFO:tasks.workunit.client.0.smithi032.stdout:6/671: chown d8/d4c 491 1 2024-06-03T01:21:14.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/702: mkdir d4/db/da/d1d/d6b/df1 0 2024-06-03T01:21:14.200 INFO:tasks.workunit.client.0.smithi032.stdout:5/941: mkdir da/dd/d12d 0 2024-06-03T01:21:14.201 INFO:tasks.workunit.client.0.smithi032.stdout:5/942: chown da/dd/df/d10/dbc/d18/d35/d44/dc3/l6a 57462531 1 2024-06-03T01:21:14.201 INFO:tasks.workunit.client.0.smithi032.stdout:1/734: creat d18/d32/fdd x:0 0 0 2024-06-03T01:21:14.202 INFO:tasks.workunit.client.0.smithi032.stdout:6/672: link d8/f6f d8/d11/d20/d3b/d66/fd7 0 2024-06-03T01:21:14.204 INFO:tasks.workunit.client.0.smithi032.stdout:4/703: rename l2 to d4/db/da/lf2 0 2024-06-03T01:21:14.204 INFO:tasks.workunit.client.0.smithi032.stdout:4/704: write d4/fcc [798677,27885] 0 2024-06-03T01:21:14.205 INFO:tasks.workunit.client.0.smithi032.stdout:1/735: creat d18/d3f/d5f/d6b/d78/d3d/d47/d6e/fde x:0 0 0 2024-06-03T01:21:14.206 INFO:tasks.workunit.client.0.smithi032.stdout:6/673: link d8/d11/d20/d3b/d66/f94 d8/d5f/fd8 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:4/705: creat d4/db/da/d5d/ff3 x:0 0 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:1/736: link d18/d32/fdd d18/d32/fdf 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:1/737: creat d18/d2d/d5e/fe0 x:0 0 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/674: mknod d8/d11/d20/d2b/d39/d7e/cd9 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/675: stat d8/dd5/f64 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/676: readlink d8/d11/d20/d2b/d39/d19/l98 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/677: truncate d8/d11/d20/d2b/d39/d19/f40 4829849 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:4/706: getdents d4/db/da/d5d/d8e 0 2024-06-03T01:21:14.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/678: getdents d8/d11/d20/d31/d3f 0 2024-06-03T01:21:14.212 INFO:tasks.workunit.client.0.smithi032.stdout:6/679: rename d8/d11/d20/d2b/d39/d7e/dc1 to d8/d50/dda 0 2024-06-03T01:21:14.212 INFO:tasks.workunit.client.0.smithi032.stdout:6/680: chown d8/d11/d20/d21/d9a/cbc 644571 1 2024-06-03T01:21:14.212 INFO:tasks.workunit.client.0.smithi032.stdout:6/681: truncate d8/d11/d20/f9f 4742773 0 2024-06-03T01:21:14.213 INFO:tasks.workunit.client.0.smithi032.stdout:6/682: symlink d8/d55/ldb 0 2024-06-03T01:21:14.218 INFO:tasks.workunit.client.0.smithi032.stdout:6/683: write d8/d11/d20/d31/f45 [1741190,19925] 0 2024-06-03T01:21:14.220 INFO:tasks.workunit.client.0.smithi032.stdout:6/684: rmdir d8/d11/d20/d89 0 2024-06-03T01:21:14.221 INFO:tasks.workunit.client.0.smithi032.stdout:6/685: getdents d8/d11/d20/d3b 0 2024-06-03T01:21:14.221 INFO:tasks.workunit.client.0.smithi032.stdout:6/686: chown d8/d11/d20/d21/d9a 30846 1 2024-06-03T01:21:14.222 INFO:tasks.workunit.client.0.smithi032.stdout:6/687: link d8/f6f d8/d11/d20/d21/d9a/fdc 0 2024-06-03T01:21:14.222 INFO:tasks.workunit.client.0.smithi032.stdout:6/688: readlink d8/d5f/l72 0 2024-06-03T01:21:14.236 INFO:tasks.workunit.client.0.smithi032.stdout:6/689: unlink d8/d11/d20/d2b/d39/d19/c1c 0 2024-06-03T01:21:14.248 INFO:tasks.workunit.client.0.smithi032.stdout:0/867: write d1/da/d10/d29/f96 [1410575,37796] 0 2024-06-03T01:21:14.250 INFO:tasks.workunit.client.0.smithi032.stdout:2/973: dwrite d5/d38/d46/d53/d57/d6d/dce/d67/f106 [0,4194304] 0 2024-06-03T01:21:14.250 INFO:tasks.workunit.client.0.smithi032.stdout:2/974: dread - d5/d38/d46/d5c/d6c/d89/fdd zero size 2024-06-03T01:21:14.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/690: fdatasync d8/d11/d20/d31/d3f/f69 0 2024-06-03T01:21:14.256 INFO:tasks.workunit.client.0.smithi032.stdout:4/707: dwrite d4/db/da/d5d/d8e/fdc [0,4194304] 0 2024-06-03T01:21:14.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/708: getdents d4/db/da/d37/d90/da1 0 2024-06-03T01:21:14.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/709: readlink d4/db/l62 0 2024-06-03T01:21:14.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/710: creat d4/db/da/d5d/ff4 x:0 0 0 2024-06-03T01:21:14.257 INFO:tasks.workunit.client.0.smithi032.stdout:2/975: unlink d5/d38/d46/d53/da9/c129 0 2024-06-03T01:21:14.266 INFO:tasks.workunit.client.0.smithi032.stdout:4/711: creat d4/db/da/d37/dab/ff5 x:0 0 0 2024-06-03T01:21:14.267 INFO:tasks.workunit.client.0.smithi032.stdout:6/691: write d8/d11/f34 [1530537,86709] 0 2024-06-03T01:21:14.268 INFO:tasks.workunit.client.0.smithi032.stdout:3/673: dwrite de/d29/d2e/d31/d23/f6e [0,4194304] 0 2024-06-03T01:21:14.268 INFO:tasks.workunit.client.0.smithi032.stdout:3/674: read - de/d29/d2e/d31/f1e zero size 2024-06-03T01:21:14.268 INFO:tasks.workunit.client.0.smithi032.stdout:3/675: chown de/d29/f3f 1 1 2024-06-03T01:21:14.268 INFO:tasks.workunit.client.0.smithi032.stdout:3/676: write de/d29/f65 [156310,73758] 0 2024-06-03T01:21:14.269 INFO:tasks.workunit.client.0.smithi032.stdout:4/712: creat d4/db/da/d5d/d8e/ff6 x:0 0 0 2024-06-03T01:21:14.271 INFO:tasks.workunit.client.0.smithi032.stdout:1/738: write d18/d3f/d5f/d6b/f81 [2937088,1632] 0 2024-06-03T01:21:14.272 INFO:tasks.workunit.client.0.smithi032.stdout:3/677: link de/d29/d2e/d31/d23/cce de/d29/d2e/d31/d24/daf/cde 0 2024-06-03T01:21:14.274 INFO:tasks.workunit.client.0.smithi032.stdout:4/713: mkdir d4/db/da/d37/dab/df7 0 2024-06-03T01:21:14.274 INFO:tasks.workunit.client.0.smithi032.stdout:7/697: dwrite dc/f2b [0,4194304] 0 2024-06-03T01:21:14.274 INFO:tasks.workunit.client.0.smithi032.stdout:7/698: readlink dc/d12/d2a/d2f/d63/lb3 0 2024-06-03T01:21:14.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/976: dwrite d5/d38/d46/d53/d57/dac/f54 [0,4194304] 0 2024-06-03T01:21:14.283 INFO:tasks.workunit.client.0.smithi032.stdout:6/692: write d8/d55/f86 [323916,94562] 0 2024-06-03T01:21:14.283 INFO:tasks.workunit.client.0.smithi032.stdout:0/868: dwrite d1/da/d28/ffd [4194304,4194304] 0 2024-06-03T01:21:14.285 INFO:tasks.workunit.client.0.smithi032.stdout:6/693: write d8/d11/d20/d2b/d39/d7e/fa3 [4559277,90244] 0 2024-06-03T01:21:14.287 INFO:tasks.workunit.client.0.smithi032.stdout:5/943: dwrite da/dd/df/d10/f42 [0,4194304] 0 2024-06-03T01:21:14.292 INFO:tasks.workunit.client.0.smithi032.stdout:3/678: rmdir de/d29/d2e/d31/d23/d53/d69 39 2024-06-03T01:21:14.298 INFO:tasks.workunit.client.0.smithi032.stdout:0/869: creat d1/da/d10/f118 x:0 0 0 2024-06-03T01:21:14.299 INFO:tasks.workunit.client.0.smithi032.stdout:6/694: symlink d8/d11/d20/d21/ldd 0 2024-06-03T01:21:14.301 INFO:tasks.workunit.client.0.smithi032.stdout:5/944: getdents da/dd/df/d10/dbc/d18/d1a/d30 0 2024-06-03T01:21:14.303 INFO:tasks.workunit.client.0.smithi032.stdout:3/679: mknod de/d29/d2e/d31/d78/cdf 0 2024-06-03T01:21:14.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/699: rename dc/d12/d1b to dc/d12/d22/d42/d61/ddd 0 2024-06-03T01:21:14.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/700: dread - dc/d12/fa1 zero size 2024-06-03T01:21:14.307 INFO:tasks.workunit.client.0.smithi032.stdout:6/695: truncate d8/d11/d20/d2b/d39/f67 2838943 0 2024-06-03T01:21:14.310 INFO:tasks.workunit.client.0.smithi032.stdout:3/680: mknod de/d29/d2e/d31/ce0 0 2024-06-03T01:21:14.311 INFO:tasks.workunit.client.0.smithi032.stdout:7/701: mknod dc/d12/d2a/d2f/cde 0 2024-06-03T01:21:14.312 INFO:tasks.workunit.client.0.smithi032.stdout:7/702: write dc/d12/d24/f38 [685139,99082] 0 2024-06-03T01:21:14.312 INFO:tasks.workunit.client.0.smithi032.stdout:7/703: fsync dc/f54 0 2024-06-03T01:21:14.314 INFO:tasks.workunit.client.0.smithi032.stdout:6/696: unlink d8/d11/d20/d3b/d66/cb6 0 2024-06-03T01:21:14.315 INFO:tasks.workunit.client.0.smithi032.stdout:5/945: link da/dd/df/cc2 da/dd/de6/d9e/d11c/c12e 0 2024-06-03T01:21:14.316 INFO:tasks.workunit.client.0.smithi032.stdout:2/977: dwrite d5/da/d2e/f3d [0,4194304] 0 2024-06-03T01:21:14.317 INFO:tasks.workunit.client.0.smithi032.stdout:7/704: symlink dc/d12/ldf 0 2024-06-03T01:21:14.319 INFO:tasks.workunit.client.0.smithi032.stdout:1/739: dwrite d18/d3f/d5f/d6b/d78/f83 [0,4194304] 0 2024-06-03T01:21:14.319 INFO:tasks.workunit.client.0.smithi032.stdout:1/740: readlink d18/d32/l4e 0 2024-06-03T01:21:14.320 INFO:tasks.workunit.client.0.smithi032.stdout:4/714: dwrite d4/db/da/d5d/d8e/ff6 [0,4194304] 0 2024-06-03T01:21:14.320 INFO:tasks.workunit.client.0.smithi032.stdout:0/870: dwrite d1/da/d10/d29/fd0 [0,4194304] 0 2024-06-03T01:21:14.320 INFO:tasks.workunit.client.0.smithi032.stdout:0/871: fsync d1/da/d10/d53/fa4 0 2024-06-03T01:21:14.320 INFO:tasks.workunit.client.0.smithi032.stdout:0/872: creat d1/da/d28/d3b/d68/d82/f119 x:0 0 0 2024-06-03T01:21:14.320 INFO:tasks.workunit.client.0.smithi032.stdout:6/697: symlink d8/d11/d20/d31/lde 0 2024-06-03T01:21:14.321 INFO:tasks.workunit.client.0.smithi032.stdout:3/681: write de/d29/d2e/d31/d78/f86 [1587547,70966] 0 2024-06-03T01:21:14.321 INFO:tasks.workunit.client.0.smithi032.stdout:3/682: truncate de/d63/f7a 452480 0 2024-06-03T01:21:14.323 INFO:tasks.workunit.client.0.smithi032.stdout:5/946: dread da/f3e [0,4194304] 0 2024-06-03T01:21:14.327 INFO:tasks.workunit.client.0.smithi032.stdout:1/741: write d18/d3f/d5f/d6b/f73 [3013303,15412] 0 2024-06-03T01:21:14.329 INFO:tasks.workunit.client.0.smithi032.stdout:2/978: rmdir d5/d38/d46/d53/d57/d6d/dce/d8f 39 2024-06-03T01:21:14.330 INFO:tasks.workunit.client.0.smithi032.stdout:7/705: mkdir dc/d12/d2a/d2f/d63/de0 0 2024-06-03T01:21:14.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/715: symlink d4/db/da/d2d/de1/lf8 0 2024-06-03T01:21:14.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/979: read d5/d38/d46/d53/fa7 [284762,90650] 0 2024-06-03T01:21:14.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/980: fsync d5/d38/d46/d53/d57/d6d/dce/d67/d104/d83/fb2 0 2024-06-03T01:21:14.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/981: fdatasync d5/da/f2c 0 2024-06-03T01:21:14.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/873: link d1/da/d28/d3b/d68/fa3 d1/da/d10/d53/d105/f11a 0 2024-06-03T01:21:14.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/874: dread - d1/da/d28/d3b/dce/f106 zero size 2024-06-03T01:21:14.346 INFO:tasks.workunit.client.0.smithi032.stdout:5/947: mkdir da/dd/df/d10/dbc/d12f 0 2024-06-03T01:21:14.346 INFO:tasks.workunit.client.0.smithi032.stdout:5/948: truncate da/dd/df/d10/dbc/d18/db9/fe4 310430 0 2024-06-03T01:21:14.357 INFO:tasks.workunit.client.0.smithi032.stdout:7/706: dread dc/d12/f25 [0,4194304] 0 2024-06-03T01:21:14.358 INFO:tasks.workunit.client.0.smithi032.stdout:7/707: dread - dc/d12/d2a/d2f/d76/fc6 zero size 2024-06-03T01:21:14.358 INFO:tasks.workunit.client.0.smithi032.stdout:6/698: dwrite d8/dd5/dae/fb0 [0,4194304] 0 2024-06-03T01:21:14.367 INFO:tasks.workunit.client.0.smithi032.stdout:4/716: creat d4/db/da/d1d/d6b/df1/ff9 x:0 0 0 2024-06-03T01:21:14.372 INFO:tasks.workunit.client.0.smithi032.stdout:0/875: rename d1/da/db2/d103 to d1/da/dd2/d11b 0 2024-06-03T01:21:14.374 INFO:tasks.workunit.client.0.smithi032.stdout:5/949: mknod da/dd/de6/deb/c130 0 2024-06-03T01:21:14.375 INFO:tasks.workunit.client.0.smithi032.stdout:1/742: rmdir d18/d32/d39/d60 39 2024-06-03T01:21:14.375 INFO:tasks.workunit.client.0.smithi032.stdout:1/743: chown d18/d32/d39/cbb 1129556 1 2024-06-03T01:21:14.380 INFO:tasks.workunit.client.0.smithi032.stdout:7/708: creat dc/d12/d24/d33/daa/fe1 x:0 0 0 2024-06-03T01:21:14.390 INFO:tasks.workunit.client.0.smithi032.stdout:6/699: rename d8/d11/d20/ca1 to d8/cdf 0 2024-06-03T01:21:14.391 INFO:tasks.workunit.client.0.smithi032.stdout:0/876: link d1/d46/l41 d1/da/dd2/d10f/l11c 0 2024-06-03T01:21:14.393 INFO:tasks.workunit.client.0.smithi032.stdout:5/950: mkdir da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/d119/d131 0 2024-06-03T01:21:14.394 INFO:tasks.workunit.client.0.smithi032.stdout:7/709: symlink dc/d12/d2a/d2f/d63/le2 0 2024-06-03T01:21:14.395 INFO:tasks.workunit.client.0.smithi032.stdout:6/700: symlink d8/dd5/da5/le0 0 2024-06-03T01:21:14.396 INFO:tasks.workunit.client.0.smithi032.stdout:1/744: dwrite d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 [0,4194304] 0 2024-06-03T01:21:14.397 INFO:tasks.workunit.client.0.smithi032.stdout:4/717: rename d4/db/da/d2d/d27/c98 to d4/d57/cfa 0 2024-06-03T01:21:14.398 INFO:tasks.workunit.client.0.smithi032.stdout:7/710: creat dc/d12/d2a/d2f/d76/fe3 x:0 0 0 2024-06-03T01:21:14.398 INFO:tasks.workunit.client.0.smithi032.stdout:7/711: dread - dc/d12/d6a/f72 zero size 2024-06-03T01:21:14.404 INFO:tasks.workunit.client.0.smithi032.stdout:5/951: read da/dd/fdf [325178,104830] 0 2024-06-03T01:21:14.405 INFO:tasks.workunit.client.0.smithi032.stdout:1/745: mkdir d18/d3f/d5f/da6/de1 0 2024-06-03T01:21:14.407 INFO:tasks.workunit.client.0.smithi032.stdout:0/877: rename d1/da/d28/ffd to d1/da/d28/d3b/d5d/da9/f11d 0 2024-06-03T01:21:14.408 INFO:tasks.workunit.client.0.smithi032.stdout:4/718: creat d4/d7f/ffb x:0 0 0 2024-06-03T01:21:14.409 INFO:tasks.workunit.client.0.smithi032.stdout:7/712: unlink dc/d12/c48 0 2024-06-03T01:21:14.410 INFO:tasks.workunit.client.0.smithi032.stdout:7/713: readlink dc/d12/d2a/d81/l96 0 2024-06-03T01:21:14.410 INFO:tasks.workunit.client.0.smithi032.stdout:6/701: getdents d8/d11/d20/d2b 0 2024-06-03T01:21:14.414 INFO:tasks.workunit.client.0.smithi032.stdout:5/952: creat da/dd/d12d/f132 x:0 0 0 2024-06-03T01:21:14.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/746: mknod d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/ce2 0 2024-06-03T01:21:14.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/747: write d18/d3f/d5f/d6b/d78/fb8 [1018885,74890] 0 2024-06-03T01:21:14.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/748: write d18/d3f/fb9 [6975447,101207] 0 2024-06-03T01:21:14.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/749: fdatasync d18/d3f/d5f/d6b/d78/d3d/d47/d6e/fde 0 2024-06-03T01:21:14.417 INFO:tasks.workunit.client.0.smithi032.stdout:0/878: rename d1/da/d28/d3b/d68/d83/lb7 to d1/d46/dc3/l11e 0 2024-06-03T01:21:14.417 INFO:tasks.workunit.client.0.smithi032.stdout:0/879: stat d1/da/d10/d80/d9f/dac 0 2024-06-03T01:21:14.418 INFO:tasks.workunit.client.0.smithi032.stdout:5/953: rmdir da/dd/df/d10/dbc/d18/d35/d44 39 2024-06-03T01:21:14.418 INFO:tasks.workunit.client.0.smithi032.stdout:5/954: chown da/f3e 36 1 2024-06-03T01:21:14.418 INFO:tasks.workunit.client.0.smithi032.stdout:5/955: chown da/dab/lfe 549193956 1 2024-06-03T01:21:14.420 INFO:tasks.workunit.client.0.smithi032.stdout:0/880: symlink d1/da/d28/d3b/d68/ddd/l11f 0 2024-06-03T01:21:14.420 INFO:tasks.workunit.client.0.smithi032.stdout:0/881: creat d1/da/d28/d3b/d68/f120 x:0 0 0 2024-06-03T01:21:14.421 INFO:tasks.workunit.client.0.smithi032.stdout:0/882: fdatasync d1/da/d10/d80/d9f/dac/fbd 0 2024-06-03T01:21:14.421 INFO:tasks.workunit.client.0.smithi032.stdout:0/883: chown d1/da/d10/d100 5 1 2024-06-03T01:21:14.423 INFO:tasks.workunit.client.0.smithi032.stdout:7/714: dread dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:14.423 INFO:tasks.workunit.client.0.smithi032.stdout:5/956: link da/f17 da/dd/de6/f133 0 2024-06-03T01:21:14.424 INFO:tasks.workunit.client.0.smithi032.stdout:4/719: dwrite d4/d7f/f92 [0,4194304] 0 2024-06-03T01:21:14.429 INFO:tasks.workunit.client.0.smithi032.stdout:0/884: rename d1/da/d28/fe4 to d1/da/d28/f121 0 2024-06-03T01:21:14.431 INFO:tasks.workunit.client.0.smithi032.stdout:6/702: write d8/d11/d20/d3b/f80 [2205805,2258] 0 2024-06-03T01:21:14.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/715: rename dc/d12/d24/d7b/c19 to dc/d12/d2a/d81/ce4 0 2024-06-03T01:21:14.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/750: dwrite f17 [0,4194304] 0 2024-06-03T01:21:14.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/751: chown d18/d32/l56 221 1 2024-06-03T01:21:14.433 INFO:tasks.workunit.client.0.smithi032.stdout:5/957: creat da/dd/de6/dc8/f134 x:0 0 0 2024-06-03T01:21:14.434 INFO:tasks.workunit.client.0.smithi032.stdout:1/752: dread d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b [0,4194304] 0 2024-06-03T01:21:14.441 INFO:tasks.workunit.client.0.smithi032.stdout:4/720: dwrite d4/db/da/f2f [0,4194304] 0 2024-06-03T01:21:14.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/753: dwrite d18/d3f/d5f/d6b/d78/d3d/f8c [4194304,4194304] 0 2024-06-03T01:21:14.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/754: readlink d18/d27/ld4 0 2024-06-03T01:21:14.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/755: dread - d18/d27/fd0 zero size 2024-06-03T01:21:14.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/756: chown d18/d32/d39/d60/ca0 49478 1 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/885: creat d1/da/d28/d3b/d5d/d8d/d9c/dc2/f122 x:0 0 0 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/886: readlink d1/da/d10/d80/l9d 0 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/887: creat d1/da/d28/f123 x:0 0 0 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/888: write d1/da/d28/f56 [292357,77100] 0 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/889: creat d1/da/d28/d3b/d68/d82/f124 x:0 0 0 2024-06-03T01:21:14.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/890: dread - d1/da/d28/d3b/d68/d82/f124 zero size 2024-06-03T01:21:14.460 INFO:tasks.workunit.client.0.smithi032.stdout:5/958: mkdir da/dd/df/d10/dd7/d135 0 2024-06-03T01:21:14.460 INFO:tasks.workunit.client.0.smithi032.stdout:5/959: dread - da/dd/df/d10/dbc/d18/d1a/d30/fd4 zero size 2024-06-03T01:21:14.462 INFO:tasks.workunit.client.0.smithi032.stdout:4/721: truncate d4/d57/d6c/f6e 869230 0 2024-06-03T01:21:14.466 INFO:tasks.workunit.client.0.smithi032.stdout:0/891: stat d1/l34 0 2024-06-03T01:21:14.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/703: rename d8/d11/f17 to d8/d50/dda/fe1 0 2024-06-03T01:21:14.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/704: chown d8/d11/d20/d3b/d66/fc3 506 1 2024-06-03T01:21:14.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/757: dwrite d18/d3f/d5f/d6b/d78/d3d/d47/d6e/fde [0,4194304] 0 2024-06-03T01:21:14.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/758: fdatasync d18/f80 0 2024-06-03T01:21:14.470 INFO:tasks.workunit.client.0.smithi032.stdout:5/960: mkdir da/dd/de6/d7a/d136 0 2024-06-03T01:21:14.470 INFO:tasks.workunit.client.0.smithi032.stdout:0/892: getdents d1/da/d28/d3b/d5d/d8d/dbe/dc4 0 2024-06-03T01:21:14.470 INFO:tasks.workunit.client.0.smithi032.stdout:0/893: chown d1/da/d10/f19 33701 1 2024-06-03T01:21:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi044 to 2241M 2024-06-03T01:21:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:14 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/705: read d8/d50/dda/fe1 [2110039,107166] 0 2024-06-03T01:21:14.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/706: write d8/f6f [3151783,28749] 0 2024-06-03T01:21:14.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/707: write d8/d11/d20/d21/d9a/fba [981061,95487] 0 2024-06-03T01:21:14.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/708: chown l7 147431 1 2024-06-03T01:21:14.478 INFO:tasks.workunit.client.0.smithi032.stdout:5/961: truncate da/dd/df/d10/dbc/d18/d35/d44/dc3/da5/fcb 3174435 0 2024-06-03T01:21:14.479 INFO:tasks.workunit.client.0.smithi032.stdout:0/894: mknod d1/da/dd2/d11b/c125 0 2024-06-03T01:21:14.482 INFO:tasks.workunit.client.0.smithi032.stdout:6/709: chown d8/d11/d20/c23 86892463 1 2024-06-03T01:21:14.482 INFO:tasks.workunit.client.0.smithi032.stdout:6/710: dread - d8/d50/fd3 zero size 2024-06-03T01:21:14.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/962: symlink da/dd/df/d10/dbc/d18/d35/d44/dc3/l137 0 2024-06-03T01:21:14.486 INFO:tasks.workunit.client.0.smithi032.stdout:7/716: rename dc/d12/d6a/d86/d88/d8b/db8 to dc/d12/d24/d33/de5 0 2024-06-03T01:21:14.486 INFO:tasks.workunit.client.0.smithi032.stdout:7/717: read - dc/d12/d22/fbf zero size 2024-06-03T01:21:14.490 INFO:tasks.workunit.client.0.smithi032.stdout:6/711: dread d8/d5f/fd8 [0,4194304] 0 2024-06-03T01:21:14.490 INFO:tasks.workunit.client.0.smithi032.stdout:6/712: getdents d8/d11/d20/d2b 0 2024-06-03T01:21:14.490 INFO:tasks.workunit.client.0.smithi032.stdout:6/713: chown d8/d11/d20/d2b/d39/d7e/fa3 165042 1 2024-06-03T01:21:14.491 INFO:tasks.workunit.client.0.smithi032.stdout:5/963: symlink da/dd/df/d10/dbc/d18/d35/d70/l138 0 2024-06-03T01:21:14.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/722: rename d4/db/da/d37/f40 to d4/d57/ffc 0 2024-06-03T01:21:14.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/723: read d4/db/da/d37/f3b [253674,52891] 0 2024-06-03T01:21:14.494 INFO:tasks.workunit.client.0.smithi032.stdout:6/714: symlink d8/d4c/daa/le2 0 2024-06-03T01:21:14.494 INFO:tasks.workunit.client.0.smithi032.stdout:6/715: write f3 [456653,127262] 0 2024-06-03T01:21:14.494 INFO:tasks.workunit.client.0.smithi032.stdout:6/716: write d8/d11/d20/d2b/f4a [1310727,31985] 0 2024-06-03T01:21:14.495 INFO:tasks.workunit.client.0.smithi032.stdout:5/964: getdents da/dd/df/d10/dbc/d18/d35 0 2024-06-03T01:21:14.495 INFO:tasks.workunit.client.0.smithi032.stdout:5/965: write da/dd/df/d10/f10a [1021328,13446] 0 2024-06-03T01:21:14.496 INFO:tasks.workunit.client.0.smithi032.stdout:5/966: chown da/dd/df/d10/dbc/d25/d74/f8e 643 1 2024-06-03T01:21:14.497 INFO:tasks.workunit.client.0.smithi032.stdout:1/759: dwrite d18/d3f/d5f/d7b/da8/fc2 [0,4194304] 0 2024-06-03T01:21:14.497 INFO:tasks.workunit.client.0.smithi032.stdout:1/760: chown d18/d2d/c63 637035 1 2024-06-03T01:21:14.497 INFO:tasks.workunit.client.0.smithi032.stdout:1/761: chown d18/d3f/d5f/d6b/d78/d3d/d47/d6e 748 1 2024-06-03T01:21:14.500 INFO:tasks.workunit.client.0.smithi032.stdout:0/895: rename d1/da/d28/d3b/d68/db5/le2 to d1/da/d28/d3b/d5d/d8d/d9c/dc2/l126 0 2024-06-03T01:21:14.501 INFO:tasks.workunit.client.0.smithi032.stdout:6/717: creat d8/d50/dda/fe3 x:0 0 0 2024-06-03T01:21:14.501 INFO:tasks.workunit.client.0.smithi032.stdout:6/718: readlink d8/d11/d20/d3b/lb8 0 2024-06-03T01:21:14.506 INFO:tasks.workunit.client.0.smithi032.stdout:1/762: mkdir d18/d3f/d5f/d6b/d78/de3 0 2024-06-03T01:21:14.507 INFO:tasks.workunit.client.0.smithi032.stdout:6/719: unlink d8/d4c/daa/ld0 0 2024-06-03T01:21:14.508 INFO:tasks.workunit.client.0.smithi032.stdout:5/967: getdents da/dd/df/d10/dbc/d25 0 2024-06-03T01:21:14.512 INFO:tasks.workunit.client.0.smithi032.stdout:6/720: symlink d8/d11/d20/le4 0 2024-06-03T01:21:14.522 INFO:tasks.workunit.client.0.smithi032.stdout:4/724: dwrite d4/db/da/fc3 [0,4194304] 0 2024-06-03T01:21:14.523 INFO:tasks.workunit.client.0.smithi032.stdout:7/718: dwrite dc/d12/d24/f6c [0,4194304] 0 2024-06-03T01:21:14.523 INFO:tasks.workunit.client.0.smithi032.stdout:0/896: dwrite d1/da/f4d [0,4194304] 0 2024-06-03T01:21:14.523 INFO:tasks.workunit.client.0.smithi032.stdout:0/897: fdatasync d1/da/d28/d3b/d68/d82/f124 0 2024-06-03T01:21:14.526 INFO:tasks.workunit.client.0.smithi032.stdout:7/719: dread dc/d12/d2a/d2f/d76/f80 [0,4194304] 0 2024-06-03T01:21:14.528 INFO:tasks.workunit.client.0.smithi032.stdout:4/725: rmdir d4/db/da/d5d 39 2024-06-03T01:21:14.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/720: dread dc/d12/d24/d33/de5/fce [0,4194304] 0 2024-06-03T01:21:14.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/721: chown dc/d12/d24/d7b/l5a 892849719 1 2024-06-03T01:21:14.533 INFO:tasks.workunit.client.0.smithi032.stdout:7/722: unlink dc/d12/d6a/d86/d88/f94 0 2024-06-03T01:21:14.534 INFO:tasks.workunit.client.0.smithi032.stdout:7/723: write dc/d12/d2a/d2f/fd2 [58053,119207] 0 2024-06-03T01:21:14.535 INFO:tasks.workunit.client.0.smithi032.stdout:4/726: rename d4/db/da/d2d/d27/d8c/db5/db3 to d4/db/da/d37/dab/dec/dfd 0 2024-06-03T01:21:14.536 INFO:tasks.workunit.client.0.smithi032.stdout:7/724: mknod dc/d12/d24/d33/ce6 0 2024-06-03T01:21:14.538 INFO:tasks.workunit.client.0.smithi032.stdout:4/727: mknod d4/db/da/d37/dab/cfe 0 2024-06-03T01:21:14.538 INFO:tasks.workunit.client.0.smithi032.stdout:4/728: readlink d4/db/da/d37/l45 0 2024-06-03T01:21:14.539 INFO:tasks.workunit.client.0.smithi032.stdout:7/725: creat dc/d12/d2a/d2f/d63/fe7 x:0 0 0 2024-06-03T01:21:14.539 INFO:tasks.workunit.client.0.smithi032.stdout:7/726: dread - dc/d12/d2a/d2f/d63/fe7 zero size 2024-06-03T01:21:14.540 INFO:tasks.workunit.client.0.smithi032.stdout:7/727: link dc/d12/d22/f29 dc/d12/d22/fe8 0 2024-06-03T01:21:14.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/728: creat dc/d12/d24/d33/de5/fe9 x:0 0 0 2024-06-03T01:21:14.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/729: dread dc/d12/d22/d42/f50 [0,4194304] 0 2024-06-03T01:21:14.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/730: chown dc/d12/d2a/d2f/d76/f7d 14649 1 2024-06-03T01:21:14.555 INFO:tasks.workunit.client.0.smithi032.stdout:1/763: dwrite d18/d3f/d5f/d6b/d78/d3d/d6a/f99 [0,4194304] 0 2024-06-03T01:21:14.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/968: dwrite da/dd/df/d10/dbc/d18/d35/d70/f84 [0,4194304] 0 2024-06-03T01:21:14.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/969: chown da/dd/de6/d7a/d136 99 1 2024-06-03T01:21:14.558 INFO:tasks.workunit.client.0.smithi032.stdout:4/729: dwrite d4/db/da/d37/dab/dec/dfd/fc4 [0,4194304] 0 2024-06-03T01:21:14.558 INFO:tasks.workunit.client.0.smithi032.stdout:4/730: write d4/db/da/d2d/f9f [1290415,54049] 0 2024-06-03T01:21:14.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/731: write d4/db/da/f3d [81010,70563] 0 2024-06-03T01:21:14.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/732: write d4/db/da/d1d/f21 [3225950,118836] 0 2024-06-03T01:21:14.561 INFO:tasks.workunit.client.0.smithi032.stdout:6/721: dwrite d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:14.561 INFO:tasks.workunit.client.0.smithi032.stdout:1/764: dread d18/d3f/d5f/d6b/d78/d3d/d6a/fa4 [0,4194304] 0 2024-06-03T01:21:14.561 INFO:tasks.workunit.client.0.smithi032.stdout:1/765: chown d18/d3f/d5f/d6b/d78/ccd 3534763 1 2024-06-03T01:21:14.561 INFO:tasks.workunit.client.0.smithi032.stdout:5/970: dread da/dd/df/d10/dbc/d18/d35/f98 [0,4194304] 0 2024-06-03T01:21:14.565 INFO:tasks.workunit.client.0.smithi032.stdout:7/731: dwrite dc/d12/f83 [0,4194304] 0 2024-06-03T01:21:14.572 INFO:tasks.workunit.client.0.smithi032.stdout:1/766: truncate d18/d2d/f50 2335871 0 2024-06-03T01:21:14.575 INFO:tasks.workunit.client.0.smithi032.stdout:5/971: symlink da/dd/de6/d7a/l139 0 2024-06-03T01:21:14.575 INFO:tasks.workunit.client.0.smithi032.stdout:5/972: stat da/dd/df/d10/dbc/d18/d35/d44/dc3/da8/ff2 0 2024-06-03T01:21:14.575 INFO:tasks.workunit.client.0.smithi032.stdout:5/973: write da/dd/df/d10/dbc/f124 [539356,46735] 0 2024-06-03T01:21:14.575 INFO:tasks.workunit.client.0.smithi032.stdout:5/974: write da/dd/df/d10/dbc/d25/f8b [5147571,96505] 0 2024-06-03T01:21:14.576 INFO:tasks.workunit.client.0.smithi032.stdout:7/732: link dc/d12/d24/d7b/c6f dc/d12/d2a/d2f/d76/cea 0 2024-06-03T01:21:14.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/975: stat da/dd/df/d10/c10e 0 2024-06-03T01:21:14.579 INFO:tasks.workunit.client.0.smithi032.stdout:1/767: getdents d18/d2d/d5e 0 2024-06-03T01:21:14.580 INFO:tasks.workunit.client.0.smithi032.stdout:1/768: truncate d18/d3f/d5f/d94/f97 699551 0 2024-06-03T01:21:14.581 INFO:tasks.workunit.client.0.smithi032.stdout:4/733: dwrite d4/f7 [0,4194304] 0 2024-06-03T01:21:14.583 INFO:tasks.workunit.client.0.smithi032.stdout:1/769: symlink d18/d32/d3c/le4 0 2024-06-03T01:21:14.585 INFO:tasks.workunit.client.0.smithi032.stdout:4/734: creat d4/db/da/d37/dab/dec/dfd/fff x:0 0 0 2024-06-03T01:21:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:14.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi044 to 2241M 2024-06-03T01:21:14.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:14 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:14.588 INFO:tasks.workunit.client.0.smithi032.stdout:4/735: rmdir d4/db/da/d37/d90/da1 39 2024-06-03T01:21:14.588 INFO:tasks.workunit.client.0.smithi032.stdout:4/736: stat d4/db/da/d2d/d27/d8c/db5/fbd 0 2024-06-03T01:21:14.591 INFO:tasks.workunit.client.0.smithi032.stdout:6/722: dwrite d8/d55/f6a [0,4194304] 0 2024-06-03T01:21:14.595 INFO:tasks.workunit.client.0.smithi032.stdout:6/723: creat d8/d11/d20/d2b/dc7/fe5 x:0 0 0 2024-06-03T01:21:14.602 INFO:tasks.workunit.client.0.smithi032.stdout:4/737: dread d4/db/da/f80 [0,4194304] 0 2024-06-03T01:21:14.602 INFO:tasks.workunit.client.0.smithi032.stdout:4/738: fdatasync d4/d57/d6c/fcf 0 2024-06-03T01:21:14.615 INFO:tasks.workunit.client.0.smithi032.stdout:4/739: dread d4/db/da/f59 [0,4194304] 0 2024-06-03T01:21:14.615 INFO:tasks.workunit.client.0.smithi032.stdout:7/733: dwrite dc/d12/d2a/d2f/d76/f7d [0,4194304] 0 2024-06-03T01:21:14.616 INFO:tasks.workunit.client.0.smithi032.stdout:4/740: mknod d4/d5c/c100 0 2024-06-03T01:21:14.617 INFO:tasks.workunit.client.0.smithi032.stdout:5/976: dwrite da/dd/df/d10/dbc/d18/d35/d44/dc3/fcf [0,4194304] 0 2024-06-03T01:21:14.617 INFO:tasks.workunit.client.0.smithi032.stdout:5/977: stat da/dd/f68 0 2024-06-03T01:21:14.617 INFO:tasks.workunit.client.0.smithi032.stdout:1/770: dwrite d18/d27/fbe [0,4194304] 0 2024-06-03T01:21:14.617 INFO:tasks.workunit.client.0.smithi032.stdout:1/771: fdatasync d18/d27/fb7 0 2024-06-03T01:21:14.617 INFO:tasks.workunit.client.0.smithi032.stdout:6/724: dwrite d8/d50/dda/fe3 [0,4194304] 0 2024-06-03T01:21:14.618 INFO:tasks.workunit.client.0.smithi032.stdout:7/734: fsync dc/d12/d2a/d2f/fd2 0 2024-06-03T01:21:14.618 INFO:tasks.workunit.client.0.smithi032.stdout:1/772: dread d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b [0,4194304] 0 2024-06-03T01:21:14.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/978: creat da/dd/df/d10/dbc/d18/d35/d44/f13a x:0 0 0 2024-06-03T01:21:14.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/979: chown da/c37 106227414 1 2024-06-03T01:21:14.628 INFO:tasks.workunit.client.0.smithi032.stdout:6/725: rename d8/d11/d20/d2b/d39/d19/f4d to d8/d11/d20/d31/fe6 0 2024-06-03T01:21:14.628 INFO:tasks.workunit.client.0.smithi032.stdout:6/726: fsync d8/d11/d20/d2b/d39/d19/f46 0 2024-06-03T01:21:14.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/741: symlink d4/db/da/d1d/l101 0 2024-06-03T01:21:14.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/735: creat dc/d12/d6a/d86/dc8/feb x:0 0 0 2024-06-03T01:21:14.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/736: write dc/d12/d2a/d2f/fd2 [745940,61991] 0 2024-06-03T01:21:14.631 INFO:tasks.workunit.client.0.smithi032.stdout:1/773: symlink d18/d2d/d5e/le5 0 2024-06-03T01:21:14.632 INFO:tasks.workunit.client.0.smithi032.stdout:1/774: dread d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b [0,4194304] 0 2024-06-03T01:21:14.633 INFO:tasks.workunit.client.0.smithi032.stdout:5/980: creat da/dd/de6/d9e/d114/f13b x:0 0 0 2024-06-03T01:21:14.634 INFO:tasks.workunit.client.0.smithi032.stdout:6/727: rename d8/lc to d8/d11/d20/d7f/le7 0 2024-06-03T01:21:14.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/728: rename d8/d11/d20/d31/fbf to d8/d11/fe8 0 2024-06-03T01:21:14.639 INFO:tasks.workunit.client.0.smithi032.stdout:6/729: creat d8/d55/fe9 x:0 0 0 2024-06-03T01:21:14.639 INFO:tasks.workunit.client.0.smithi032.stdout:6/730: write d8/d11/d20/d2b/d39/d19/f46 [6040632,27246] 0 2024-06-03T01:21:14.640 INFO:tasks.workunit.client.0.smithi032.stdout:6/731: fsync d8/d11/d20/d3b/fd1 0 2024-06-03T01:21:14.644 INFO:tasks.workunit.client.0.smithi032.stdout:6/732: dread d8/d11/d20/d2b/f4a [0,4194304] 0 2024-06-03T01:21:14.644 INFO:tasks.workunit.client.0.smithi032.stdout:6/733: read d8/d4c/fc0 [28004,29595] 0 2024-06-03T01:21:14.647 INFO:tasks.workunit.client.0.smithi032.stdout:6/734: rename d8/d11/d20/d31/f96 to d8/d11/d20/d31/d3f/fea 0 2024-06-03T01:21:14.647 INFO:tasks.workunit.client.0.smithi032.stdout:6/735: write d8/d55/fe9 [21245,109666] 0 2024-06-03T01:21:14.647 INFO:tasks.workunit.client.0.smithi032.stdout:7/737: write dc/d12/d22/d42/d61/ddd/f21 [517,23880] 0 2024-06-03T01:21:14.647 INFO:tasks.workunit.client.0.smithi032.stdout:7/738: chown dc/c70 923264 1 2024-06-03T01:21:14.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/981: dread da/dd/df/d10/dbc/d18/d1a/fb0 [0,4194304] 0 2024-06-03T01:21:14.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/982: chown da/dd/df/d10/dbc/d18/d35/d70/fff 0 1 2024-06-03T01:21:14.648 INFO:tasks.workunit.client.0.smithi032.stdout:6/736: mknod d8/dd5/ceb 0 2024-06-03T01:21:14.649 INFO:tasks.workunit.client.0.smithi032.stdout:6/737: truncate d8/d55/fe9 390039 0 2024-06-03T01:21:14.650 INFO:tasks.workunit.client.0.smithi032.stdout:7/739: symlink dc/d12/d2a/d2f/lec 0 2024-06-03T01:21:14.652 INFO:tasks.workunit.client.0.smithi032.stdout:5/983: unlink da/dd/df/d10/dbc/d18/d35/d44/dc3/l6a 0 2024-06-03T01:21:14.653 INFO:tasks.workunit.client.0.smithi032.stdout:1/775: dwrite d18/d32/d39/d60/fd3 [0,4194304] 0 2024-06-03T01:21:14.653 INFO:tasks.workunit.client.0.smithi032.stdout:6/738: mknod d8/d4c/cec 0 2024-06-03T01:21:14.653 INFO:tasks.workunit.client.0.smithi032.stdout:6/739: chown d8/d11/d20/d2b/d39/d19/la7 868409191 1 2024-06-03T01:21:14.654 INFO:tasks.workunit.client.0.smithi032.stdout:4/742: dwrite d4/db/da/d37/f7b [0,4194304] 0 2024-06-03T01:21:14.654 INFO:tasks.workunit.client.0.smithi032.stdout:4/743: chown d4/db/da/d5d/d8e/ff6 1 1 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:7/740: truncate dc/d12/d24/d33/f66 3606601 0 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/984: creat da/dd/df/d10/dd7/f13c x:0 0 0 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:7/741: write dc/d12/d6a/fa7 [20956,73019] 0 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/985: creat da/dd/de6/d9e/d114/f13d x:0 0 0 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/986: write da/dd/df/d10/f116 [1124592,102461] 0 2024-06-03T01:21:14.655 INFO:tasks.workunit.client.0.smithi032.stdout:5/987: dread - da/dd/de6/dc8/f134 zero size 2024-06-03T01:21:14.657 INFO:tasks.workunit.client.0.smithi032.stdout:1/776: mkdir d18/dc0/de6 0 2024-06-03T01:21:14.657 INFO:tasks.workunit.client.0.smithi032.stdout:6/740: readlink d8/d55/l5e 0 2024-06-03T01:21:14.661 INFO:tasks.workunit.client.0.smithi032.stdout:7/742: getdents dc/d12/d2a/d2f/d9a 0 2024-06-03T01:21:14.661 INFO:tasks.workunit.client.0.smithi032.stdout:7/743: stat dc/d12/d2a/d2f/c6d 0 2024-06-03T01:21:14.663 INFO:tasks.workunit.client.0.smithi032.stdout:6/741: creat d8/d11/d20/d2b/d39/fed x:0 0 0 2024-06-03T01:21:14.665 INFO:tasks.workunit.client.0.smithi032.stdout:1/777: symlink d18/d3f/d5f/d98/le7 0 2024-06-03T01:21:14.667 INFO:tasks.workunit.client.0.smithi032.stdout:6/742: rmdir d8/d4c 39 2024-06-03T01:21:14.669 INFO:tasks.workunit.client.0.smithi032.stdout:1/778: symlink d18/d3f/le8 0 2024-06-03T01:21:14.671 INFO:tasks.workunit.client.0.smithi032.stdout:6/743: mknod d8/d11/d20/d3b/d66/cee 0 2024-06-03T01:21:14.673 INFO:tasks.workunit.client.0.smithi032.stdout:7/744: dread dc/d12/d22/d42/d61/ddd/f5b [0,4194304] 0 2024-06-03T01:21:14.674 INFO:tasks.workunit.client.0.smithi032.stdout:7/745: write dc/d12/d6a/d86/d89/fca [781862,82251] 0 2024-06-03T01:21:14.674 INFO:tasks.workunit.client.0.smithi032.stdout:5/988: dwrite da/f101 [0,4194304] 0 2024-06-03T01:21:14.680 INFO:tasks.workunit.client.0.smithi032.stdout:4/744: dwrite d4/db/da/f59 [0,4194304] 0 2024-06-03T01:21:14.682 INFO:tasks.workunit.client.0.smithi032.stdout:5/989: link da/dd/df/d10/dbc/d18/d1a/d30/f5a da/dd/df/d10/dbc/d18/d35/d44/f13e 0 2024-06-03T01:21:14.682 INFO:tasks.workunit.client.0.smithi032.stdout:5/990: chown da/dab/dcd/cdd 31385 1 2024-06-03T01:21:14.694 INFO:tasks.workunit.client.0.smithi032.stdout:1/779: dwrite d18/f80 [0,4194304] 0 2024-06-03T01:21:14.701 INFO:tasks.workunit.client.0.smithi032.stdout:7/746: dwrite dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:14.702 INFO:tasks.workunit.client.0.smithi032.stdout:7/747: write dc/d12/d2a/d2f/d76/fc6 [868987,19902] 0 2024-06-03T01:21:14.702 INFO:tasks.workunit.client.0.smithi032.stdout:7/748: chown dc/d12/d24/d33/daa/dda 8 1 2024-06-03T01:21:14.702 INFO:tasks.workunit.client.0.smithi032.stdout:6/744: dwrite d8/d11/d20/d2b/d39/fed [0,4194304] 0 2024-06-03T01:21:14.704 INFO:tasks.workunit.client.0.smithi032.stdout:1/780: rmdir d18/d3f 39 2024-06-03T01:21:14.705 INFO:tasks.workunit.client.0.smithi032.stdout:4/745: dwrite d4/db/da/d37/f7b [0,4194304] 0 2024-06-03T01:21:14.705 INFO:tasks.workunit.client.0.smithi032.stdout:4/746: write d4/db/da/f3f [961449,60601] 0 2024-06-03T01:21:14.714 INFO:tasks.workunit.client.0.smithi032.stdout:6/745: creat d8/dd5/fef x:0 0 0 2024-06-03T01:21:14.715 INFO:tasks.workunit.client.0.smithi032.stdout:1/781: rename d18/d3f/d5f/d6b/d78/d3d/d6a/f99 to d18/d3f/d5f/d6b/d78/d3d/d6a/fe9 0 2024-06-03T01:21:14.715 INFO:tasks.workunit.client.0.smithi032.stdout:1/782: fsync d18/d2d/d5e/f64 0 2024-06-03T01:21:14.715 INFO:tasks.workunit.client.0.smithi032.stdout:1/783: dread - d18/d32/d3c/fb0 zero size 2024-06-03T01:21:14.717 INFO:tasks.workunit.client.0.smithi032.stdout:1/784: unlink d18/d3f/d5f/d6b/d78/f45 0 2024-06-03T01:21:14.718 INFO:tasks.workunit.client.0.smithi032.stdout:4/747: dread d4/db/da/d5d/d8e/ff6 [0,4194304] 0 2024-06-03T01:21:14.718 INFO:tasks.workunit.client.0.smithi032.stdout:4/748: readlink d4/db/da/d1d/l101 0 2024-06-03T01:21:14.719 INFO:tasks.workunit.client.0.smithi032.stdout:1/785: link d18/d27/d89/cca d18/d3f/d5f/d6b/d78/d3d/d47/cea 0 2024-06-03T01:21:14.719 INFO:tasks.workunit.client.0.smithi032.stdout:1/786: write d18/d3f/d5f/d94/f97 [706042,121883] 0 2024-06-03T01:21:14.720 INFO:tasks.workunit.client.0.smithi032.stdout:4/749: mkdir d4/db/da/d37/dab/dec/dfd/d102 0 2024-06-03T01:21:14.721 INFO:tasks.workunit.client.0.smithi032.stdout:6/746: dread d8/d50/fac [0,4194304] 0 2024-06-03T01:21:14.721 INFO:tasks.workunit.client.0.smithi032.stdout:6/747: readlink d8/d11/d20/d31/l33 0 2024-06-03T01:21:14.721 INFO:tasks.workunit.client.0.smithi032.stdout:6/748: stat d8/d11/d20/d21/fa2 0 2024-06-03T01:21:14.721 INFO:tasks.workunit.client.0.smithi032.stdout:6/749: creat d8/d11/d20/d2b/dc7/ff0 x:0 0 0 2024-06-03T01:21:14.721 INFO:tasks.workunit.client.0.smithi032.stdout:1/787: getdents d18/d3f/d5f/d98 0 2024-06-03T01:21:14.724 INFO:tasks.workunit.client.0.smithi032.stdout:7/749: dwrite dc/f2b [4194304,4194304] 0 2024-06-03T01:21:14.724 INFO:tasks.workunit.client.0.smithi032.stdout:4/750: creat d4/db/da/d37/dab/dec/dfd/ddd/f103 x:0 0 0 2024-06-03T01:21:14.728 INFO:tasks.workunit.client.0.smithi032.stdout:6/750: truncate d8/d5f/fd8 2031350 0 2024-06-03T01:21:14.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/751: dread d8/d11/d20/d2b/f93 [0,4194304] 0 2024-06-03T01:21:14.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/752: write d8/d11/d20/d31/d3f/fea [990916,82866] 0 2024-06-03T01:21:14.751 INFO:tasks.workunit.client.0.smithi032.stdout:4/751: dwrite d4/f6 [0,4194304] 0 2024-06-03T01:21:14.751 INFO:tasks.workunit.client.0.smithi032.stdout:7/750: dwrite dc/f2d [4194304,4194304] 0 2024-06-03T01:21:14.753 INFO:tasks.workunit.client.0.smithi032.stdout:7/751: dread dc/d12/d24/d33/de5/fce [0,4194304] 0 2024-06-03T01:21:14.758 INFO:tasks.workunit.client.0.smithi032.stdout:7/752: truncate dc/f3d 2773371 0 2024-06-03T01:21:14.760 INFO:tasks.workunit.client.0.smithi032.stdout:4/752: getdents d4/d78 0 2024-06-03T01:21:14.765 INFO:tasks.workunit.client.0.smithi032.stdout:4/753: unlink d4/db/da/d2d/d27/d8c/fb4 0 2024-06-03T01:21:14.765 INFO:tasks.workunit.client.0.smithi032.stdout:4/754: dread - d4/db/da/d37/dab/fb8 zero size 2024-06-03T01:21:14.767 INFO:tasks.workunit.client.0.smithi032.stdout:4/755: symlink d4/db/da/d2d/d27/d53/l104 0 2024-06-03T01:21:14.767 INFO:tasks.workunit.client.0.smithi032.stdout:4/756: write d4/db/da/f1b [589658,19423] 0 2024-06-03T01:21:14.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/753: truncate d8/d11/d20/d3b/d66/f94 2482190 0 2024-06-03T01:21:14.771 INFO:tasks.workunit.client.0.smithi032.stdout:7/753: dwrite dc/d12/d2a/d2f/d76/f80 [0,4194304] 0 2024-06-03T01:21:14.772 INFO:tasks.workunit.client.0.smithi032.stdout:4/757: mknod d4/db/da/d1d/c105 0 2024-06-03T01:21:14.775 INFO:tasks.workunit.client.0.smithi032.stdout:6/754: symlink d8/d11/lf1 0 2024-06-03T01:21:14.776 INFO:tasks.workunit.client.0.smithi032.stdout:4/758: rmdir d4/d57 39 2024-06-03T01:21:14.776 INFO:tasks.workunit.client.0.smithi032.stdout:4/759: write d4/db/da/d5d/ff3 [1006243,83371] 0 2024-06-03T01:21:14.777 INFO:tasks.workunit.client.0.smithi032.stdout:4/760: stat d4/db/da/d37/d90/ccb 0 2024-06-03T01:21:14.779 INFO:tasks.workunit.client.0.smithi032.stdout:7/754: write dc/d12/d24/f6c [614409,131020] 0 2024-06-03T01:21:14.779 INFO:tasks.workunit.client.0.smithi032.stdout:7/755: fdatasync dc/d12/d6a/fcd 0 2024-06-03T01:21:14.779 INFO:tasks.workunit.client.0.smithi032.stdout:7/756: dread - dc/d12/d22/fad zero size 2024-06-03T01:21:14.799 INFO:tasks.workunit.client.0.smithi032.stdout:4/761: read d4/db/da/f54 [2442579,9179] 0 2024-06-03T01:21:14.799 INFO:tasks.workunit.client.0.smithi032.stdout:6/755: dwrite d8/d11/d20/d3b/fd1 [0,4194304] 0 2024-06-03T01:21:14.799 INFO:tasks.workunit.client.0.smithi032.stdout:7/757: dwrite dc/d12/d24/d33/daa/fe1 [0,4194304] 0 2024-06-03T01:21:14.800 INFO:tasks.workunit.client.0.smithi032.stdout:4/762: creat d4/db/da/d5d/d8e/f106 x:0 0 0 2024-06-03T01:21:14.802 INFO:tasks.workunit.client.0.smithi032.stdout:6/756: mkdir d8/d11/d20/d31/df2 0 2024-06-03T01:21:14.806 INFO:tasks.workunit.client.0.smithi032.stdout:6/757: creat d8/d11/d20/d21/ff3 x:0 0 0 2024-06-03T01:21:14.806 INFO:tasks.workunit.client.0.smithi032.stdout:6/758: write d8/d11/d20/d21/fa2 [217153,45366] 0 2024-06-03T01:21:14.806 INFO:tasks.workunit.client.0.smithi032.stdout:6/759: chown d8/d55/l5c 191381613 1 2024-06-03T01:21:14.807 INFO:tasks.workunit.client.0.smithi032.stdout:6/760: stat d8/fb 0 2024-06-03T01:21:14.807 INFO:tasks.workunit.client.0.smithi032.stdout:6/761: creat d8/d11/d20/d2b/d39/d19/ff4 x:0 0 0 2024-06-03T01:21:14.807 INFO:tasks.workunit.client.0.smithi032.stdout:6/762: stat d8/d11/d20/le4 0 2024-06-03T01:21:14.808 INFO:tasks.workunit.client.0.smithi032.stdout:6/763: readlink d8/d11/d20/d31/l33 0 2024-06-03T01:21:14.809 INFO:tasks.workunit.client.0.smithi032.stdout:7/758: rmdir dc/d12/d22/d9f 0 2024-06-03T01:21:14.809 INFO:tasks.workunit.client.0.smithi032.stdout:4/763: write d4/db/da/f49 [2773915,79178] 0 2024-06-03T01:21:14.810 INFO:tasks.workunit.client.0.smithi032.stdout:4/764: mknod d4/db/da/d2d/d27/d53/dea/c107 0 2024-06-03T01:21:14.811 INFO:tasks.workunit.client.0.smithi032.stdout:4/765: symlink d4/db/da/d2d/d27/d8c/db5/l108 0 2024-06-03T01:21:14.813 INFO:tasks.workunit.client.0.smithi032.stdout:4/766: link d4/db/da/d37/d90/da1/cac d4/db/da/d1d/d6b/c109 0 2024-06-03T01:21:14.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/767: symlink d4/l10a 0 2024-06-03T01:21:14.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/768: fsync d4/db/da/d37/f70 0 2024-06-03T01:21:14.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/769: readlink d4/db/da/d5d/lc0 0 2024-06-03T01:21:14.817 INFO:tasks.workunit.client.0.smithi032.stdout:4/770: symlink d4/db/da/d37/d90/l10b 0 2024-06-03T01:21:14.817 INFO:tasks.workunit.client.0.smithi032.stdout:4/771: write d4/db/da/f96 [254149,119381] 0 2024-06-03T01:21:14.821 INFO:tasks.workunit.client.0.smithi032.stdout:6/764: write d8/d11/d20/d7f/f91 [1725674,1275] 0 2024-06-03T01:21:14.821 INFO:tasks.workunit.client.0.smithi032.stdout:6/765: dread - d8/d11/d20/d2b/fb3 zero size 2024-06-03T01:21:14.822 INFO:tasks.workunit.client.0.smithi032.stdout:6/766: write d8/d11/d20/d21/ff3 [73544,65584] 0 2024-06-03T01:21:14.822 INFO:tasks.workunit.client.0.smithi032.stdout:6/767: getdents d8/d11/d20/d21/d9a 0 2024-06-03T01:21:14.822 INFO:tasks.workunit.client.0.smithi032.stdout:6/768: read d8/fb [3009610,27544] 0 2024-06-03T01:21:14.822 INFO:tasks.workunit.client.0.smithi032.stdout:7/759: dread dc/d12/d24/d33/daa/fe1 [0,4194304] 0 2024-06-03T01:21:14.822 INFO:tasks.workunit.client.0.smithi032.stdout:7/760: link dc/d12/d2a/d2f/l73 dc/d12/d6a/d86/led 0 2024-06-03T01:21:14.824 INFO:tasks.workunit.client.0.smithi032.stdout:7/761: rename dc/d12/l62 to dc/d12/d6a/lee 0 2024-06-03T01:21:14.824 INFO:tasks.workunit.client.0.smithi032.stdout:7/762: fsync dc/d12/d22/f29 0 2024-06-03T01:21:14.826 INFO:tasks.workunit.client.0.smithi032.stdout:7/763: unlink dc/d12/d22/d42/fa0 0 2024-06-03T01:21:14.828 INFO:tasks.workunit.client.0.smithi032.stdout:4/772: dread d4/db/da/f49 [0,4194304] 0 2024-06-03T01:21:14.829 INFO:tasks.workunit.client.0.smithi032.stdout:9/766: sync 2024-06-03T01:21:14.830 INFO:tasks.workunit.client.0.smithi032.stdout:6/769: write d8/d11/f35 [3189367,36262] 0 2024-06-03T01:21:14.834 INFO:tasks.workunit.client.0.smithi032.stdout:9/767: creat d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/ffa x:0 0 0 2024-06-03T01:21:14.836 INFO:tasks.workunit.client.0.smithi032.stdout:6/770: unlink d8/d11/d20/d21/d9a/cbc 0 2024-06-03T01:21:14.837 INFO:tasks.workunit.client.0.smithi032.stdout:4/773: dread d4/db/da/d1d/f21 [0,4194304] 0 2024-06-03T01:21:14.837 INFO:tasks.workunit.client.0.smithi032.stdout:9/768: rmdir d0/ddd 39 2024-06-03T01:21:14.838 INFO:tasks.workunit.client.0.smithi032.stdout:6/771: unlink d8/d55/l30 0 2024-06-03T01:21:14.839 INFO:tasks.workunit.client.0.smithi032.stdout:6/772: creat d8/d11/d20/d2b/dc7/ff5 x:0 0 0 2024-06-03T01:21:14.839 INFO:tasks.workunit.client.0.smithi032.stdout:4/774: rmdir d4/db/da/d2d/d27/d53/dea 39 2024-06-03T01:21:14.841 INFO:tasks.workunit.client.0.smithi032.stdout:9/769: mknod d0/d11/d9c/cfb 0 2024-06-03T01:21:14.841 INFO:tasks.workunit.client.0.smithi032.stdout:7/764: dwrite dc/d12/d22/fad [0,4194304] 0 2024-06-03T01:21:14.846 INFO:tasks.workunit.client.0.smithi032.stdout:4/775: rmdir d4/db/da/d37/dab/df7 0 2024-06-03T01:21:14.846 INFO:tasks.workunit.client.0.smithi032.stdout:4/776: chown d4/d7f/ffb 14253 1 2024-06-03T01:21:14.846 INFO:tasks.workunit.client.0.smithi032.stdout:4/777: creat d4/d5c/f10c x:0 0 0 2024-06-03T01:21:14.850 INFO:tasks.workunit.client.0.smithi032.stdout:9/770: write d0/da4/fbe [3605589,59326] 0 2024-06-03T01:21:14.857 INFO:tasks.workunit.client.0.smithi032.stdout:9/771: rename d0/d11/c44 to d0/cfc 0 2024-06-03T01:21:14.857 INFO:tasks.workunit.client.0.smithi032.stdout:9/772: creat d0/da4/ffd x:0 0 0 2024-06-03T01:21:14.857 INFO:tasks.workunit.client.0.smithi032.stdout:9/773: write d0/d11/d1d/d38/d4f/fc3 [1147214,45725] 0 2024-06-03T01:21:14.861 INFO:tasks.workunit.client.0.smithi032.stdout:4/778: dwrite d4/db/da/f15 [0,4194304] 0 2024-06-03T01:21:14.861 INFO:tasks.workunit.client.0.smithi032.stdout:4/779: chown d4/d7f/ffb 211338 1 2024-06-03T01:21:14.877 INFO:tasks.workunit.client.0.smithi032.stdout:9/774: dwrite d0/d11/fd2 [0,4194304] 0 2024-06-03T01:21:14.890 INFO:tasks.workunit.client.0.smithi032.stdout:9/775: dwrite d0/d11/d1d/f8f [0,4194304] 0 2024-06-03T01:21:14.890 INFO:tasks.workunit.client.0.smithi032.stdout:9/776: getdents d0/d11/d1d/dd5 0 2024-06-03T01:21:14.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/780: creat d4/db/da/d37/f10d x:0 0 0 2024-06-03T01:21:14.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/781: readlink d4/db/da/d5d/lbc 0 2024-06-03T01:21:14.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/782: readlink d4/db/da/d5d/lbc 0 2024-06-03T01:21:14.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/783: chown d4/db/da/d37/f10d 15 1 2024-06-03T01:21:14.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/784: fsync d4/db/da/d1d/f21 0 2024-06-03T01:21:14.929 INFO:tasks.workunit.client.0.smithi032.stdout:7/765: link dc/d12/d6a/lee dc/d12/d2a/d81/lef 0 2024-06-03T01:21:14.932 INFO:tasks.workunit.client.0.smithi032.stdout:7/766: dread dc/d12/d6a/fcd [0,4194304] 0 2024-06-03T01:21:14.937 INFO:tasks.workunit.client.0.smithi032.stdout:6/773: rmdir d8/d11/d20/d21/d9a 39 2024-06-03T01:21:14.949 INFO:tasks.workunit.client.0.smithi032.stdout:9/777: creat d0/d11/d1d/d38/d4f/db0/d41/ffe x:0 0 0 2024-06-03T01:21:14.949 INFO:tasks.workunit.client.0.smithi032.stdout:9/778: chown d0/d11/d9c/cc4 1425 1 2024-06-03T01:21:14.950 INFO:tasks.workunit.client.0.smithi032.stdout:4/785: getdents d4/db/da/d1d 0 2024-06-03T01:21:14.952 INFO:tasks.workunit.client.0.smithi032.stdout:7/767: creat dc/d12/ddb/ff0 x:0 0 0 2024-06-03T01:21:14.954 INFO:tasks.workunit.client.0.smithi032.stdout:7/768: dread dc/d12/d22/f52 [0,4194304] 0 2024-06-03T01:21:14.957 INFO:tasks.workunit.client.0.smithi032.stdout:6/774: mknod d8/d11/d20/d21/d9a/cf6 0 2024-06-03T01:21:14.962 INFO:tasks.workunit.client.0.smithi032.stdout:4/786: read d4/db/f1c [2689223,11065] 0 2024-06-03T01:21:14.964 INFO:tasks.workunit.client.0.smithi032.stdout:9/779: rmdir d0/d11/d1d/d38/d4f/db0/d41/dc6 39 2024-06-03T01:21:14.964 INFO:tasks.workunit.client.0.smithi032.stdout:9/780: chown d0/d11/d1d/d38/l3a 2 1 2024-06-03T01:21:14.968 INFO:tasks.workunit.client.0.smithi032.stdout:4/787: creat d4/db/da/f10e x:0 0 0 2024-06-03T01:21:14.972 INFO:tasks.workunit.client.0.smithi032.stdout:4/788: getdents d4 0 2024-06-03T01:21:14.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/789: rmdir d4/db/da/d1d/d6b/df1 39 2024-06-03T01:21:14.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/790: truncate d4/db/da/f80 5117984 0 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:4/791: unlink d4/db/da/lf2 0 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:4/792: fsync d4/db/f1c 0 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:4/793: fsync d4/db/da/d1d/d6b/f7c 0 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:4/794: fdatasync d4/db/da/f2c 0 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:4/795: dread - d4/d5c/f10c zero size 2024-06-03T01:21:14.978 INFO:tasks.workunit.client.0.smithi032.stdout:9/781: dread d0/d11/d1d/da6/fa2 [0,4194304] 0 2024-06-03T01:21:14.979 INFO:tasks.workunit.client.0.smithi032.stdout:9/782: mknod d0/ddd/cff 0 2024-06-03T01:21:14.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/775: dwrite d8/d11/d20/d2b/d39/d19/f40 [0,4194304] 0 2024-06-03T01:21:14.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/776: readlink d8/d11/d20/d31/d3f/lcf 0 2024-06-03T01:21:14.984 INFO:tasks.workunit.client.0.smithi032.stdout:9/783: symlink d0/d11/d1d/d38/d4f/l100 0 2024-06-03T01:21:14.984 INFO:tasks.workunit.client.0.smithi032.stdout:9/784: creat d0/d11/d1d/d38/d4f/db0/d41/f101 x:0 0 0 2024-06-03T01:21:14.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/777: getdents d8/d5f 0 2024-06-03T01:21:14.985 INFO:tasks.workunit.client.0.smithi032.stdout:6/778: truncate d8/d11/d20/d21/d9a/fba 1603445 0 2024-06-03T01:21:14.990 INFO:tasks.workunit.client.0.smithi032.stdout:9/785: symlink d0/d23/d25/de1/l102 0 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:9/786: read - d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/ffa zero size 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:9/787: write d0/d11/f22 [13276551,20817] 0 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/779: mknod d8/dd5/da5/cf7 0 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/780: truncate d8/d50/fac 4775144 0 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/781: write d8/d11/d20/d2b/dc7/ff0 [355621,114309] 0 2024-06-03T01:21:14.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/782: chown d8/d11/d20/d2b/d39/d7e 3250256 1 2024-06-03T01:21:14.993 INFO:tasks.workunit.client.0.smithi032.stdout:9/788: dread d0/d23/d24/f9d [0,4194304] 0 2024-06-03T01:21:14.993 INFO:tasks.workunit.client.0.smithi032.stdout:4/796: truncate d4/db/da/f96 18290 0 2024-06-03T01:21:14.995 INFO:tasks.workunit.client.0.smithi032.stdout:4/797: link d4/db/da/d37/l47 d4/db/da/d5d/l10f 0 2024-06-03T01:21:14.996 INFO:tasks.workunit.client.0.smithi032.stdout:4/798: mkdir d4/db/da/d1d/d6b/df1/d110 0 2024-06-03T01:21:15.007 INFO:tasks.workunit.client.0.smithi032.stdout:6/783: dread d8/d11/d20/d2b/d39/d19/f40 [0,4194304] 0 2024-06-03T01:21:15.009 INFO:tasks.workunit.client.0.smithi032.stdout:9/789: write d0/d11/f86 [3422632,7304] 0 2024-06-03T01:21:15.019 INFO:tasks.workunit.client.0.smithi032.stdout:6/784: dread d8/d11/d20/d21/d9a/fdc [0,4194304] 0 2024-06-03T01:21:15.020 INFO:tasks.workunit.client.0.smithi032.stdout:6/785: mkdir d8/d11/d20/d31/df2/df8 0 2024-06-03T01:21:15.025 INFO:tasks.workunit.client.0.smithi032.stdout:7/769: dwrite dc/d12/d22/d42/f51 [4194304,4194304] 0 2024-06-03T01:21:15.025 INFO:tasks.workunit.client.0.smithi032.stdout:7/770: write dc/d12/d2a/d2f/d63/d69/f5e [2623101,56061] 0 2024-06-03T01:21:15.026 INFO:tasks.workunit.client.0.smithi032.stdout:7/771: mkdir dc/d12/d22/d42/d61/df1 0 2024-06-03T01:21:15.034 INFO:tasks.workunit.client.0.smithi032.stdout:6/786: dread d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:15.036 INFO:tasks.workunit.client.0.smithi032.stdout:9/790: dwrite d0/d23/d25/de1/df1/dea/f96 [0,4194304] 0 2024-06-03T01:21:15.036 INFO:tasks.workunit.client.0.smithi032.stdout:6/787: unlink d8/d11/d20/d31/d3f/fea 0 2024-06-03T01:21:15.040 INFO:tasks.workunit.client.0.smithi032.stdout:6/788: rename d8/d11/d20/d2b/d39/c22 to d8/d11/d20/d2b/cf9 0 2024-06-03T01:21:15.040 INFO:tasks.workunit.client.0.smithi032.stdout:8/736: sync 2024-06-03T01:21:15.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/772: dwrite dc/d12/d6a/d86/dc8/feb [0,4194304] 0 2024-06-03T01:21:15.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/773: creat dc/d12/d2a/d81/ff2 x:0 0 0 2024-06-03T01:21:15.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/774: chown dc/d12/d2a/d2f/fd2 96 1 2024-06-03T01:21:15.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/775: fdatasync dc/d12/d6a/d86/d88/d8b/fb5 0 2024-06-03T01:21:15.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/776: dread - dc/d12/d6a/f72 zero size 2024-06-03T01:21:15.056 INFO:tasks.workunit.client.0.smithi032.stdout:9/791: dwrite d0/d11/d76/ff3 [0,4194304] 0 2024-06-03T01:21:15.056 INFO:tasks.workunit.client.0.smithi032.stdout:9/792: chown d0/d11/d1d/d38/c9e 73225 1 2024-06-03T01:21:15.072 INFO:tasks.workunit.client.0.smithi032.stdout:9/793: write d0/fe3 [827957,90231] 0 2024-06-03T01:21:15.073 INFO:tasks.workunit.client.0.smithi032.stdout:9/794: creat d0/d11/d79/f103 x:0 0 0 2024-06-03T01:21:15.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/737: dwrite dc/d25/f6c [0,4194304] 0 2024-06-03T01:21:15.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/738: getdents dc/d25/d28/d51/d9a 0 2024-06-03T01:21:15.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/739: chown f9 947 1 2024-06-03T01:21:15.080 INFO:tasks.workunit.client.0.smithi032.stdout:6/789: dwrite d8/d11/d20/f9f [0,4194304] 0 2024-06-03T01:21:15.080 INFO:tasks.workunit.client.0.smithi032.stdout:7/777: dwrite dc/d12/d2a/f43 [0,4194304] 0 2024-06-03T01:21:15.083 INFO:tasks.workunit.client.0.smithi032.stdout:8/740: symlink dc/de/d1e/d43/d9b/d76/le1 0 2024-06-03T01:21:15.084 INFO:tasks.workunit.client.0.smithi032.stdout:6/790: dread d8/d11/d20/d31/d3f/f69 [0,4194304] 0 2024-06-03T01:21:15.084 INFO:tasks.workunit.client.0.smithi032.stdout:7/778: truncate dc/d12/f6b 2938605 0 2024-06-03T01:21:15.086 INFO:tasks.workunit.client.0.smithi032.stdout:8/741: rename dc/d25/f55 to dc/d25/d28/d58/d8d/d94/fe2 0 2024-06-03T01:21:15.087 INFO:tasks.workunit.client.0.smithi032.stdout:6/791: mkdir d8/d11/d20/d31/dfa 0 2024-06-03T01:21:15.089 INFO:tasks.workunit.client.0.smithi032.stdout:8/742: mkdir dc/d14/de3 0 2024-06-03T01:21:15.089 INFO:tasks.workunit.client.0.smithi032.stdout:8/743: mknod dc/d25/d64/ce4 0 2024-06-03T01:21:15.090 INFO:tasks.workunit.client.0.smithi032.stdout:8/744: fdatasync dc/d25/d28/d32/f7d 0 2024-06-03T01:21:15.090 INFO:tasks.workunit.client.0.smithi032.stdout:8/745: creat dc/d25/d28/d33/fe5 x:0 0 0 2024-06-03T01:21:15.091 INFO:tasks.workunit.client.0.smithi032.stdout:8/746: link dc/d25/d28/d32/f60 dc/d25/d28/d33/fe6 0 2024-06-03T01:21:15.091 INFO:tasks.workunit.client.0.smithi032.stdout:8/747: truncate dc/de/d1e/d43/d9b/d76/fc0 275515 0 2024-06-03T01:21:15.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/748: creat dc/d25/d28/d58/d8d/d94/fe7 x:0 0 0 2024-06-03T01:21:15.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/749: readlink dc/de/d1e/d43/d9b/d76/le1 0 2024-06-03T01:21:15.093 INFO:tasks.workunit.client.0.smithi032.stdout:8/750: mknod dc/de/d1e/d43/d9b/ce8 0 2024-06-03T01:21:15.095 INFO:tasks.workunit.client.0.smithi032.stdout:9/795: dwrite d0/d11/d1d/d38/f61 [0,4194304] 0 2024-06-03T01:21:15.098 INFO:tasks.workunit.client.0.smithi032.stdout:9/796: creat d0/d11/d1d/d38/d4f/db0/f104 x:0 0 0 2024-06-03T01:21:15.098 INFO:tasks.workunit.client.0.smithi032.stdout:9/797: readlink d0/d11/d1d/d38/l64 0 2024-06-03T01:21:15.098 INFO:tasks.workunit.client.0.smithi032.stdout:7/779: write dc/d12/d2a/d2f/d76/f7d [1903710,11095] 0 2024-06-03T01:21:15.098 INFO:tasks.workunit.client.0.smithi032.stdout:7/780: write dc/d12/d24/d33/de5/fe9 [539824,59434] 0 2024-06-03T01:21:15.099 INFO:tasks.workunit.client.0.smithi032.stdout:7/781: dread - dc/d12/d2a/d81/ff2 zero size 2024-06-03T01:21:15.100 INFO:tasks.workunit.client.0.smithi032.stdout:7/782: creat dc/d12/d6a/d86/ff3 x:0 0 0 2024-06-03T01:21:15.101 INFO:tasks.workunit.client.0.smithi032.stdout:7/783: mknod dc/d12/d2a/d2f/cf4 0 2024-06-03T01:21:15.102 INFO:tasks.workunit.client.0.smithi032.stdout:6/792: dread d8/d11/d20/d3b/fd1 [0,4194304] 0 2024-06-03T01:21:15.109 INFO:tasks.workunit.client.0.smithi032.stdout:7/784: dread dc/d12/f6b [0,4194304] 0 2024-06-03T01:21:15.113 INFO:tasks.workunit.client.0.smithi032.stdout:7/785: symlink dc/d12/d22/d42/lf5 0 2024-06-03T01:21:15.115 INFO:tasks.workunit.client.0.smithi032.stdout:8/751: dwrite f8 [0,4194304] 0 2024-06-03T01:21:15.115 INFO:tasks.workunit.client.0.smithi032.stdout:8/752: fdatasync dc/d25/f35 0 2024-06-03T01:21:15.121 INFO:tasks.workunit.client.0.smithi032.stdout:7/786: write dc/d12/d2a/f43 [679094,108916] 0 2024-06-03T01:21:15.122 INFO:tasks.workunit.client.0.smithi032.stdout:7/787: rename dc/d12/d2a/d2f/d76/fc6 to dc/d12/ff6 0 2024-06-03T01:21:15.123 INFO:tasks.workunit.client.0.smithi032.stdout:7/788: unlink dc/d12/f6b 0 2024-06-03T01:21:15.125 INFO:tasks.workunit.client.0.smithi032.stdout:7/789: rename dc/d12/d22/l58 to dc/d12/ddb/lf7 0 2024-06-03T01:21:15.127 INFO:tasks.workunit.client.0.smithi032.stdout:9/798: dwrite d0/d11/d1d/d38/f61 [4194304,4194304] 0 2024-06-03T01:21:15.127 INFO:tasks.workunit.client.0.smithi032.stdout:9/799: stat d0/d11/d1d/d38/d4f/db0/d41/c5e 0 2024-06-03T01:21:15.127 INFO:tasks.workunit.client.0.smithi032.stdout:9/800: write d0/da4/fe9 [1849534,59834] 0 2024-06-03T01:21:15.128 INFO:tasks.workunit.client.0.smithi032.stdout:6/793: dwrite d8/dd5/fef [0,4194304] 0 2024-06-03T01:21:15.130 INFO:tasks.workunit.client.0.smithi032.stdout:9/801: creat d0/d11/d1d/d38/d4f/db0/d41/d9b/f105 x:0 0 0 2024-06-03T01:21:15.130 INFO:tasks.workunit.client.0.smithi032.stdout:8/753: dread dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:15.134 INFO:tasks.workunit.client.0.smithi032.stdout:8/754: getdents dc/d25/d28/d33 0 2024-06-03T01:21:15.134 INFO:tasks.workunit.client.0.smithi032.stdout:6/794: dread d8/d11/d20/d2b/d39/f67 [0,4194304] 0 2024-06-03T01:21:15.134 INFO:tasks.workunit.client.0.smithi032.stdout:6/795: write d8/d11/d20/d3b/d66/f94 [1103716,85883] 0 2024-06-03T01:21:15.137 INFO:tasks.workunit.client.0.smithi032.stdout:8/755: link dc/de/f91 dc/de/d1e/d43/d9b/fe9 0 2024-06-03T01:21:15.138 INFO:tasks.workunit.client.0.smithi032.stdout:6/796: chown d8/d11/f34 4 1 2024-06-03T01:21:15.140 INFO:tasks.workunit.client.0.smithi032.stdout:9/802: read d0/d11/d1d/f47 [1575003,32903] 0 2024-06-03T01:21:15.141 INFO:tasks.workunit.client.0.smithi032.stdout:9/803: fdatasync d0/d23/d25/f4c 0 2024-06-03T01:21:15.141 INFO:tasks.workunit.client.0.smithi032.stdout:8/756: mkdir dc/d25/d28/dea 0 2024-06-03T01:21:15.141 INFO:tasks.workunit.client.0.smithi032.stdout:9/804: write d0/d11/d1d/d38/d4f/db0/d41/fb3 [180589,126557] 0 2024-06-03T01:21:15.141 INFO:tasks.workunit.client.0.smithi032.stdout:9/805: chown d0/d23/d25/de1/df1 44368709 1 2024-06-03T01:21:15.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/790: dwrite dc/d12/d22/d42/d61/ddd/f36 [0,4194304] 0 2024-06-03T01:21:15.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/791: write dc/d12/fa1 [207974,8407] 0 2024-06-03T01:21:15.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/792: chown dc/d12/d78/cbb 14947 1 2024-06-03T01:21:15.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/793: chown dc/d12/f77 2304284 1 2024-06-03T01:21:15.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/797: link d8/d55/f6a d8/ffb 0 2024-06-03T01:21:15.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/798: write d8/d11/d20/d31/fe6 [4589428,4247] 0 2024-06-03T01:21:15.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/799: stat d8/d11/d20/d3b/fc2 0 2024-06-03T01:21:15.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/800: fdatasync d8/d4c/fc0 0 2024-06-03T01:21:15.146 INFO:tasks.workunit.client.0.smithi032.stdout:8/757: symlink dc/d25/d28/d33/leb 0 2024-06-03T01:21:15.151 INFO:tasks.workunit.client.0.smithi032.stdout:6/801: creat d8/d11/d20/d7f/ffc x:0 0 0 2024-06-03T01:21:15.153 INFO:tasks.workunit.client.0.smithi032.stdout:8/758: mknod dc/de/d1e/d43/cec 0 2024-06-03T01:21:15.153 INFO:tasks.workunit.client.0.smithi032.stdout:8/759: write dc/d25/d28/d58/f86 [1510930,28070] 0 2024-06-03T01:21:15.154 INFO:tasks.workunit.client.0.smithi032.stdout:6/802: mkdir d8/d50/dda/dfd 0 2024-06-03T01:21:15.154 INFO:tasks.workunit.client.0.smithi032.stdout:6/803: chown d8/dd5/dae/fb0 0 1 2024-06-03T01:21:15.158 INFO:tasks.workunit.client.0.smithi032.stdout:9/806: dwrite d0/d11/d1d/d38/f61 [0,4194304] 0 2024-06-03T01:21:15.159 INFO:tasks.workunit.client.0.smithi032.stdout:8/760: rename f1 to dc/d25/d28/d58/d8d/d94/da6/fed 0 2024-06-03T01:21:15.161 INFO:tasks.workunit.client.0.smithi032.stdout:9/807: dread d0/d23/d25/de1/df1/dea/fc7 [0,4194304] 0 2024-06-03T01:21:15.161 INFO:tasks.workunit.client.0.smithi032.stdout:7/794: dwrite dc/d12/d22/fc0 [0,4194304] 0 2024-06-03T01:21:15.170 INFO:tasks.workunit.client.0.smithi032.stdout:9/808: rename d0/d23/ccd to d0/d11/d9c/c106 0 2024-06-03T01:21:15.170 INFO:tasks.workunit.client.0.smithi032.stdout:9/809: getdents d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/dee 0 2024-06-03T01:21:15.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/795: truncate dc/d12/f25 495714 0 2024-06-03T01:21:15.171 INFO:tasks.workunit.client.0.smithi032.stdout:8/761: mkdir dc/d14/dee 0 2024-06-03T01:21:15.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/762: chown dc/d25/d64/db5 525827 1 2024-06-03T01:21:15.172 INFO:tasks.workunit.client.0.smithi032.stdout:7/796: dread dc/d12/d6a/fcd [0,4194304] 0 2024-06-03T01:21:15.172 INFO:tasks.workunit.client.0.smithi032.stdout:7/797: stat dc/d12/d22/d42/d61/ddd/f2c 0 2024-06-03T01:21:15.173 INFO:tasks.workunit.client.0.smithi032.stdout:9/810: symlink d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/de8/l107 0 2024-06-03T01:21:15.185 INFO:tasks.workunit.client.0.smithi032.stdout:9/811: dread d0/d23/d25/de1/df1/dea/f96 [0,4194304] 0 2024-06-03T01:21:15.185 INFO:tasks.workunit.client.0.smithi032.stdout:9/812: truncate d0/d11/d1d/d38/d4f/db0/ddb/fed 195379 0 2024-06-03T01:21:15.200 INFO:tasks.workunit.client.0.smithi032.stdout:9/813: dwrite d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/fdf [0,4194304] 0 2024-06-03T01:21:15.205 INFO:tasks.workunit.client.0.smithi032.stdout:7/798: mknod dc/d12/d24/dd3/cf8 0 2024-06-03T01:21:15.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/763: dwrite dc/de/f95 [0,4194304] 0 2024-06-03T01:21:15.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/764: truncate dc/de/f91 5443261 0 2024-06-03T01:21:15.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/765: dread dc/d25/d28/d58/f75 [0,4194304] 0 2024-06-03T01:21:15.230 INFO:tasks.workunit.client.0.smithi032.stdout:9/814: unlink d0/d23/d25/de1/df1/dea/ca9 0 2024-06-03T01:21:15.230 INFO:tasks.workunit.client.0.smithi032.stdout:9/815: creat d0/d23/d25/dd6/f108 x:0 0 0 2024-06-03T01:21:15.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/799: creat dc/ff9 x:0 0 0 2024-06-03T01:21:15.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/800: creat dc/d12/d6a/d86/dc8/ffa x:0 0 0 2024-06-03T01:21:15.240 INFO:tasks.workunit.client.0.smithi032.stdout:8/766: dwrite dc/de/d1e/d43/d9b/d76/fc0 [0,4194304] 0 2024-06-03T01:21:15.241 INFO:tasks.workunit.client.0.smithi032.stdout:8/767: write dc/d25/d64/fbd [3834213,28055] 0 2024-06-03T01:21:15.242 INFO:tasks.workunit.client.0.smithi032.stdout:8/768: read dc/de/f9f [380662,84456] 0 2024-06-03T01:21:15.242 INFO:tasks.workunit.client.0.smithi032.stdout:8/769: chown dc/d25/f35 7700959 1 2024-06-03T01:21:15.286 INFO:tasks.workunit.client.0.smithi032.stdout:9/816: creat d0/ddd/f109 x:0 0 0 2024-06-03T01:21:15.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/801: symlink dc/d12/d24/d33/daa/lfb 0 2024-06-03T01:21:15.292 INFO:tasks.workunit.client.0.smithi032.stdout:8/770: mknod dc/de/d1e/d43/d9b/d76/cef 0 2024-06-03T01:21:15.292 INFO:tasks.workunit.client.0.smithi032.stdout:8/771: read - dc/d25/d28/d58/d8d/d94/d59/fd1 zero size 2024-06-03T01:21:15.302 INFO:tasks.workunit.client.0.smithi032.stdout:3/683: sync 2024-06-03T01:21:15.302 INFO:tasks.workunit.client.0.smithi032.stdout:3/684: chown de/d29/d2e/d31/d23/d53 0 1 2024-06-03T01:21:15.303 INFO:tasks.workunit.client.0.smithi032.stdout:3/685: truncate de/d29/f95 3968229 0 2024-06-03T01:21:15.304 INFO:tasks.workunit.client.0.smithi032.stdout:3/686: mkdir de/d29/d2e/d31/d8d/de1 0 2024-06-03T01:21:15.307 INFO:tasks.workunit.client.0.smithi032.stdout:7/802: dwrite dc/d12/d22/d42/d61/ddd/f21 [0,4194304] 0 2024-06-03T01:21:15.308 INFO:tasks.workunit.client.0.smithi032.stdout:7/803: readlink dc/d12/d22/d42/lf5 0 2024-06-03T01:21:15.310 INFO:tasks.workunit.client.0.smithi032.stdout:3/687: read de/d29/d2e/d31/d3c/f44 [2762351,21115] 0 2024-06-03T01:21:15.313 INFO:tasks.workunit.client.0.smithi032.stdout:3/688: link de/d29/d2e/d31/ce0 de/d63/ce2 0 2024-06-03T01:21:15.313 INFO:tasks.workunit.client.0.smithi032.stdout:3/689: chown de/d29/d2e/d31/d23/c49 195580 1 2024-06-03T01:21:15.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/804: unlink dc/d12/d22/fe8 0 2024-06-03T01:21:15.328 INFO:tasks.workunit.client.0.smithi032.stdout:7/805: write dc/f2b [2981020,55335] 0 2024-06-03T01:21:15.328 INFO:tasks.workunit.client.0.smithi032.stdout:3/690: dwrite de/d29/f3b [0,4194304] 0 2024-06-03T01:21:15.330 INFO:tasks.workunit.client.0.smithi032.stdout:3/691: dread de/d29/d2e/d31/d3c/f54 [0,4194304] 0 2024-06-03T01:21:15.330 INFO:tasks.workunit.client.0.smithi032.stdout:7/806: symlink dc/d12/d6a/lfc 0 2024-06-03T01:21:15.332 INFO:tasks.workunit.client.0.smithi032.stdout:3/692: symlink de/d29/d2e/d31/d23/d53/d79/dab/le3 0 2024-06-03T01:21:15.345 INFO:tasks.workunit.client.0.smithi032.stdout:8/772: mkdir dc/de/df0 0 2024-06-03T01:21:15.346 INFO:tasks.workunit.client.0.smithi032.stdout:8/773: mknod dc/d25/d28/d58/d8d/d94/d59/cf1 0 2024-06-03T01:21:15.346 INFO:tasks.workunit.client.0.smithi032.stdout:8/774: fdatasync dc/de/d1e/d43/d9b/d76/fb7 0 2024-06-03T01:21:15.348 INFO:tasks.workunit.client.0.smithi032.stdout:8/775: truncate dc/d25/f35 953348 0 2024-06-03T01:21:15.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/776: fsync fa 0 2024-06-03T01:21:15.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/777: truncate dc/d25/d28/d58/dad/fd0 651715 0 2024-06-03T01:21:15.353 INFO:tasks.workunit.client.0.smithi032.stdout:9/817: mknod d0/d11/d1d/d38/d4f/db0/c10a 0 2024-06-03T01:21:15.354 INFO:tasks.workunit.client.0.smithi032.stdout:9/818: getdents d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9 0 2024-06-03T01:21:15.356 INFO:tasks.workunit.client.0.smithi032.stdout:7/807: dwrite dc/ff9 [0,4194304] 0 2024-06-03T01:21:15.357 INFO:tasks.workunit.client.0.smithi032.stdout:9/819: creat d0/da4/dc1/f10b x:0 0 0 2024-06-03T01:21:15.360 INFO:tasks.workunit.client.0.smithi032.stdout:7/808: truncate f6 415796 0 2024-06-03T01:21:15.360 INFO:tasks.workunit.client.0.smithi032.stdout:7/809: creat dc/d12/d2a/d2f/d76/ffd x:0 0 0 2024-06-03T01:21:15.360 INFO:tasks.workunit.client.0.smithi032.stdout:7/810: write dc/d12/d24/d7b/d84/fc2 [1025145,13882] 0 2024-06-03T01:21:15.361 INFO:tasks.workunit.client.0.smithi032.stdout:3/693: dwrite de/d29/d2e/d31/d24/fa5 [0,4194304] 0 2024-06-03T01:21:15.361 INFO:tasks.workunit.client.0.smithi032.stdout:3/694: stat de/f10 0 2024-06-03T01:21:15.362 INFO:tasks.workunit.client.0.smithi032.stdout:9/820: rename d0/d11/d1d/d38/d4f/db0/ddb/cec to d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/c10c 0 2024-06-03T01:21:15.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/695: rename c3 to de/ce4 0 2024-06-03T01:21:15.368 INFO:tasks.workunit.client.0.smithi032.stdout:8/778: dwrite dc/d25/f6c [0,4194304] 0 2024-06-03T01:21:15.369 INFO:tasks.workunit.client.0.smithi032.stdout:3/696: symlink de/d29/d2e/d31/da1/le5 0 2024-06-03T01:21:15.370 INFO:tasks.workunit.client.0.smithi032.stdout:8/779: mkdir dc/de/d1e/d43/d9b/d76/dae/df2 0 2024-06-03T01:21:15.372 INFO:tasks.workunit.client.0.smithi032.stdout:7/811: write dc/d12/d2a/d2f/d76/f80 [2961033,73670] 0 2024-06-03T01:21:15.376 INFO:tasks.workunit.client.0.smithi032.stdout:8/780: unlink dc/d25/d64/l96 0 2024-06-03T01:21:15.378 INFO:tasks.workunit.client.0.smithi032.stdout:8/781: rmdir dc/de/d1e/d43/d9b/dd7/dda 0 2024-06-03T01:21:15.378 INFO:tasks.workunit.client.0.smithi032.stdout:8/782: readlink dc/de/d1e/l5b 0 2024-06-03T01:21:15.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/783: readlink dc/d25/d28/d33/l63 0 2024-06-03T01:21:15.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/784: dread - dc/d25/d28/d32/fa2 zero size 2024-06-03T01:21:15.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/785: dread - dc/de/d1e/fa3 zero size 2024-06-03T01:21:15.380 INFO:tasks.workunit.client.0.smithi032.stdout:3/697: dread de/d29/d2e/d31/d23/f59 [0,4194304] 0 2024-06-03T01:21:15.382 INFO:tasks.workunit.client.0.smithi032.stdout:3/698: creat de/fe6 x:0 0 0 2024-06-03T01:21:15.385 INFO:tasks.workunit.client.0.smithi032.stdout:3/699: getdents de/d29/d2e/d31/d3c/d50/d62 0 2024-06-03T01:21:15.385 INFO:tasks.workunit.client.0.smithi032.stdout:3/700: getdents de/d29/d2e/d31/d8d/de1 0 2024-06-03T01:21:15.388 INFO:tasks.workunit.client.0.smithi032.stdout:3/701: mkdir de/d29/d2e/d31/d3c/d50/d56/de7 0 2024-06-03T01:21:15.388 INFO:tasks.workunit.client.0.smithi032.stdout:8/786: dread f2 [0,4194304] 0 2024-06-03T01:21:15.388 INFO:tasks.workunit.client.0.smithi032.stdout:8/787: chown dc/d14/d31/l3d 5 1 2024-06-03T01:21:15.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/812: dwrite dc/d12/d2a/d2f/d63/d69/f5f [0,4194304] 0 2024-06-03T01:21:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/702: truncate de/d29/d2e/d31/d8d/dc3/fbd 4055231 0 2024-06-03T01:21:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/703: stat de/d29/d2e/d31/d78/f86 0 2024-06-03T01:21:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/704: fsync de/d29/fc6 0 2024-06-03T01:21:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/705: readlink de/d29/d2e/d31/l22 0 2024-06-03T01:21:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:3/706: chown de/d29/d2e/d31/d3c 53519362 1 2024-06-03T01:21:15.394 INFO:tasks.workunit.client.0.smithi032.stdout:7/813: mknod dc/d12/d24/d33/de5/cfe 0 2024-06-03T01:21:15.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/707: creat de/d29/d2e/d31/d23/d53/d79/dab/fe8 x:0 0 0 2024-06-03T01:21:15.399 INFO:tasks.workunit.client.0.smithi032.stdout:3/708: symlink de/d29/d2e/d31/da1/le9 0 2024-06-03T01:21:15.400 INFO:tasks.workunit.client.0.smithi032.stdout:3/709: mkdir de/d29/d2e/d31/d8d/de1/dea 0 2024-06-03T01:21:15.401 INFO:tasks.workunit.client.0.smithi032.stdout:3/710: creat de/d29/d2e/d31/d3c/d50/d56/dc7/feb x:0 0 0 2024-06-03T01:21:15.401 INFO:tasks.workunit.client.0.smithi032.stdout:3/711: fdatasync de/d29/d2e/d31/d3c/f54 0 2024-06-03T01:21:15.401 INFO:tasks.workunit.client.0.smithi032.stdout:3/712: write de/d29/d2e/d31/d24/f25 [2657928,37039] 0 2024-06-03T01:21:15.401 INFO:tasks.workunit.client.0.smithi032.stdout:3/713: write de/fe6 [660283,42089] 0 2024-06-03T01:21:15.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/714: creat de/d29/d2e/d31/d8d/de1/fec x:0 0 0 2024-06-03T01:21:15.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/715: write de/d63/f76 [491084,107162] 0 2024-06-03T01:21:15.411 INFO:tasks.workunit.client.0.smithi032.stdout:7/814: read dc/d12/d22/f29 [3188806,131046] 0 2024-06-03T01:21:15.462 INFO:tasks.workunit.client.0.smithi032.stdout:7/815: unlink lb 0 2024-06-03T01:21:15.462 INFO:tasks.workunit.client.0.smithi032.stdout:7/816: chown dc/d12/d2a/d2f/d9a 507276 1 2024-06-03T01:21:15.463 INFO:tasks.workunit.client.0.smithi032.stdout:7/817: rename dc/d12/d24/f38 to dc/d12/d24/d7b/d9d/fff 0 2024-06-03T01:21:15.466 INFO:tasks.workunit.client.0.smithi032.stdout:8/788: dwrite dc/d14/d31/d3b/fe0 [0,4194304] 0 2024-06-03T01:21:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:15 smithi044 ceph-mon[36956]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 270 MiB/s rd, 327 MiB/s wr, 550 op/s 2024-06-03T01:21:15.472 INFO:tasks.workunit.client.0.smithi032.stdout:8/789: rmdir dc/d25/d28/d3c 39 2024-06-03T01:21:15.472 INFO:tasks.workunit.client.0.smithi032.stdout:8/790: stat dc/d25/d28/d58/d8d/d94/d59/cf1 0 2024-06-03T01:21:15.472 INFO:tasks.workunit.client.0.smithi032.stdout:8/791: chown dc/d14/l9d 15591 1 2024-06-03T01:21:15.483 INFO:tasks.workunit.client.0.smithi032.stdout:8/792: read dc/d25/d64/fbd [3775604,29595] 0 2024-06-03T01:21:15.484 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:15 smithi032 ceph-mon[29360]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 270 MiB/s rd, 327 MiB/s wr, 550 op/s 2024-06-03T01:21:15.484 INFO:tasks.workunit.client.0.smithi032.stdout:8/793: creat dc/de/d1e/ff3 x:0 0 0 2024-06-03T01:21:15.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/716: dwrite de/d29/d2e/d31/d8d/dc3/fa6 [0,4194304] 0 2024-06-03T01:21:15.488 INFO:tasks.workunit.client.0.smithi032.stdout:3/717: link de/d29/fb5 de/d29/d2e/d31/d23/fed 0 2024-06-03T01:21:15.488 INFO:tasks.workunit.client.0.smithi032.stdout:3/718: write de/d63/fb2 [982382,50150] 0 2024-06-03T01:21:15.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/719: unlink de/d29/d2e/d31/d3c/d50/d56/dc7/feb 0 2024-06-03T01:21:15.492 INFO:tasks.workunit.client.0.smithi032.stdout:8/794: dread dc/d14/d31/f7b [0,4194304] 0 2024-06-03T01:21:15.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/795: dread dc/d25/d28/d3c/f3f [0,4194304] 0 2024-06-03T01:21:15.504 INFO:tasks.workunit.client.0.smithi032.stdout:8/796: creat dc/d93/ff4 x:0 0 0 2024-06-03T01:21:15.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/720: dwrite de/d29/f7f [0,4194304] 0 2024-06-03T01:21:15.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/721: dread - de/d29/d2e/d31/f1e zero size 2024-06-03T01:21:15.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/797: write dc/d25/d28/d58/d8d/d94/d59/f9e [87338,87975] 0 2024-06-03T01:21:15.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/818: dwrite dc/d12/d6a/f72 [0,4194304] 0 2024-06-03T01:21:15.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/819: chown dc/d12/d2a/d2f/d63/l7e 274113919 1 2024-06-03T01:21:15.532 INFO:tasks.workunit.client.0.smithi032.stdout:3/722: dwrite de/d29/d2e/d31/d8d/dc3/fbd [4194304,4194304] 0 2024-06-03T01:21:15.533 INFO:tasks.workunit.client.0.smithi032.stdout:3/723: write de/d29/d2e/d31/d23/d53/dbf/fc4 [706638,104504] 0 2024-06-03T01:21:15.533 INFO:tasks.workunit.client.0.smithi032.stdout:8/798: dwrite dc/d25/d64/db5/fc7 [0,4194304] 0 2024-06-03T01:21:15.535 INFO:tasks.workunit.client.0.smithi032.stdout:8/799: read dc/d25/d28/fb8 [312757,16738] 0 2024-06-03T01:21:15.535 INFO:tasks.workunit.client.0.smithi032.stdout:8/800: fdatasync dc/d25/d28/d58/dad/fd0 0 2024-06-03T01:21:15.538 INFO:tasks.workunit.client.0.smithi032.stdout:8/801: truncate dc/f15 1238528 0 2024-06-03T01:21:15.540 INFO:tasks.workunit.client.0.smithi032.stdout:8/802: mkdir dc/d25/d28/d3c/df5 0 2024-06-03T01:21:15.540 INFO:tasks.workunit.client.0.smithi032.stdout:8/803: write dc/d25/d28/d32/fa2 [137961,52504] 0 2024-06-03T01:21:15.540 INFO:tasks.workunit.client.0.smithi032.stdout:8/804: write dc/d25/d28/d58/f86 [1978082,38290] 0 2024-06-03T01:21:15.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/820: dread dc/d12/d6a/d86/dc8/feb [0,4194304] 0 2024-06-03T01:21:15.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/821: creat dc/d12/ddb/f100 x:0 0 0 2024-06-03T01:21:15.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/822: chown dc/d12/d24/d7b/d9d 1961 1 2024-06-03T01:21:15.548 INFO:tasks.workunit.client.0.smithi032.stdout:3/724: dwrite de/d29/d2e/d31/d23/d53/d69/d8c/fa9 [0,4194304] 0 2024-06-03T01:21:15.553 INFO:tasks.workunit.client.0.smithi032.stdout:3/725: mkdir de/d29/d2e/d31/d23/d53/dbf/dee 0 2024-06-03T01:21:15.559 INFO:tasks.workunit.client.0.smithi032.stdout:8/805: dwrite dc/d25/f6c [4194304,4194304] 0 2024-06-03T01:21:15.562 INFO:tasks.workunit.client.0.smithi032.stdout:8/806: unlink dc/d14/d31/d3b/fe0 0 2024-06-03T01:21:15.564 INFO:tasks.workunit.client.0.smithi032.stdout:8/807: link dc/d25/d28/d33/lbc dc/d25/d28/lf6 0 2024-06-03T01:21:15.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/726: dwrite f9 [0,4194304] 0 2024-06-03T01:21:15.581 INFO:tasks.workunit.client.0.smithi032.stdout:8/808: dwrite dc/d25/d28/d58/f86 [0,4194304] 0 2024-06-03T01:21:15.582 INFO:tasks.workunit.client.0.smithi032.stdout:3/727: rename de/d29/fb5 to de/d29/d2e/d31/d3c/d50/d56/fef 0 2024-06-03T01:21:15.585 INFO:tasks.workunit.client.0.smithi032.stdout:8/809: mknod dc/d25/d28/d32/dbb/cf7 0 2024-06-03T01:21:15.586 INFO:tasks.workunit.client.0.smithi032.stdout:8/810: fsync dc/de/f91 0 2024-06-03T01:21:15.588 INFO:tasks.workunit.client.0.smithi032.stdout:3/728: unlink de/d29/c4c 0 2024-06-03T01:21:15.590 INFO:tasks.workunit.client.0.smithi032.stdout:3/729: creat de/d29/d2e/d31/da1/ff0 x:0 0 0 2024-06-03T01:21:15.590 INFO:tasks.workunit.client.0.smithi032.stdout:3/730: creat de/d29/d2e/d31/d3c/d50/d56/dc7/ff1 x:0 0 0 2024-06-03T01:21:15.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/811: dread dc/d25/d64/fbd [0,4194304] 0 2024-06-03T01:21:15.593 INFO:tasks.workunit.client.0.smithi032.stdout:3/731: creat de/d29/d2e/d31/ff2 x:0 0 0 2024-06-03T01:21:15.594 INFO:tasks.workunit.client.0.smithi032.stdout:8/812: creat dc/d25/d64/d8b/ff8 x:0 0 0 2024-06-03T01:21:15.594 INFO:tasks.workunit.client.0.smithi032.stdout:8/813: read - dc/d25/d28/d33/fe6 zero size 2024-06-03T01:21:15.595 INFO:tasks.workunit.client.0.smithi032.stdout:8/814: fdatasync dc/de/d1e/ff3 0 2024-06-03T01:21:15.596 INFO:tasks.workunit.client.0.smithi032.stdout:8/815: mknod dc/d25/d28/d51/d5d/cf9 0 2024-06-03T01:21:15.596 INFO:tasks.workunit.client.0.smithi032.stdout:8/816: creat dc/d93/ffa x:0 0 0 2024-06-03T01:21:15.600 INFO:tasks.workunit.client.0.smithi032.stdout:8/817: mkdir dc/de/dfb 0 2024-06-03T01:21:15.600 INFO:tasks.workunit.client.0.smithi032.stdout:8/818: write dc/de/f9f [291820,94298] 0 2024-06-03T01:21:15.600 INFO:tasks.workunit.client.0.smithi032.stdout:8/819: readlink dc/de/l65 0 2024-06-03T01:21:15.600 INFO:tasks.workunit.client.0.smithi032.stdout:8/820: stat dc/d14/lcb 0 2024-06-03T01:21:15.602 INFO:tasks.workunit.client.0.smithi032.stdout:3/732: dread de/d29/d2e/d31/f21 [0,4194304] 0 2024-06-03T01:21:15.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/733: read de/f11 [4667229,2398] 0 2024-06-03T01:21:15.608 INFO:tasks.workunit.client.0.smithi032.stdout:8/821: dread dc/d25/d28/d58/f86 [0,4194304] 0 2024-06-03T01:21:15.609 INFO:tasks.workunit.client.0.smithi032.stdout:8/822: rmdir dc/de/d1e/d43 39 2024-06-03T01:21:15.609 INFO:tasks.workunit.client.0.smithi032.stdout:8/823: readlink dc/d14/l52 0 2024-06-03T01:21:15.613 INFO:tasks.workunit.client.0.smithi032.stdout:3/734: write de/d29/d2e/d31/d3c/d50/d56/f5c [3390945,93486] 0 2024-06-03T01:21:15.613 INFO:tasks.workunit.client.0.smithi032.stdout:3/735: chown de/d29/d2e/d31/d24/fc8 2798 1 2024-06-03T01:21:15.613 INFO:tasks.workunit.client.0.smithi032.stdout:3/736: fsync de/d29/f3f 0 2024-06-03T01:21:15.621 INFO:tasks.workunit.client.0.smithi032.stdout:8/824: rename dc/d25/d64 to dc/de/d1e/d43/d9b/dfc 0 2024-06-03T01:21:15.623 INFO:tasks.workunit.client.0.smithi032.stdout:8/825: dread dc/d25/d28/fd5 [0,4194304] 0 2024-06-03T01:21:15.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/826: link dc/d14/lcb dc/de/d1e/d43/d9b/dfc/lfd 0 2024-06-03T01:21:15.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/827: write dc/d25/d28/d58/d8d/d94/d59/fd1 [797472,65101] 0 2024-06-03T01:21:15.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/828: fsync dc/d25/d28/d58/f86 0 2024-06-03T01:21:15.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/829: creat dc/d25/d28/d58/d8d/d94/ffe x:0 0 0 2024-06-03T01:21:15.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/830: chown dc/de/d1e/l5b 7 1 2024-06-03T01:21:15.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/831: link dc/d25/d28/d51/d5d/d5e/f84 dc/d14/d31/fff 0 2024-06-03T01:21:15.631 INFO:tasks.workunit.client.0.smithi032.stdout:3/737: dwrite de/d29/d2e/d31/d23/d53/d69/dca/fcc [0,4194304] 0 2024-06-03T01:21:15.631 INFO:tasks.workunit.client.0.smithi032.stdout:3/738: chown de/d29/d2e/d31/d23/d53/dbf/fc4 28 1 2024-06-03T01:21:15.633 INFO:tasks.workunit.client.0.smithi032.stdout:8/832: dread dc/d25/d28/d3c/f77 [0,4194304] 0 2024-06-03T01:21:15.636 INFO:tasks.workunit.client.0.smithi032.stdout:8/833: creat dc/d14/d31/f100 x:0 0 0 2024-06-03T01:21:15.636 INFO:tasks.workunit.client.0.smithi032.stdout:8/834: stat dc/de/d1e/d43/d9b/dfc/lfd 0 2024-06-03T01:21:15.643 INFO:tasks.workunit.client.0.smithi032.stdout:8/835: write dc/d14/f2d [706349,121672] 0 2024-06-03T01:21:15.645 INFO:tasks.workunit.client.0.smithi032.stdout:8/836: truncate dc/d25/d28/d58/d8d/d94/d59/fc9 135660 0 2024-06-03T01:21:15.645 INFO:tasks.workunit.client.0.smithi032.stdout:8/837: fsync dc/d25/d28/d58/d8d/d94/da6/fed 0 2024-06-03T01:21:15.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/838: dread dc/de/d1e/d43/d9b/dfc/fbd [0,4194304] 0 2024-06-03T01:21:15.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/839: write dc/d25/d28/d58/d8d/d94/ffe [834640,92329] 0 2024-06-03T01:21:15.650 INFO:tasks.workunit.client.0.smithi032.stdout:8/840: fsync dc/de/d1e/d43/d9b/d76/fab 0 2024-06-03T01:21:15.651 INFO:tasks.workunit.client.0.smithi032.stdout:8/841: symlink dc/d25/l101 0 2024-06-03T01:21:15.651 INFO:tasks.workunit.client.0.smithi032.stdout:8/842: dread - dc/d25/d28/d58/d8d/d94/fe7 zero size 2024-06-03T01:21:15.651 INFO:tasks.workunit.client.0.smithi032.stdout:8/843: write dc/de/d1e/d43/d9b/dfc/f89 [2757013,42714] 0 2024-06-03T01:21:15.652 INFO:tasks.workunit.client.0.smithi032.stdout:8/844: link dc/de/l2c dc/de/d1e/l102 0 2024-06-03T01:21:15.652 INFO:tasks.workunit.client.0.smithi032.stdout:8/845: chown dc/d14/lcb 30303109 1 2024-06-03T01:21:15.653 INFO:tasks.workunit.client.0.smithi032.stdout:8/846: creat dc/d25/d28/f103 x:0 0 0 2024-06-03T01:21:15.654 INFO:tasks.workunit.client.0.smithi032.stdout:8/847: creat dc/d25/d28/d32/dbb/f104 x:0 0 0 2024-06-03T01:21:15.654 INFO:tasks.workunit.client.0.smithi032.stdout:8/848: write dc/d25/d28/d58/d8d/d94/d59/fd1 [849388,98635] 0 2024-06-03T01:21:15.660 INFO:tasks.workunit.client.0.smithi032.stdout:8/849: dread dc/d25/d28/d58/d8d/d94/d59/fd1 [0,4194304] 0 2024-06-03T01:21:15.660 INFO:tasks.workunit.client.0.smithi032.stdout:8/850: write dc/d25/d28/d32/fa2 [1238893,13208] 0 2024-06-03T01:21:15.661 INFO:tasks.workunit.client.0.smithi032.stdout:8/851: mkdir dc/d14/d31/d3b/d105 0 2024-06-03T01:21:15.662 INFO:tasks.workunit.client.0.smithi032.stdout:8/852: creat dc/de/d1e/d43/d9b/dd7/f106 x:0 0 0 2024-06-03T01:21:15.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/853: write dc/d25/d28/f103 [328899,45961] 0 2024-06-03T01:21:15.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/854: chown dc/d25/f6c 2369 1 2024-06-03T01:21:15.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/855: mknod dc/de/d1e/d43/c107 0 2024-06-03T01:21:15.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/856: link dc/d25/d28/d51/c53 dc/d25/d28/d33/c108 0 2024-06-03T01:21:15.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/857: chown dc/fd 11926 1 2024-06-03T01:21:15.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/858: creat dc/d25/d28/d58/d8d/d94/d59/f109 x:0 0 0 2024-06-03T01:21:15.665 INFO:tasks.workunit.client.0.smithi032.stdout:8/859: fdatasync dc/d25/d28/d58/d8d/d94/d59/fc9 0 2024-06-03T01:21:15.667 INFO:tasks.workunit.client.0.smithi032.stdout:8/860: creat dc/de/d1e/d43/f10a x:0 0 0 2024-06-03T01:21:15.671 INFO:tasks.workunit.client.0.smithi032.stdout:8/861: write f2 [3209924,7218] 0 2024-06-03T01:21:15.671 INFO:tasks.workunit.client.0.smithi032.stdout:8/862: truncate dc/de/d1e/d43/d9b/dfc/db5/fd3 284376 0 2024-06-03T01:21:15.672 INFO:tasks.workunit.client.0.smithi032.stdout:8/863: mkdir dc/d14/d31/d3b/d10b 0 2024-06-03T01:21:15.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/864: truncate dc/d25/d28/d33/d78/f82 3302306 0 2024-06-03T01:21:15.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/865: creat dc/d25/d28/d58/d8d/d94/da6/f10c x:0 0 0 2024-06-03T01:21:15.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/866: creat dc/d25/d28/d33/d78/f10d x:0 0 0 2024-06-03T01:21:15.693 INFO:tasks.workunit.client.0.smithi032.stdout:8/867: dwrite dc/de/d1e/d43/d9b/d76/dae/fb3 [0,4194304] 0 2024-06-03T01:21:15.693 INFO:tasks.workunit.client.0.smithi032.stdout:8/868: readlink dc/d25/d28/l4d 0 2024-06-03T01:21:15.693 INFO:tasks.workunit.client.0.smithi032.stdout:8/869: read dc/d14/f2d [1394466,95395] 0 2024-06-03T01:21:15.693 INFO:tasks.workunit.client.0.smithi032.stdout:8/870: chown dc/de/d1e/d43/d9b/dfc/fa4 166 1 2024-06-03T01:21:15.704 INFO:tasks.workunit.client.0.smithi032.stdout:8/871: dread dc/fd [4194304,4194304] 0 2024-06-03T01:21:15.704 INFO:tasks.workunit.client.0.smithi032.stdout:8/872: write dc/d25/d28/d58/d8d/d94/d59/fc9 [234311,112706] 0 2024-06-03T01:21:15.705 INFO:tasks.workunit.client.0.smithi032.stdout:8/873: creat dc/de/f10e x:0 0 0 2024-06-03T01:21:15.705 INFO:tasks.workunit.client.0.smithi032.stdout:8/874: chown dc/fd 113146857 1 2024-06-03T01:21:15.705 INFO:tasks.workunit.client.0.smithi032.stdout:8/875: symlink dc/d25/d28/l10f 0 2024-06-03T01:21:15.714 INFO:tasks.workunit.client.0.smithi032.stdout:8/876: write dc/de/f37 [1197362,41682] 0 2024-06-03T01:21:15.719 INFO:tasks.workunit.client.0.smithi032.stdout:8/877: dread dc/fd [4194304,4194304] 0 2024-06-03T01:21:15.720 INFO:tasks.workunit.client.0.smithi032.stdout:8/878: getdents dc/de/d1e/d43/d9b/d76/dae 0 2024-06-03T01:21:15.735 INFO:tasks.workunit.client.0.smithi032.stdout:8/879: dwrite dc/d25/d28/d33/d78/f7e [0,4194304] 0 2024-06-03T01:21:15.738 INFO:tasks.workunit.client.0.smithi032.stdout:8/880: creat dc/d14/de3/f110 x:0 0 0 2024-06-03T01:21:15.740 INFO:tasks.workunit.client.0.smithi032.stdout:8/881: unlink dc/d14/l4f 0 2024-06-03T01:21:15.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/882: mkdir dc/d93/d111 0 2024-06-03T01:21:15.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/883: write dc/d25/d28/d58/d8d/d94/fe7 [231607,81831] 0 2024-06-03T01:21:15.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/884: fsync dc/d25/d28/d32/f50 0 2024-06-03T01:21:15.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/885: readlink dc/de/d1e/l4a 0 2024-06-03T01:21:15.750 INFO:tasks.workunit.client.0.smithi032.stdout:8/886: dread dc/d25/f6c [4194304,4194304] 0 2024-06-03T01:21:15.751 INFO:tasks.workunit.client.0.smithi032.stdout:8/887: dread dc/de/d1e/d43/d9b/dfc/db5/fd3 [0,4194304] 0 2024-06-03T01:21:15.754 INFO:tasks.workunit.client.0.smithi032.stdout:8/888: dread dc/d93/fde [0,4194304] 0 2024-06-03T01:21:15.759 INFO:tasks.workunit.client.0.smithi032.stdout:2/982: sync 2024-06-03T01:21:15.762 INFO:tasks.workunit.client.0.smithi032.stdout:8/889: write dc/d25/d28/d3c/f5a [3982165,66433] 0 2024-06-03T01:21:15.762 INFO:tasks.workunit.client.0.smithi032.stdout:2/983: dread d5/da/f39 [0,4194304] 0 2024-06-03T01:21:15.762 INFO:tasks.workunit.client.0.smithi032.stdout:2/984: fsync d5/da/f29 0 2024-06-03T01:21:15.764 INFO:tasks.workunit.client.0.smithi032.stdout:2/985: symlink d5/d38/d46/d53/da9/l146 0 2024-06-03T01:21:15.768 INFO:tasks.workunit.client.0.smithi032.stdout:8/890: dread dc/de/d1e/d43/d9b/fe9 [0,4194304] 0 2024-06-03T01:21:15.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/986: mkdir d5/d38/d46/d53/d57/d6d/dce/d8f/d147 0 2024-06-03T01:21:15.821 INFO:tasks.workunit.client.0.smithi032.stdout:8/891: rmdir dc/de/d1e/d43 39 2024-06-03T01:21:15.824 INFO:tasks.workunit.client.0.smithi032.stdout:2/987: dread d5/d38/d46/d5c/d6c/d89/d10c/f9b [0,4194304] 0 2024-06-03T01:21:15.828 INFO:tasks.workunit.client.0.smithi032.stdout:2/988: link d5/d38/d46/d5c/f8b d5/d38/d46/dbe/d141/f148 0 2024-06-03T01:21:15.828 INFO:tasks.workunit.client.0.smithi032.stdout:2/989: chown d5/d38/d46/f81 68845 1 2024-06-03T01:21:15.831 INFO:tasks.workunit.client.0.smithi032.stdout:2/990: write d5/d38/d46/d5c/d6c/d89/d10c/f9b [1271505,16725] 0 2024-06-03T01:21:15.832 INFO:tasks.workunit.client.0.smithi032.stdout:2/991: stat d5/d38/d46/dbe/d141/f148 0 2024-06-03T01:21:15.832 INFO:tasks.workunit.client.0.smithi032.stdout:2/992: write d5/d38/d46/d53/da9/f100 [330796,100190] 0 2024-06-03T01:21:15.832 INFO:tasks.workunit.client.0.smithi032.stdout:2/993: stat d5/d38/d46/d53/da9/f100 0 2024-06-03T01:21:15.833 INFO:tasks.workunit.client.0.smithi032.stdout:2/994: symlink d5/d38/d46/d53/d57/d6d/dce/d67/d104/dea/l149 0 2024-06-03T01:21:15.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/995: dread d5/d38/d46/d5c/d6c/d89/f93 [0,4194304] 0 2024-06-03T01:21:15.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/996: stat d5/da/f2c 0 2024-06-03T01:21:15.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/997: read d5/d38/d46/d53/da9/f100 [154034,91798] 0 2024-06-03T01:21:15.836 INFO:tasks.workunit.client.0.smithi032.stdout:8/892: dwrite dc/f97 [4194304,4194304] 0 2024-06-03T01:21:15.836 INFO:tasks.workunit.client.0.smithi032.stdout:2/998: mknod d5/d38/d46/d53/d57/d6d/c14a 0 2024-06-03T01:21:15.837 INFO:tasks.workunit.client.0.smithi032.stdout:2/999: creat d5/d38/d46/d53/d57/dac/de0/f14b x:0 0 0 2024-06-03T01:21:15.837 INFO:tasks.workunit.client.0.smithi032.stdout:8/893: mknod dc/de/d1e/d43/d9b/dd7/c112 0 2024-06-03T01:21:15.841 INFO:tasks.workunit.client.0.smithi032.stdout:8/894: dread dc/de/f91 [0,4194304] 0 2024-06-03T01:21:15.841 INFO:tasks.workunit.client.0.smithi032.stdout:8/895: read dc/de/d1e/d43/d9b/dfc/f89 [3580666,55954] 0 2024-06-03T01:21:15.843 INFO:tasks.workunit.client.0.smithi032.stdout:8/896: rename dc/de/f91 to dc/de/d1e/d43/d9b/f113 0 2024-06-03T01:21:15.851 INFO:tasks.workunit.client.0.smithi032.stdout:8/897: write fa [1835483,128013] 0 2024-06-03T01:21:15.851 INFO:tasks.workunit.client.0.smithi032.stdout:8/898: chown dc/d25/d28/d32/f7d 2 1 2024-06-03T01:21:15.867 INFO:tasks.workunit.client.0.smithi032.stdout:8/899: dwrite dc/de/d1e/d43/d9b/d76/dae/fdc [0,4194304] 0 2024-06-03T01:21:15.867 INFO:tasks.workunit.client.0.smithi032.stdout:8/900: chown dc/de/d1e/d43/cec 2031865 1 2024-06-03T01:21:15.869 INFO:tasks.workunit.client.0.smithi032.stdout:8/901: link dc/de/d1e/d43/d9b/f9c dc/d25/d28/d33/d78/f114 0 2024-06-03T01:21:15.870 INFO:tasks.workunit.client.0.smithi032.stdout:8/902: chown dc/d25/d28/d33/d78/f7e 30693 1 2024-06-03T01:21:15.870 INFO:tasks.workunit.client.0.smithi032.stdout:8/903: mknod dc/d93/c115 0 2024-06-03T01:21:15.871 INFO:tasks.workunit.client.0.smithi032.stdout:8/904: symlink dc/d25/d28/d51/d5d/d5e/l116 0 2024-06-03T01:21:15.880 INFO:tasks.workunit.client.0.smithi032.stdout:8/905: dread dc/d25/d28/d51/fc2 [0,4194304] 0 2024-06-03T01:21:15.880 INFO:tasks.workunit.client.0.smithi032.stdout:8/906: dread - dc/d25/d28/d32/dbb/f104 zero size 2024-06-03T01:21:15.881 INFO:tasks.workunit.client.0.smithi032.stdout:8/907: rename dc/d93/la5 to dc/d25/d28/d58/d8d/l117 0 2024-06-03T01:21:15.882 INFO:tasks.workunit.client.0.smithi032.stdout:8/908: symlink dc/d25/d28/d58/l118 0 2024-06-03T01:21:15.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/909: dwrite dc/de/d1e/d43/d9b/f113 [0,4194304] 0 2024-06-03T01:21:15.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/910: truncate dc/de/f9f 1076705 0 2024-06-03T01:21:15.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/911: fsync dc/d25/d28/d3c/f3f 0 2024-06-03T01:21:15.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/912: write dc/d25/ddb/fdf [5017140,77325] 0 2024-06-03T01:21:15.900 INFO:tasks.workunit.client.0.smithi032.stdout:0/898: sync 2024-06-03T01:21:15.900 INFO:tasks.workunit.client.0.smithi032.stdout:0/899: write d1/da/d10/d29/f96 [1374593,11233] 0 2024-06-03T01:21:15.902 INFO:tasks.workunit.client.0.smithi032.stdout:8/913: rename dc/d25/d28/d51 to dc/de/df0/d119 0 2024-06-03T01:21:15.904 INFO:tasks.workunit.client.0.smithi032.stdout:8/914: mkdir dc/d14/de3/d11a 0 2024-06-03T01:21:15.913 INFO:tasks.workunit.client.0.smithi032.stdout:8/915: dread dc/d14/d31/fff [0,4194304] 0 2024-06-03T01:21:15.914 INFO:tasks.workunit.client.0.smithi032.stdout:0/900: dwrite d1/da/d28/f8b [0,4194304] 0 2024-06-03T01:21:15.914 INFO:tasks.workunit.client.0.smithi032.stdout:0/901: chown d1/da/d28/f121 0 1 2024-06-03T01:21:15.915 INFO:tasks.workunit.client.0.smithi032.stdout:0/902: getdents d1/da/d10/d80 0 2024-06-03T01:21:15.917 INFO:tasks.workunit.client.0.smithi032.stdout:0/903: rename d1/da/d10/d80/d9f/d117 to d1/da/d10/d29/d6c/d127 0 2024-06-03T01:21:15.921 INFO:tasks.workunit.client.0.smithi032.stdout:8/916: truncate dc/d25/ddb/fdf 4507149 0 2024-06-03T01:21:15.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/904: dwrite d1/da/d28/d3b/d5d/da9/f11d [0,4194304] 0 2024-06-03T01:21:15.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/905: write d1/da/d28/d3b/d68/f120 [51086,52469] 0 2024-06-03T01:21:15.937 INFO:tasks.workunit.client.0.smithi032.stdout:0/906: getdents d1/da/d28/d3b/d68/ddd 0 2024-06-03T01:21:15.937 INFO:tasks.workunit.client.0.smithi032.stdout:0/907: write d1/da/d28/d3b/d50/ff3 [498453,130265] 0 2024-06-03T01:21:15.938 INFO:tasks.workunit.client.0.smithi032.stdout:0/908: symlink d1/d46/dc3/l128 0 2024-06-03T01:21:15.941 INFO:tasks.workunit.client.0.smithi032.stdout:0/909: link d1/da/dd2/d11b/c125 d1/da/d28/d3b/d5d/c129 0 2024-06-03T01:21:15.942 INFO:tasks.workunit.client.0.smithi032.stdout:0/910: read - d1/da/d28/d3b/d50/f8a zero size 2024-06-03T01:21:15.942 INFO:tasks.workunit.client.0.smithi032.stdout:0/911: truncate d1/da/d28/d3b/d50/f8a 853823 0 2024-06-03T01:21:15.942 INFO:tasks.workunit.client.0.smithi032.stdout:0/912: chown d1/da/db2 216552568 1 2024-06-03T01:21:15.942 INFO:tasks.workunit.client.0.smithi032.stdout:0/913: fsync d1/da/d28/d3b/d68/db5/fff 0 2024-06-03T01:21:15.948 INFO:tasks.workunit.client.0.smithi032.stdout:0/914: mknod d1/da/d10/d29/d10e/c12a 0 2024-06-03T01:21:15.948 INFO:tasks.workunit.client.0.smithi032.stdout:0/915: dread - d1/da/d28/d3b/d5d/d8d/d9c/fa0 zero size 2024-06-03T01:21:15.963 INFO:tasks.workunit.client.0.smithi032.stdout:0/916: dwrite d1/f113 [0,4194304] 0 2024-06-03T01:21:15.966 INFO:tasks.workunit.client.0.smithi032.stdout:0/917: symlink d1/da/d10/l12b 0 2024-06-03T01:21:15.968 INFO:tasks.workunit.client.0.smithi032.stdout:0/918: mkdir d1/da/d28/d3b/d68/d82/d12c 0 2024-06-03T01:21:15.968 INFO:tasks.workunit.client.0.smithi032.stdout:0/919: chown d1/da/d28/d3b/l86 2 1 2024-06-03T01:21:15.969 INFO:tasks.workunit.client.0.smithi032.stdout:0/920: creat d1/da/dd2/d10f/f12d x:0 0 0 2024-06-03T01:21:15.969 INFO:tasks.workunit.client.0.smithi032.stdout:0/921: stat d1/da/d28/d3b/d5d/d8d/d9c 0 2024-06-03T01:21:15.970 INFO:tasks.workunit.client.0.smithi032.stdout:0/922: write d1/da/d28/d3b/d50/ff3 [414335,90844] 0 2024-06-03T01:21:15.970 INFO:tasks.workunit.client.0.smithi032.stdout:0/923: write d1/da/d28/d3b/d5d/d8d/d9c/f102 [395098,115045] 0 2024-06-03T01:21:15.972 INFO:tasks.workunit.client.0.smithi032.stdout:0/924: mkdir d1/da/d28/d3b/d5d/d8d/d9c/dc2/d12e 0 2024-06-03T01:21:15.972 INFO:tasks.workunit.client.0.smithi032.stdout:0/925: stat d1/da/d28/d3b/d50/l5e 0 2024-06-03T01:21:15.974 INFO:tasks.workunit.client.0.smithi032.stdout:0/926: creat d1/da/d10/d80/de5/f12f x:0 0 0 2024-06-03T01:21:15.975 INFO:tasks.workunit.client.0.smithi032.stdout:0/927: getdents d1/da/dd2/d11b 0 2024-06-03T01:21:15.975 INFO:tasks.workunit.client.0.smithi032.stdout:0/928: chown d1/da/d10/d80/de5 719 1 2024-06-03T01:21:15.978 INFO:tasks.workunit.client.0.smithi032.stdout:0/929: rename d1/da/d28/d3b/d5d/d8d/d9c/fa0 to d1/da/d28/d3b/d68/f130 0 2024-06-03T01:21:15.978 INFO:tasks.workunit.client.0.smithi032.stdout:0/930: chown d1/cc1 25199489 1 2024-06-03T01:21:15.978 INFO:tasks.workunit.client.0.smithi032.stdout:0/931: truncate d1/da/d28/d3b/d5d/da9/ffa 217013 0 2024-06-03T01:21:15.979 INFO:tasks.workunit.client.0.smithi032.stdout:0/932: creat d1/da/d10/d29/d6c/d127/f131 x:0 0 0 2024-06-03T01:21:15.980 INFO:tasks.workunit.client.0.smithi032.stdout:0/933: symlink d1/da/d10/d29/d6c/d127/l132 0 2024-06-03T01:21:15.981 INFO:tasks.workunit.client.0.smithi032.stdout:0/934: write d1/da/d28/d3b/d5d/d8d/dbe/dc4/fe8 [1936468,102946] 0 2024-06-03T01:21:15.996 INFO:tasks.workunit.client.0.smithi032.stdout:0/935: dwrite d1/da/d10/d29/d6c/d127/f131 [0,4194304] 0 2024-06-03T01:21:15.997 INFO:tasks.workunit.client.0.smithi032.stdout:0/936: chown d1/da/dd2/c10c 2928 1 2024-06-03T01:21:15.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/937: creat d1/da/d28/d3b/d50/dec/f133 x:0 0 0 2024-06-03T01:21:15.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/938: creat d1/da/d10/d80/f134 x:0 0 0 2024-06-03T01:21:16.001 INFO:tasks.workunit.client.0.smithi032.stdout:0/939: mknod d1/da/d28/c135 0 2024-06-03T01:21:16.001 INFO:tasks.workunit.client.0.smithi032.stdout:0/940: creat d1/da/dd2/d10f/f136 x:0 0 0 2024-06-03T01:21:16.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/941: creat d1/da/d28/d3b/d68/ddd/f137 x:0 0 0 2024-06-03T01:21:16.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/942: creat d1/da/dd2/d10f/f138 x:0 0 0 2024-06-03T01:21:16.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/943: write d1/da/d10/f4f [4959666,61168] 0 2024-06-03T01:21:16.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/944: creat d1/da/d28/f139 x:0 0 0 2024-06-03T01:21:16.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/945: creat d1/da/d28/d3b/d50/dec/f13a x:0 0 0 2024-06-03T01:21:16.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/946: creat d1/da/d28/d3b/d5d/f13b x:0 0 0 2024-06-03T01:21:16.007 INFO:tasks.workunit.client.0.smithi032.stdout:0/947: creat d1/da/d28/d3b/d50/dec/f13c x:0 0 0 2024-06-03T01:21:16.007 INFO:tasks.workunit.client.0.smithi032.stdout:0/948: chown d1/da/c23 964 1 2024-06-03T01:21:16.012 INFO:tasks.workunit.client.0.smithi032.stdout:0/949: mkdir d1/da/d10/d80/de5/d13d 0 2024-06-03T01:21:16.014 INFO:tasks.workunit.client.0.smithi032.stdout:0/950: mkdir d1/da/d10/d29/d6c/d13e 0 2024-06-03T01:21:16.014 INFO:tasks.workunit.client.0.smithi032.stdout:0/951: chown d1/da/d10/cf7 1014469273 1 2024-06-03T01:21:16.014 INFO:tasks.workunit.client.0.smithi032.stdout:0/952: readlink d1/da/d10/l22 0 2024-06-03T01:21:16.015 INFO:tasks.workunit.client.0.smithi032.stdout:0/953: chown d1/da/d10/d80/d9f/dac 67 1 2024-06-03T01:21:16.015 INFO:tasks.workunit.client.0.smithi032.stdout:0/954: chown d1/da/dd2/d10f 6 1 2024-06-03T01:21:16.028 INFO:tasks.workunit.client.0.smithi032.stdout:0/955: dwrite d1/da/d28/d3b/d68/fbf [4194304,4194304] 0 2024-06-03T01:21:16.029 INFO:tasks.workunit.client.0.smithi032.stdout:0/956: symlink d1/l13f 0 2024-06-03T01:21:16.030 INFO:tasks.workunit.client.0.smithi032.stdout:0/957: write d1/da/d28/d3b/d5d/f65 [1150351,9064] 0 2024-06-03T01:21:16.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/958: dwrite d1/da/d28/d3b/d68/d82/ff8 [0,4194304] 0 2024-06-03T01:21:16.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/959: write d1/da/d28/d3b/d68/f130 [878364,30202] 0 2024-06-03T01:21:16.050 INFO:tasks.workunit.client.0.smithi032.stdout:0/960: rmdir d1/da/d28/d3b/d5d 39 2024-06-03T01:21:16.071 INFO:tasks.workunit.client.0.smithi032.stdout:0/961: dwrite d1/da/d28/d3b/d5d/d8d/f91 [0,4194304] 0 2024-06-03T01:21:16.134 INFO:tasks.workunit.client.0.smithi032.stdout:0/962: mkdir d1/d46/d140 0 2024-06-03T01:21:16.134 INFO:tasks.workunit.client.0.smithi032.stdout:0/963: creat d1/da/d28/f141 x:0 0 0 2024-06-03T01:21:16.137 INFO:tasks.workunit.client.0.smithi032.stdout:0/964: mknod d1/da/db2/d110/c142 0 2024-06-03T01:21:16.142 INFO:tasks.workunit.client.0.smithi032.stdout:0/965: getdents d1/d46 0 2024-06-03T01:21:16.160 INFO:tasks.workunit.client.0.smithi032.stdout:0/966: dwrite d1/da/d28/d3b/f76 [0,4194304] 0 2024-06-03T01:21:16.162 INFO:tasks.workunit.client.0.smithi032.stdout:0/967: unlink d1/da/d10/f5b 0 2024-06-03T01:21:16.254 INFO:tasks.workunit.client.0.smithi032.stdout:5/991: sync 2024-06-03T01:21:16.254 INFO:tasks.workunit.client.0.smithi032.stdout:1/788: sync 2024-06-03T01:21:16.282 INFO:tasks.workunit.client.0.smithi032.stdout:5/992: dwrite f4 [0,4194304] 0 2024-06-03T01:21:16.283 INFO:tasks.workunit.client.0.smithi032.stdout:1/789: dwrite d18/f19 [0,4194304] 0 2024-06-03T01:21:16.285 INFO:tasks.workunit.client.0.smithi032.stdout:5/993: creat da/dd/df/d10/dbc/d18/d1a/d30/f13f x:0 0 0 2024-06-03T01:21:16.287 INFO:tasks.workunit.client.0.smithi032.stdout:5/994: rename da/dd/df/d10/dbc/d18/db9/fe4 to da/dd/de6/d7a/d136/f140 0 2024-06-03T01:21:16.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/790: dread d18/d3f/d5f/f84 [0,4194304] 0 2024-06-03T01:21:16.293 INFO:tasks.workunit.client.0.smithi032.stdout:1/791: creat d18/d3f/d5f/d6b/d78/d3d/feb x:0 0 0 2024-06-03T01:21:16.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/792: symlink d18/dc0/de6/lec 0 2024-06-03T01:21:16.295 INFO:tasks.workunit.client.0.smithi032.stdout:1/793: rename d18/d27/d2b/fb1 to d18/d3f/d5f/d6b/fed 0 2024-06-03T01:21:16.296 INFO:tasks.workunit.client.0.smithi032.stdout:1/794: rename d18/d2d/d5e/le5 to d18/d2d/lee 0 2024-06-03T01:21:16.297 INFO:tasks.workunit.client.0.smithi032.stdout:1/795: symlink d18/dc0/lef 0 2024-06-03T01:21:16.298 INFO:tasks.workunit.client.0.smithi032.stdout:1/796: creat d18/d3f/d5f/da6/ff0 x:0 0 0 2024-06-03T01:21:16.299 INFO:tasks.workunit.client.0.smithi032.stdout:1/797: symlink d18/d3f/d5f/d6b/d78/d3d/d47/d6e/lf1 0 2024-06-03T01:21:16.300 INFO:tasks.workunit.client.0.smithi032.stdout:1/798: mknod d18/d3f/d5f/d6b/d78/d3d/d6a/cf2 0 2024-06-03T01:21:16.306 INFO:tasks.workunit.client.0.smithi032.stdout:5/995: dwrite da/dab/f118 [0,4194304] 0 2024-06-03T01:21:16.309 INFO:tasks.workunit.client.0.smithi032.stdout:5/996: link da/dd/df/d10/dbc/d18/d1a/l11f da/dd/df/d10/dbc/d25/l141 0 2024-06-03T01:21:16.311 INFO:tasks.workunit.client.0.smithi032.stdout:5/997: link da/dd/df/d10/dbc/d18/c108 da/c142 0 2024-06-03T01:21:16.313 INFO:tasks.workunit.client.0.smithi032.stdout:5/998: unlink da/dd/df/d10/dbc/d18/d35/d44/dc3/f111 0 2024-06-03T01:21:16.314 INFO:tasks.workunit.client.0.smithi032.stdout:1/799: dwrite d18/d3f/d5f/d6b/d78/fb8 [0,4194304] 0 2024-06-03T01:21:16.315 INFO:tasks.workunit.client.0.smithi032.stdout:5/999: link da/dd/df/d10/dbc/l90 da/dab/de2/l143 0 2024-06-03T01:21:16.323 INFO:tasks.workunit.client.0.smithi032.stdout:1/800: dread d18/d3f/fcc [0,4194304] 0 2024-06-03T01:21:16.343 INFO:tasks.workunit.client.0.smithi032.stdout:1/801: dwrite d18/d3f/d5f/d6b/d9d/fac [0,4194304] 0 2024-06-03T01:21:16.345 INFO:tasks.workunit.client.0.smithi032.stdout:1/802: mkdir d18/d3f/d5f/d6b/d78/d3d/df3 0 2024-06-03T01:21:16.345 INFO:tasks.workunit.client.0.smithi032.stdout:1/803: chown d18/d32/d3c/f67 3795 1 2024-06-03T01:21:16.346 INFO:tasks.workunit.client.0.smithi032.stdout:1/804: rename d18/d2d/d5e/fe0 to d18/d3f/d5f/d7b/ff4 0 2024-06-03T01:21:16.534 INFO:tasks.workunit.client.0.smithi032.stdout:4/799: sync 2024-06-03T01:21:16.534 INFO:tasks.workunit.client.0.smithi032.stdout:4/800: write d4/db/da/d37/f3b [495535,87250] 0 2024-06-03T01:21:16.534 INFO:tasks.workunit.client.0.smithi032.stdout:4/801: fdatasync d4/db/da/d37/f84 0 2024-06-03T01:21:16.539 INFO:tasks.workunit.client.0.smithi032.stdout:4/802: rename d4/db/da/d1d/d6b/df1/d110 to d4/db/da/d2d/d27/d8c/db5/d111 0 2024-06-03T01:21:16.547 INFO:tasks.workunit.client.0.smithi032.stdout:4/803: dread d4/db/da/f7d [0,4194304] 0 2024-06-03T01:21:16.549 INFO:tasks.workunit.client.0.smithi032.stdout:4/804: mkdir d4/d57/d6c/d112 0 2024-06-03T01:21:16.549 INFO:tasks.workunit.client.0.smithi032.stdout:4/805: chown d4/db/da/d2d/c52 252831 1 2024-06-03T01:21:16.550 INFO:tasks.workunit.client.0.smithi032.stdout:4/806: symlink d4/db/da/d2d/de1/l113 0 2024-06-03T01:21:16.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/807: write d4/db/da/d37/f7b [2575025,125847] 0 2024-06-03T01:21:16.561 INFO:tasks.workunit.client.0.smithi032.stdout:4/808: truncate d4/db/da/d5d/d8e/ff6 714756 0 2024-06-03T01:21:16.562 INFO:tasks.workunit.client.0.smithi032.stdout:4/809: creat d4/db/da/d2d/f114 x:0 0 0 2024-06-03T01:21:16.562 INFO:tasks.workunit.client.0.smithi032.stdout:4/810: dread - d4/db/da/f10e zero size 2024-06-03T01:21:16.563 INFO:tasks.workunit.client.0.smithi032.stdout:4/811: stat d4/db/da/d37/f7b 0 2024-06-03T01:21:16.563 INFO:tasks.workunit.client.0.smithi032.stdout:4/812: chown d4/db/da/f3d 403 1 2024-06-03T01:21:16.567 INFO:tasks.workunit.client.0.smithi032.stdout:4/813: write d4/d57/d6c/f6e [1678169,62714] 0 2024-06-03T01:21:16.568 INFO:tasks.workunit.client.0.smithi032.stdout:4/814: mknod d4/db/da/d37/dab/c115 0 2024-06-03T01:21:16.569 INFO:tasks.workunit.client.0.smithi032.stdout:4/815: rmdir d4/db/da/d2d/d27/d53 39 2024-06-03T01:21:16.571 INFO:tasks.workunit.client.0.smithi032.stdout:4/816: rename d4/db/da/d2d/c35 to d4/db/da/d37/df0/c116 0 2024-06-03T01:21:16.571 INFO:tasks.workunit.client.0.smithi032.stdout:4/817: readlink d4/l6d 0 2024-06-03T01:21:16.573 INFO:tasks.workunit.client.0.smithi032.stdout:4/818: link d4/db/da/d37/dab/dec/dfd/ddd/cde d4/db/da/d5d/c117 0 2024-06-03T01:21:16.588 INFO:tasks.workunit.client.0.smithi032.stdout:4/819: dwrite d4/db/da/d2d/d27/d8c/db5/fbd [0,4194304] 0 2024-06-03T01:21:16.590 INFO:tasks.workunit.client.0.smithi032.stdout:4/820: creat d4/db/da/d1d/f118 x:0 0 0 2024-06-03T01:21:16.607 INFO:tasks.workunit.client.0.smithi032.stdout:4/821: dwrite d4/db/da/f80 [0,4194304] 0 2024-06-03T01:21:16.615 INFO:tasks.workunit.client.0.smithi032.stdout:4/822: dread d4/db/da/f2c [0,4194304] 0 2024-06-03T01:21:16.616 INFO:tasks.workunit.client.0.smithi032.stdout:4/823: truncate d4/db/da/d37/f48 3063236 0 2024-06-03T01:21:16.617 INFO:tasks.workunit.client.0.smithi032.stdout:4/824: symlink d4/db/da/d2d/d27/d8c/db5/d111/l119 0 2024-06-03T01:21:16.618 INFO:tasks.workunit.client.0.smithi032.stdout:4/825: stat d4/db/da/d1d/d6b/c109 0 2024-06-03T01:21:16.619 INFO:tasks.workunit.client.0.smithi032.stdout:4/826: symlink d4/db/da/l11a 0 2024-06-03T01:21:16.637 INFO:tasks.workunit.client.0.smithi032.stdout:4/827: dwrite d4/db/da/d1d/f4c [0,4194304] 0 2024-06-03T01:21:16.659 INFO:tasks.workunit.client.0.smithi032.stdout:4/828: dwrite d4/d57/ffc [4194304,4194304] 0 2024-06-03T01:21:16.681 INFO:tasks.workunit.client.0.smithi032.stdout:4/829: dwrite d4/db/da/d1d/d6b/f7c [0,4194304] 0 2024-06-03T01:21:16.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/830: rename d4/db/da/d2d/d27/d8c/db5/ld1 to d4/db/da/l11b 0 2024-06-03T01:21:16.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/831: creat d4/d5c/f11c x:0 0 0 2024-06-03T01:21:16.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/832: readlink d4/db/da/d37/l76 0 2024-06-03T01:21:16.688 INFO:tasks.workunit.client.0.smithi032.stdout:4/833: mknod d4/db/c11d 0 2024-06-03T01:21:16.688 INFO:tasks.workunit.client.0.smithi032.stdout:4/834: readlink d4/db/da/d37/l88 0 2024-06-03T01:21:16.689 INFO:tasks.workunit.client.0.smithi032.stdout:4/835: truncate d4/db/da/d1d/f33 4164304 0 2024-06-03T01:21:16.689 INFO:tasks.workunit.client.0.smithi032.stdout:4/836: write d4/db/ff [4216534,108252] 0 2024-06-03T01:21:16.691 INFO:tasks.workunit.client.0.smithi032.stdout:4/837: mkdir d4/db/da/d37/d90/da1/d11e 0 2024-06-03T01:21:16.692 INFO:tasks.workunit.client.0.smithi032.stdout:4/838: mknod d4/db/da/d37/dab/dec/dfd/d102/c11f 0 2024-06-03T01:21:16.694 INFO:tasks.workunit.client.0.smithi032.stdout:6/804: sync 2024-06-03T01:21:16.694 INFO:tasks.workunit.client.0.smithi032.stdout:7/823: sync 2024-06-03T01:21:16.694 INFO:tasks.workunit.client.0.smithi032.stdout:1/805: sync 2024-06-03T01:21:16.694 INFO:tasks.workunit.client.0.smithi032.stdout:0/968: sync 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:8/917: sync 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:9/821: sync 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/739: sync 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/740: fsync de/d29/f2b 0 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/741: chown de/d29/f3f 790467673 1 2024-06-03T01:21:16.695 INFO:tasks.workunit.client.0.smithi032.stdout:4/839: dread d4/db/da/d37/f84 [4194304,4194304] 0 2024-06-03T01:21:16.697 INFO:tasks.workunit.client.0.smithi032.stdout:7/824: dread dc/d12/d22/f52 [0,4194304] 0 2024-06-03T01:21:16.698 INFO:tasks.workunit.client.0.smithi032.stdout:6/805: unlink d8/d50/fac 0 2024-06-03T01:21:16.700 INFO:tasks.workunit.client.0.smithi032.stdout:1/806: unlink d18/d3f/d5f/d94/fa5 0 2024-06-03T01:21:16.702 INFO:tasks.workunit.client.0.smithi032.stdout:0/969: mkdir d1/da/dd2/d143 0 2024-06-03T01:21:16.702 INFO:tasks.workunit.client.0.smithi032.stdout:0/970: write d1/da/d28/d3b/d68/db5/fd7 [357195,83409] 0 2024-06-03T01:21:16.703 INFO:tasks.workunit.client.0.smithi032.stdout:9/822: creat d0/d23/d24/f10d x:0 0 0 2024-06-03T01:21:16.706 INFO:tasks.workunit.client.0.smithi032.stdout:3/742: creat de/d29/d2e/d31/d23/d53/d69/d90/ff3 x:0 0 0 2024-06-03T01:21:16.708 INFO:tasks.workunit.client.0.smithi032.stdout:3/743: dread de/f5b [0,4194304] 0 2024-06-03T01:21:16.709 INFO:tasks.workunit.client.0.smithi032.stdout:4/840: unlink d4/db/da/d2d/d27/d53/dea/c107 0 2024-06-03T01:21:16.711 INFO:tasks.workunit.client.0.smithi032.stdout:4/841: dread d4/db/da/d37/d90/f95 [0,4194304] 0 2024-06-03T01:21:16.713 INFO:tasks.workunit.client.0.smithi032.stdout:6/806: link d8/d11/d20/d3b/d66/fc3 d8/d11/d20/d31/d3f/ffe 0 2024-06-03T01:21:16.715 INFO:tasks.workunit.client.0.smithi032.stdout:7/825: creat dc/d12/d22/d42/f101 x:0 0 0 2024-06-03T01:21:16.715 INFO:tasks.workunit.client.0.smithi032.stdout:7/826: truncate dc/d12/d22/d42/f101 413043 0 2024-06-03T01:21:16.715 INFO:tasks.workunit.client.0.smithi032.stdout:7/827: chown dc/d12/d24/d7b/f20 91617855 1 2024-06-03T01:21:16.716 INFO:tasks.workunit.client.0.smithi032.stdout:4/842: dread d4/db/f1c [0,4194304] 0 2024-06-03T01:21:16.717 INFO:tasks.workunit.client.0.smithi032.stdout:1/807: stat d18/d3f/d5f/d6b/d9d/la9 0 2024-06-03T01:21:16.718 INFO:tasks.workunit.client.0.smithi032.stdout:9/823: rmdir d0/d11/d1d/d38/d4f 39 2024-06-03T01:21:16.718 INFO:tasks.workunit.client.0.smithi032.stdout:9/824: dread - d0/d23/fb5 zero size 2024-06-03T01:21:16.721 INFO:tasks.workunit.client.0.smithi032.stdout:4/843: mknod d4/db/da/d37/df0/c120 0 2024-06-03T01:21:16.723 INFO:tasks.workunit.client.0.smithi032.stdout:1/808: symlink d18/d3f/d5f/d7b/da8/lf5 0 2024-06-03T01:21:16.723 INFO:tasks.workunit.client.0.smithi032.stdout:9/825: dread d0/d11/d1d/da6/fa2 [0,4194304] 0 2024-06-03T01:21:16.723 INFO:tasks.workunit.client.0.smithi032.stdout:9/826: write d0/f1 [2005371,17062] 0 2024-06-03T01:21:16.723 INFO:tasks.workunit.client.0.smithi032.stdout:9/827: fsync d0/d23/d25/dd6/f108 0 2024-06-03T01:21:16.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/971: dwrite d1/f113 [0,4194304] 0 2024-06-03T01:21:16.727 INFO:tasks.workunit.client.0.smithi032.stdout:3/744: dread de/d29/d2e/d31/d78/f86 [0,4194304] 0 2024-06-03T01:21:16.727 INFO:tasks.workunit.client.0.smithi032.stdout:6/807: link d8/d55/c73 d8/d55/cff 0 2024-06-03T01:21:16.727 INFO:tasks.workunit.client.0.smithi032.stdout:6/808: write d8/d11/d20/d7f/f91 [3809357,97735] 0 2024-06-03T01:21:16.729 INFO:tasks.workunit.client.0.smithi032.stdout:7/828: mkdir dc/d12/d22/d42/d102 0 2024-06-03T01:21:16.731 INFO:tasks.workunit.client.0.smithi032.stdout:1/809: creat d18/d27/da7/ff6 x:0 0 0 2024-06-03T01:21:16.734 INFO:tasks.workunit.client.0.smithi032.stdout:0/972: rename d1/da/d10/c70 to d1/da/d10/d29/d10e/c144 0 2024-06-03T01:21:16.736 INFO:tasks.workunit.client.0.smithi032.stdout:6/809: symlink d8/d11/d20/d2b/d39/l100 0 2024-06-03T01:21:16.737 INFO:tasks.workunit.client.0.smithi032.stdout:6/810: write f4 [523498,62517] 0 2024-06-03T01:21:16.740 INFO:tasks.workunit.client.0.smithi032.stdout:0/973: symlink d1/da/d28/d3b/d68/d82/l145 0 2024-06-03T01:21:16.742 INFO:tasks.workunit.client.0.smithi032.stdout:1/810: dread d18/d3f/d5f/d6b/d78/f5c [0,4194304] 0 2024-06-03T01:21:16.743 INFO:tasks.workunit.client.0.smithi032.stdout:3/745: rmdir de/d29/d2e/d31/d24 39 2024-06-03T01:21:16.743 INFO:tasks.workunit.client.0.smithi032.stdout:3/746: stat de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:16.743 INFO:tasks.workunit.client.0.smithi032.stdout:3/747: write de/d63/f76 [748371,75698] 0 2024-06-03T01:21:16.743 INFO:tasks.workunit.client.0.smithi032.stdout:3/748: truncate de/fe6 1276244 0 2024-06-03T01:21:16.744 INFO:tasks.workunit.client.0.smithi032.stdout:4/844: dwrite d4/db/da/d37/d90/f95 [0,4194304] 0 2024-06-03T01:21:16.746 INFO:tasks.workunit.client.0.smithi032.stdout:3/749: dread de/d29/d2e/d31/d3c/d50/d66/f9a [0,4194304] 0 2024-06-03T01:21:16.747 INFO:tasks.workunit.client.0.smithi032.stdout:3/750: readlink de/d29/d2e/d31/d23/l93 0 2024-06-03T01:21:16.747 INFO:tasks.workunit.client.0.smithi032.stdout:7/829: dwrite dc/ff9 [0,4194304] 0 2024-06-03T01:21:16.747 INFO:tasks.workunit.client.0.smithi032.stdout:6/811: creat d8/d11/d20/d31/d3f/f101 x:0 0 0 2024-06-03T01:21:16.747 INFO:tasks.workunit.client.0.smithi032.stdout:6/812: write d8/d11/d20/d2b/d39/fa4 [910251,61790] 0 2024-06-03T01:21:16.753 INFO:tasks.workunit.client.0.smithi032.stdout:4/845: link d4/db/da/d1d/le6 d4/db/da/d2d/d27/d8c/db5/d111/l121 0 2024-06-03T01:21:16.753 INFO:tasks.workunit.client.0.smithi032.stdout:4/846: stat d4/db/da/d2d/d27/d53/c61 0 2024-06-03T01:21:16.755 INFO:tasks.workunit.client.0.smithi032.stdout:3/751: creat de/d29/d2e/d31/d3c/d50/ff4 x:0 0 0 2024-06-03T01:21:16.755 INFO:tasks.workunit.client.0.smithi032.stdout:3/752: dread - de/d29/d2e/d31/da1/ff0 zero size 2024-06-03T01:21:16.758 INFO:tasks.workunit.client.0.smithi032.stdout:7/830: link dc/d12/d2a/d2f/d63/le2 dc/d12/ddb/l103 0 2024-06-03T01:21:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/813: mkdir d8/d11/d20/d21/d9a/d102 0 2024-06-03T01:21:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/814: creat d8/d4c/daa/f103 x:0 0 0 2024-06-03T01:21:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/815: fsync d8/d11/d20/d21/f27 0 2024-06-03T01:21:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/816: dread - d8/d50/dda/fce zero size 2024-06-03T01:21:16.763 INFO:tasks.workunit.client.0.smithi032.stdout:4/847: link d4/db/da/d2d/d27/d8c/db5/d111/l121 d4/db/da/d37/dab/dec/l122 0 2024-06-03T01:21:16.766 INFO:tasks.workunit.client.0.smithi032.stdout:3/753: link de/d29/d2e/lc2 de/d29/d2e/d31/lf5 0 2024-06-03T01:21:16.766 INFO:tasks.workunit.client.0.smithi032.stdout:3/754: creat de/d29/d2e/d31/d78/ff6 x:0 0 0 2024-06-03T01:21:16.766 INFO:tasks.workunit.client.0.smithi032.stdout:3/755: creat de/d29/d2e/d31/d3c/d50/ff7 x:0 0 0 2024-06-03T01:21:16.767 INFO:tasks.workunit.client.0.smithi032.stdout:7/831: dread dc/d12/d2a/d2f/d63/d69/f5f [0,4194304] 0 2024-06-03T01:21:16.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/817: mkdir d8/dd5/da5/dc8/d104 0 2024-06-03T01:21:16.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/818: creat d8/d50/f105 x:0 0 0 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:4/848: mknod d4/db/da/d1d/d6b/c123 0 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:4/849: chown d4/db/le 48813820 1 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/811: dwrite d18/f8e [0,4194304] 0 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/812: stat d18/d3f/d5f/d94 0 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/813: creat d18/d32/ff7 x:0 0 0 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/814: read - d18/d27/da7/ff6 zero size 2024-06-03T01:21:16.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/815: readlink d18/d27/d2b/l44 0 2024-06-03T01:21:16.772 INFO:tasks.workunit.client.0.smithi032.stdout:0/974: dwrite d1/da/d28/d3b/d5d/fa5 [0,4194304] 0 2024-06-03T01:21:16.779 INFO:tasks.workunit.client.0.smithi032.stdout:3/756: dread de/d29/d2e/d31/d24/fa5 [0,4194304] 0 2024-06-03T01:21:16.779 INFO:tasks.workunit.client.0.smithi032.stdout:3/757: readlink de/d29/l87 0 2024-06-03T01:21:16.782 INFO:tasks.workunit.client.0.smithi032.stdout:6/819: link d8/d11/d20/d3b/fd1 d8/d11/d20/d31/df2/df8/f106 0 2024-06-03T01:21:16.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/832: dread dc/f3d [0,4194304] 0 2024-06-03T01:21:16.782 INFO:tasks.workunit.client.0.smithi032.stdout:0/975: symlink d1/da/d28/d3b/d5d/d8d/d9c/dc2/l146 0 2024-06-03T01:21:16.782 INFO:tasks.workunit.client.0.smithi032.stdout:0/976: read d1/da/d10/d53/d105/f11a [3487276,76443] 0 2024-06-03T01:21:16.786 INFO:tasks.workunit.client.0.smithi032.stdout:7/833: rename dc/d12/d6a/d86/d88/f98 to dc/d12/d22/d42/d61/f104 0 2024-06-03T01:21:16.787 INFO:tasks.workunit.client.0.smithi032.stdout:0/977: unlink d1/da/d10/l12b 0 2024-06-03T01:21:16.790 INFO:tasks.workunit.client.0.smithi032.stdout:7/834: mkdir dc/d12/d6a/d86/dc8/d105 0 2024-06-03T01:21:16.791 INFO:tasks.workunit.client.0.smithi032.stdout:0/978: rmdir d1/da/d28/d3b/d5d/da9/daa 39 2024-06-03T01:21:16.791 INFO:tasks.workunit.client.0.smithi032.stdout:4/850: dwrite d4/db/da/d2d/d27/f4f [4194304,4194304] 0 2024-06-03T01:21:16.791 INFO:tasks.workunit.client.0.smithi032.stdout:4/851: write d4/db/da/d2d/d27/d8c/db5/fb0 [103774,112132] 0 2024-06-03T01:21:16.792 INFO:tasks.workunit.client.0.smithi032.stdout:1/816: dwrite d18/d32/fdf [0,4194304] 0 2024-06-03T01:21:16.794 INFO:tasks.workunit.client.0.smithi032.stdout:4/852: rename d4/db/c14 to d4/db/da/d2d/d27/d8c/c124 0 2024-06-03T01:21:16.798 INFO:tasks.workunit.client.0.smithi032.stdout:1/817: truncate d18/d32/d39/d60/f72 1620799 0 2024-06-03T01:21:16.799 INFO:tasks.workunit.client.0.smithi032.stdout:6/820: dwrite d8/d11/d20/d21/f27 [0,4194304] 0 2024-06-03T01:21:16.801 INFO:tasks.workunit.client.0.smithi032.stdout:1/818: mkdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/df8 0 2024-06-03T01:21:16.801 INFO:tasks.workunit.client.0.smithi032.stdout:7/835: dread dc/d12/d22/d42/d61/ddd/f36 [0,4194304] 0 2024-06-03T01:21:16.801 INFO:tasks.workunit.client.0.smithi032.stdout:7/836: chown dc/d12/d6a/c91 18659 1 2024-06-03T01:21:16.801 INFO:tasks.workunit.client.0.smithi032.stdout:7/837: creat dc/f106 x:0 0 0 2024-06-03T01:21:16.802 INFO:tasks.workunit.client.0.smithi032.stdout:6/821: mkdir d8/d11/d20/d31/df2/df8/d107 0 2024-06-03T01:21:16.805 INFO:tasks.workunit.client.0.smithi032.stdout:1/819: mkdir d18/d3f/d5f/d6b/d78/de3/df9 0 2024-06-03T01:21:16.805 INFO:tasks.workunit.client.0.smithi032.stdout:0/979: write d1/da/f4d [3716038,24071] 0 2024-06-03T01:21:16.805 INFO:tasks.workunit.client.0.smithi032.stdout:0/980: readlink d1/l7 0 2024-06-03T01:21:16.808 INFO:tasks.workunit.client.0.smithi032.stdout:0/981: mknod d1/da/d10/d29/d6c/d127/c147 0 2024-06-03T01:21:16.811 INFO:tasks.workunit.client.0.smithi032.stdout:0/982: read d1/da/d28/d3b/d5d/d8d/d9c/f102 [97362,20233] 0 2024-06-03T01:21:16.814 INFO:tasks.workunit.client.0.smithi032.stdout:0/983: creat d1/da/d10/d100/f148 x:0 0 0 2024-06-03T01:21:16.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/853: dwrite d4/f6 [4194304,4194304] 0 2024-06-03T01:21:16.818 INFO:tasks.workunit.client.0.smithi032.stdout:4/854: mkdir d4/db/da/d37/d90/d125 0 2024-06-03T01:21:16.823 INFO:tasks.workunit.client.0.smithi032.stdout:7/838: dwrite dc/d12/d22/fad [0,4194304] 0 2024-06-03T01:21:16.826 INFO:tasks.workunit.client.0.smithi032.stdout:1/820: dwrite d18/d3f/d5f/d6b/fd5 [0,4194304] 0 2024-06-03T01:21:16.829 INFO:tasks.workunit.client.0.smithi032.stdout:0/984: getdents d1/da/d28/d3b/d5d/d8d/d9c/dc2 0 2024-06-03T01:21:16.834 INFO:tasks.workunit.client.0.smithi032.stdout:4/855: dwrite d4/d5c/f5e [0,4194304] 0 2024-06-03T01:21:16.834 INFO:tasks.workunit.client.0.smithi032.stdout:4/856: truncate d4/db/da/d1d/f118 318047 0 2024-06-03T01:21:16.838 INFO:tasks.workunit.client.0.smithi032.stdout:1/821: write d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 [3225167,85294] 0 2024-06-03T01:21:16.843 INFO:tasks.workunit.client.0.smithi032.stdout:4/857: write d4/db/da/d1d/d6b/f7c [3052851,41765] 0 2024-06-03T01:21:16.845 INFO:tasks.workunit.client.0.smithi032.stdout:7/839: dwrite dc/d12/d22/d42/d61/ddd/f21 [0,4194304] 0 2024-06-03T01:21:16.847 INFO:tasks.workunit.client.0.smithi032.stdout:1/822: rename d18/dc0 to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa 0 2024-06-03T01:21:16.849 INFO:tasks.workunit.client.0.smithi032.stdout:7/840: unlink dc/d12/d2a/d2f/lec 0 2024-06-03T01:21:16.852 INFO:tasks.workunit.client.0.smithi032.stdout:4/858: rename d4/db/da/d5d/ff3 to d4/db/da/d37/d90/f126 0 2024-06-03T01:21:16.852 INFO:tasks.workunit.client.0.smithi032.stdout:4/859: chown d4/db/da/d5d/d8e/ff6 308448028 1 2024-06-03T01:21:16.854 INFO:tasks.workunit.client.0.smithi032.stdout:1/823: unlink d18/d27/d89/fbf 0 2024-06-03T01:21:16.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/841: mknod dc/d12/d2a/d2f/c107 0 2024-06-03T01:21:16.856 INFO:tasks.workunit.client.0.smithi032.stdout:4/860: symlink d4/db/da/d2d/d27/d53/dea/l127 0 2024-06-03T01:21:16.858 INFO:tasks.workunit.client.0.smithi032.stdout:1/824: rename d18/d3f/d5f/d6b/d78/d3d/d47/d6e/fde to d18/d3f/d5f/d7b/ffb 0 2024-06-03T01:21:16.862 INFO:tasks.workunit.client.0.smithi032.stdout:1/825: chown d18/d3f/d5f/d6b/d9d 5897486 1 2024-06-03T01:21:16.862 INFO:tasks.workunit.client.0.smithi032.stdout:1/826: read d18/f19 [4338467,73566] 0 2024-06-03T01:21:16.863 INFO:tasks.workunit.client.0.smithi032.stdout:7/842: mkdir dc/d12/d24/d7b/d108 0 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:7/843: truncate dc/d12/d22/d42/f101 547981 0 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:7/844: write dc/f14 [2677362,61399] 0 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:4/861: creat d4/db/f128 x:0 0 0 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:4/862: chown d4/d57/d6c/d112 14157462 1 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:4/863: chown d4/db/da/d2d/f9f 8 1 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:4/864: truncate d4/db/da/d37/dab/dec/dfd/ddd/f103 864676 0 2024-06-03T01:21:16.864 INFO:tasks.workunit.client.0.smithi032.stdout:0/985: rmdir d1/da/d10 39 2024-06-03T01:21:16.865 INFO:tasks.workunit.client.0.smithi032.stdout:1/827: mkdir d18/d3f/d5f/d6b/d78/d3d/df3/dfc 0 2024-06-03T01:21:16.865 INFO:tasks.workunit.client.0.smithi032.stdout:7/845: creat dc/d12/d24/d33/daa/dda/f109 x:0 0 0 2024-06-03T01:21:16.865 INFO:tasks.workunit.client.0.smithi032.stdout:4/865: mknod d4/d57/d6c/c129 0 2024-06-03T01:21:16.866 INFO:tasks.workunit.client.0.smithi032.stdout:4/866: chown d4/d57 1024612081 1 2024-06-03T01:21:16.866 INFO:tasks.workunit.client.0.smithi032.stdout:4/867: fsync d4/db/da/d1d/faf 0 2024-06-03T01:21:16.866 INFO:tasks.workunit.client.0.smithi032.stdout:0/986: creat d1/f149 x:0 0 0 2024-06-03T01:21:16.868 INFO:tasks.workunit.client.0.smithi032.stdout:4/868: mknod d4/d7f/c12a 0 2024-06-03T01:21:16.869 INFO:tasks.workunit.client.0.smithi032.stdout:0/987: mkdir d1/da/dd2/d11b/d14a 0 2024-06-03T01:21:16.870 INFO:tasks.workunit.client.0.smithi032.stdout:0/988: write d1/da/d10/d29/fd0 [4942538,25836] 0 2024-06-03T01:21:16.870 INFO:tasks.workunit.client.0.smithi032.stdout:0/989: dread - d1/da/db2/f109 zero size 2024-06-03T01:21:16.872 INFO:tasks.workunit.client.0.smithi032.stdout:7/846: write dc/f15 [2482740,51954] 0 2024-06-03T01:21:16.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/990: rename d1/da/d28/d3b/d68/fa3 to d1/da/d28/d3b/d68/d82/d12c/f14b 0 2024-06-03T01:21:16.877 INFO:tasks.workunit.client.0.smithi032.stdout:0/991: read d1/da/d28/d3b/d68/f130 [248682,501] 0 2024-06-03T01:21:16.881 INFO:tasks.workunit.client.0.smithi032.stdout:7/847: link dc/d12/d2a/d2f/d63/fe7 dc/d12/d22/d42/d102/f10a 0 2024-06-03T01:21:16.887 INFO:tasks.workunit.client.0.smithi032.stdout:7/848: rename dc/d12/d2a/d2f/d76 to dc/d12/d24/d33/d10b 0 2024-06-03T01:21:16.890 INFO:tasks.workunit.client.0.smithi032.stdout:4/869: dwrite d4/db/da/d37/d90/f126 [0,4194304] 0 2024-06-03T01:21:16.892 INFO:tasks.workunit.client.0.smithi032.stdout:4/870: link d4/db/da/d37/dab/dec/dfd/ddd/fd5 d4/db/da/d2d/d27/d8c/f12b 0 2024-06-03T01:21:16.904 INFO:tasks.workunit.client.0.smithi032.stdout:1/828: dwrite d18/d3f/d5f/d98/fa2 [0,4194304] 0 2024-06-03T01:21:16.907 INFO:tasks.workunit.client.0.smithi032.stdout:1/829: symlink d18/d3f/d5f/da6/de1/lfd 0 2024-06-03T01:21:16.907 INFO:tasks.workunit.client.0.smithi032.stdout:1/830: fdatasync d18/f1a 0 2024-06-03T01:21:16.909 INFO:tasks.workunit.client.0.smithi032.stdout:1/831: getdents d18/d3f/d5f/d94 0 2024-06-03T01:21:16.909 INFO:tasks.workunit.client.0.smithi032.stdout:1/832: fsync d18/d32/d39/f4a 0 2024-06-03T01:21:16.909 INFO:tasks.workunit.client.0.smithi032.stdout:1/833: truncate d18/d27/fb7 690420 0 2024-06-03T01:21:16.910 INFO:tasks.workunit.client.0.smithi032.stdout:0/992: dwrite d1/da/d10/d29/f45 [0,4194304] 0 2024-06-03T01:21:16.911 INFO:tasks.workunit.client.0.smithi032.stdout:1/834: mkdir d18/d3f/d5f/d6b/d78/de3/df9/dfe 0 2024-06-03T01:21:16.914 INFO:tasks.workunit.client.0.smithi032.stdout:4/871: dwrite d4/db/da/f82 [0,4194304] 0 2024-06-03T01:21:16.914 INFO:tasks.workunit.client.0.smithi032.stdout:7/849: dwrite dc/d12/d22/fc0 [0,4194304] 0 2024-06-03T01:21:16.914 INFO:tasks.workunit.client.0.smithi032.stdout:7/850: stat dc/d12/d6a/d86/dc8/d105 0 2024-06-03T01:21:16.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/872: symlink d4/d5c/l12c 0 2024-06-03T01:21:16.918 INFO:tasks.workunit.client.0.smithi032.stdout:4/873: creat d4/db/da/d37/d90/da1/d11e/f12d x:0 0 0 2024-06-03T01:21:16.920 INFO:tasks.workunit.client.0.smithi032.stdout:7/851: rmdir dc/d12/d6a/d86/d89 39 2024-06-03T01:21:16.920 INFO:tasks.workunit.client.0.smithi032.stdout:7/852: chown dc/d12/d24/d33/d10b/l79 336015 1 2024-06-03T01:21:16.920 INFO:tasks.workunit.client.0.smithi032.stdout:7/853: chown dc/d12/d2a/d2f/c8a 2 1 2024-06-03T01:21:16.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/874: mkdir d4/db/da/d37/d90/da1/d12e 0 2024-06-03T01:21:16.927 INFO:tasks.workunit.client.0.smithi032.stdout:1/835: dwrite d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b [0,4194304] 0 2024-06-03T01:21:16.928 INFO:tasks.workunit.client.0.smithi032.stdout:0/993: write d1/da/d10/d29/f45 [4673935,129651] 0 2024-06-03T01:21:16.930 INFO:tasks.workunit.client.0.smithi032.stdout:4/875: dread d4/db/da/d37/f3b [0,4194304] 0 2024-06-03T01:21:16.932 INFO:tasks.workunit.client.0.smithi032.stdout:1/836: unlink d18/l1f 0 2024-06-03T01:21:16.932 INFO:tasks.workunit.client.0.smithi032.stdout:1/837: read d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [243695,90560] 0 2024-06-03T01:21:16.934 INFO:tasks.workunit.client.0.smithi032.stdout:0/994: unlink d1/da/d28/d3b/d5d/da9/daa/lad 0 2024-06-03T01:21:16.935 INFO:tasks.workunit.client.0.smithi032.stdout:1/838: rmdir d18/d3f/d5f/d98 39 2024-06-03T01:21:16.938 INFO:tasks.workunit.client.0.smithi032.stdout:0/995: creat d1/da/d10/d100/f14c x:0 0 0 2024-06-03T01:21:16.939 INFO:tasks.workunit.client.0.smithi032.stdout:7/854: dwrite dc/f2d [0,4194304] 0 2024-06-03T01:21:16.939 INFO:tasks.workunit.client.0.smithi032.stdout:0/996: truncate d1/da/d28/d3b/d68/db5/fde 162016 0 2024-06-03T01:21:16.939 INFO:tasks.workunit.client.0.smithi032.stdout:0/997: dread - d1/da/d28/f141 zero size 2024-06-03T01:21:16.941 INFO:tasks.workunit.client.0.smithi032.stdout:7/855: rename dc/d12/d22/d42/d61/f104 to dc/d12/d24/d33/d10b/f10c 0 2024-06-03T01:21:16.944 INFO:tasks.workunit.client.0.smithi032.stdout:0/998: rename d1/da/d28/d3b/d68/cb1 to d1/da/d28/d3b/dce/dd4/c14d 0 2024-06-03T01:21:16.944 INFO:tasks.workunit.client.0.smithi032.stdout:0/999: chown d1/da/dd2/d10f 2947 1 2024-06-03T01:21:16.946 INFO:tasks.workunit.client.0.smithi032.stdout:7/856: unlink dc/d12/d24/d33/ldc 0 2024-06-03T01:21:16.948 INFO:tasks.workunit.client.0.smithi032.stdout:7/857: mknod dc/d12/d6a/d86/d88/d8b/c10d 0 2024-06-03T01:21:16.948 INFO:tasks.workunit.client.0.smithi032.stdout:4/876: dwrite d4/db/da/d37/d90/f95 [4194304,4194304] 0 2024-06-03T01:21:16.952 INFO:tasks.workunit.client.0.smithi032.stdout:4/877: link d4/d57/d6c/fcf d4/db/da/d37/d90/f12f 0 2024-06-03T01:21:16.953 INFO:tasks.workunit.client.0.smithi032.stdout:4/878: getdents d4/db/da/d37/d90/da1/d12e 0 2024-06-03T01:21:16.954 INFO:tasks.workunit.client.0.smithi032.stdout:4/879: getdents d4/db/da/d2d/d27/d53 0 2024-06-03T01:21:16.959 INFO:tasks.workunit.client.0.smithi032.stdout:7/858: write dc/d12/d24/d33/daa/fe1 [608236,98136] 0 2024-06-03T01:21:16.961 INFO:tasks.workunit.client.0.smithi032.stdout:1/839: dwrite d18/d32/d39/fc5 [0,4194304] 0 2024-06-03T01:21:16.963 INFO:tasks.workunit.client.0.smithi032.stdout:1/840: mkdir d18/d2d/dff 0 2024-06-03T01:21:16.963 INFO:tasks.workunit.client.0.smithi032.stdout:1/841: write d18/d27/fd0 [194719,60998] 0 2024-06-03T01:21:16.967 INFO:tasks.workunit.client.0.smithi032.stdout:4/880: dread d4/db/da/f49 [0,4194304] 0 2024-06-03T01:21:16.968 INFO:tasks.workunit.client.0.smithi032.stdout:4/881: mkdir d4/db/da/d37/df0/d130 0 2024-06-03T01:21:16.970 INFO:tasks.workunit.client.0.smithi032.stdout:1/842: write d18/d32/d39/d60/fd3 [4701330,121962] 0 2024-06-03T01:21:16.980 INFO:tasks.workunit.client.0.smithi032.stdout:7/859: dwrite dc/f30 [0,4194304] 0 2024-06-03T01:21:16.983 INFO:tasks.workunit.client.0.smithi032.stdout:7/860: rename dc/d12/d2a/fd6 to dc/d12/d22/d42/f10e 0 2024-06-03T01:21:16.983 INFO:tasks.workunit.client.0.smithi032.stdout:7/861: chown dc/d12/d6a/d86/d89 484678 1 2024-06-03T01:21:16.983 INFO:tasks.workunit.client.0.smithi032.stdout:4/882: dread d4/db/da/d1d/f4c [0,4194304] 0 2024-06-03T01:21:16.984 INFO:tasks.workunit.client.0.smithi032.stdout:7/862: symlink dc/dbe/l10f 0 2024-06-03T01:21:16.991 INFO:tasks.workunit.client.0.smithi032.stdout:7/863: write dc/d12/d22/fad [3478775,36712] 0 2024-06-03T01:21:16.994 INFO:tasks.workunit.client.0.smithi032.stdout:4/883: dread d4/db/f68 [0,4194304] 0 2024-06-03T01:21:16.994 INFO:tasks.workunit.client.0.smithi032.stdout:1/843: dwrite d18/d3f/d5f/d6b/d78/f5c [0,4194304] 0 2024-06-03T01:21:16.997 INFO:tasks.workunit.client.0.smithi032.stdout:4/884: mkdir d4/db/da/d37/dab/dec/d131 0 2024-06-03T01:21:16.997 INFO:tasks.workunit.client.0.smithi032.stdout:4/885: dread - d4/db/da/d2d/d27/d8c/db5/fef zero size 2024-06-03T01:21:16.998 INFO:tasks.workunit.client.0.smithi032.stdout:1/844: symlink d18/d3f/d5f/d6b/d78/d3d/d6a/l100 0 2024-06-03T01:21:17.005 INFO:tasks.workunit.client.0.smithi032.stdout:4/886: mkdir d4/db/da/d37/dab/dec/dfd/d102/d132 0 2024-06-03T01:21:17.005 INFO:tasks.workunit.client.0.smithi032.stdout:4/887: chown d4/db/da/d5d/cbb 26984 1 2024-06-03T01:21:17.007 INFO:tasks.workunit.client.0.smithi032.stdout:4/888: creat d4/db/da/d37/dab/dec/d131/f133 x:0 0 0 2024-06-03T01:21:17.009 INFO:tasks.workunit.client.0.smithi032.stdout:1/845: getdents d18/d27/da7 0 2024-06-03T01:21:17.011 INFO:tasks.workunit.client.0.smithi032.stdout:4/889: rmdir d4/d57/d6c/d112 0 2024-06-03T01:21:17.013 INFO:tasks.workunit.client.0.smithi032.stdout:4/890: link d4/db/da/d37/f48 d4/db/da/d37/dab/dec/f134 0 2024-06-03T01:21:17.017 INFO:tasks.workunit.client.0.smithi032.stdout:4/891: creat d4/db/da/d2d/d27/f135 x:0 0 0 2024-06-03T01:21:17.017 INFO:tasks.workunit.client.0.smithi032.stdout:1/846: dread d18/f80 [0,4194304] 0 2024-06-03T01:21:17.017 INFO:tasks.workunit.client.0.smithi032.stdout:1/847: write d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/fb2 [589309,22096] 0 2024-06-03T01:21:17.019 INFO:tasks.workunit.client.0.smithi032.stdout:4/892: rename d4/db/da/d5d/l10f to d4/db/da/d37/d90/l136 0 2024-06-03T01:21:17.019 INFO:tasks.workunit.client.0.smithi032.stdout:4/893: creat d4/db/da/d2d/f137 x:0 0 0 2024-06-03T01:21:17.020 INFO:tasks.workunit.client.0.smithi032.stdout:1/848: creat d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/f101 x:0 0 0 2024-06-03T01:21:17.022 INFO:tasks.workunit.client.0.smithi032.stdout:4/894: creat d4/db/da/d2d/de1/f138 x:0 0 0 2024-06-03T01:21:17.022 INFO:tasks.workunit.client.0.smithi032.stdout:4/895: write d4/db/da/d2d/d27/f89 [5012035,60151] 0 2024-06-03T01:21:17.024 INFO:tasks.workunit.client.0.smithi032.stdout:1/849: unlink d18/d27/d2b/c65 0 2024-06-03T01:21:17.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/896: mkdir d4/db/da/d37/dab/dec/d139 0 2024-06-03T01:21:17.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/897: creat d4/db/da/d2d/d27/f13a x:0 0 0 2024-06-03T01:21:17.025 INFO:tasks.workunit.client.0.smithi032.stdout:4/898: fsync d4/db/da/d1d/d6b/feb 0 2024-06-03T01:21:17.026 INFO:tasks.workunit.client.0.smithi032.stdout:1/850: mknod d18/d2d/dff/c102 0 2024-06-03T01:21:17.028 INFO:tasks.workunit.client.0.smithi032.stdout:1/851: creat d18/d3f/d5f/d94/f103 x:0 0 0 2024-06-03T01:21:17.030 INFO:tasks.workunit.client.0.smithi032.stdout:1/852: rename d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/f101 to d18/f104 0 2024-06-03T01:21:17.033 INFO:tasks.workunit.client.0.smithi032.stdout:1/853: rmdir d18/d3f/d5f/d6b/d78/de3/df9/dfe 0 2024-06-03T01:21:17.033 INFO:tasks.workunit.client.0.smithi032.stdout:4/899: write d4/d5c/f5e [2179961,103060] 0 2024-06-03T01:21:17.033 INFO:tasks.workunit.client.0.smithi032.stdout:4/900: truncate d4/db/da/d2d/d27/d8c/db5/fef 868550 0 2024-06-03T01:21:17.034 INFO:tasks.workunit.client.0.smithi032.stdout:1/854: mknod d18/d3f/d5f/d7b/c105 0 2024-06-03T01:21:17.037 INFO:tasks.workunit.client.0.smithi032.stdout:4/901: mkdir d4/db/da/d37/d90/d13b 0 2024-06-03T01:21:17.039 INFO:tasks.workunit.client.0.smithi032.stdout:4/902: creat d4/db/da/d37/d90/d125/f13c x:0 0 0 2024-06-03T01:21:17.039 INFO:tasks.workunit.client.0.smithi032.stdout:4/903: creat d4/db/f13d x:0 0 0 2024-06-03T01:21:17.039 INFO:tasks.workunit.client.0.smithi032.stdout:4/904: write d4/f29 [4615454,21214] 0 2024-06-03T01:21:17.053 INFO:tasks.workunit.client.0.smithi032.stdout:4/905: dwrite d4/db/ff [4194304,4194304] 0 2024-06-03T01:21:17.054 INFO:tasks.workunit.client.0.smithi032.stdout:4/906: fdatasync d4/db/da/d2d/d27/f89 0 2024-06-03T01:21:17.055 INFO:tasks.workunit.client.0.smithi032.stdout:4/907: dread d4/db/da/d2d/d27/d8c/db5/fb0 [0,4194304] 0 2024-06-03T01:21:17.055 INFO:tasks.workunit.client.0.smithi032.stdout:4/908: creat d4/db/da/d2d/f13e x:0 0 0 2024-06-03T01:21:17.055 INFO:tasks.workunit.client.0.smithi032.stdout:4/909: fsync d4/db/da/d2d/d27/d8c/f12b 0 2024-06-03T01:21:17.055 INFO:tasks.workunit.client.0.smithi032.stdout:4/910: chown d4/fcc 2144 1 2024-06-03T01:21:17.057 INFO:tasks.workunit.client.0.smithi032.stdout:4/911: mkdir d4/db/da/d37/d90/d13b/d13f 0 2024-06-03T01:21:17.058 INFO:tasks.workunit.client.0.smithi032.stdout:4/912: dread d4/d7f/f8b [0,4194304] 0 2024-06-03T01:21:17.059 INFO:tasks.workunit.client.0.smithi032.stdout:4/913: truncate d4/db/fa0 2848469 0 2024-06-03T01:21:17.060 INFO:tasks.workunit.client.0.smithi032.stdout:4/914: symlink d4/db/da/d1d/d6b/l140 0 2024-06-03T01:21:17.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/915: link d4/db/da/d5d/lbc d4/d78/l141 0 2024-06-03T01:21:17.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/916: chown d4/db/da/d2d/l8d 392387 1 2024-06-03T01:21:17.064 INFO:tasks.workunit.client.0.smithi032.stdout:4/917: symlink d4/db/da/d37/dab/dec/dfd/d102/l142 0 2024-06-03T01:21:17.064 INFO:tasks.workunit.client.0.smithi032.stdout:4/918: write d4/db/da/f96 [192694,57421] 0 2024-06-03T01:21:17.064 INFO:tasks.workunit.client.0.smithi032.stdout:4/919: creat d4/db/da/d37/d90/da1/d11e/f143 x:0 0 0 2024-06-03T01:21:17.069 INFO:tasks.workunit.client.0.smithi032.stdout:4/920: mknod d4/db/da/d2d/d27/c144 0 2024-06-03T01:21:17.077 INFO:tasks.workunit.client.0.smithi032.stdout:4/921: read d4/d7f/f92 [1193287,96791] 0 2024-06-03T01:21:17.077 INFO:tasks.workunit.client.0.smithi032.stdout:4/922: dread - d4/db/da/d2d/f13e zero size 2024-06-03T01:21:17.085 INFO:tasks.workunit.client.0.smithi032.stdout:4/923: write d4/db/ff [1544059,19627] 0 2024-06-03T01:21:17.094 INFO:tasks.workunit.client.0.smithi032.stdout:4/924: dread d4/db/da/d1d/fb7 [0,4194304] 0 2024-06-03T01:21:17.098 INFO:tasks.workunit.client.0.smithi032.stdout:4/925: dread d4/d5c/f5e [0,4194304] 0 2024-06-03T01:21:17.098 INFO:tasks.workunit.client.0.smithi032.stdout:4/926: fsync d4/db/da/d1d/f4c 0 2024-06-03T01:21:17.098 INFO:tasks.workunit.client.0.smithi032.stdout:4/927: readlink d4/l9b 0 2024-06-03T01:21:17.100 INFO:tasks.workunit.client.0.smithi032.stdout:4/928: mkdir d4/db/da/d2d/d145 0 2024-06-03T01:21:17.100 INFO:tasks.workunit.client.0.smithi032.stdout:4/929: getdents d4/db/da/d5d/dc5 0 2024-06-03T01:21:17.101 INFO:tasks.workunit.client.0.smithi032.stdout:4/930: mknod d4/db/da/d5d/c146 0 2024-06-03T01:21:17.103 INFO:tasks.workunit.client.0.smithi032.stdout:4/931: creat d4/db/da/d2d/d27/d8c/f147 x:0 0 0 2024-06-03T01:21:17.104 INFO:tasks.workunit.client.0.smithi032.stdout:4/932: read d4/db/da/d2d/f9f [190641,105995] 0 2024-06-03T01:21:17.106 INFO:tasks.workunit.client.0.smithi032.stdout:4/933: creat d4/db/da/d1d/d6b/df1/f148 x:0 0 0 2024-06-03T01:21:17.107 INFO:tasks.workunit.client.0.smithi032.stdout:4/934: dread d4/db/da/f3f [0,4194304] 0 2024-06-03T01:21:17.108 INFO:tasks.workunit.client.0.smithi032.stdout:4/935: getdents d4/db/da/d2d/d27/d53/dea 0 2024-06-03T01:21:17.111 INFO:tasks.workunit.client.0.smithi032.stdout:4/936: rename d4/db/da/d5d/cbb to d4/d5c/c149 0 2024-06-03T01:21:17.112 INFO:tasks.workunit.client.0.smithi032.stdout:4/937: mknod d4/db/da/d37/d90/d125/c14a 0 2024-06-03T01:21:17.113 INFO:tasks.workunit.client.0.smithi032.stdout:4/938: rmdir d4/db/da/d37/d90/da1 39 2024-06-03T01:21:17.114 INFO:tasks.workunit.client.0.smithi032.stdout:4/939: chown d4/db/da/d2d 1001663007 1 2024-06-03T01:21:17.114 INFO:tasks.workunit.client.0.smithi032.stdout:4/940: readlink d4/db/da/d1d/d6b/l140 0 2024-06-03T01:21:17.115 INFO:tasks.workunit.client.0.smithi032.stdout:4/941: symlink d4/db/da/d2d/d27/d53/l14b 0 2024-06-03T01:21:17.117 INFO:tasks.workunit.client.0.smithi032.stdout:4/942: rename d4/db/da/d5d/fd4 to d4/db/da/d37/d90/da1/f14c 0 2024-06-03T01:21:17.118 INFO:tasks.workunit.client.0.smithi032.stdout:4/943: mknod d4/db/da/d37/dab/dec/dfd/c14d 0 2024-06-03T01:21:17.119 INFO:tasks.workunit.client.0.smithi032.stdout:4/944: creat d4/f14e x:0 0 0 2024-06-03T01:21:17.119 INFO:tasks.workunit.client.0.smithi032.stdout:4/945: mknod d4/db/da/d37/df0/d130/c14f 0 2024-06-03T01:21:17.120 INFO:tasks.workunit.client.0.smithi032.stdout:4/946: symlink d4/db/da/d37/d90/da1/l150 0 2024-06-03T01:21:17.122 INFO:tasks.workunit.client.0.smithi032.stdout:4/947: mkdir d4/d7f/d151 0 2024-06-03T01:21:17.123 INFO:tasks.workunit.client.0.smithi032.stdout:4/948: creat d4/db/da/d37/dab/dec/d139/f152 x:0 0 0 2024-06-03T01:21:17.123 INFO:tasks.workunit.client.0.smithi032.stdout:4/949: truncate d4/db/da/f80 5875941 0 2024-06-03T01:21:17.123 INFO:tasks.workunit.client.0.smithi032.stdout:4/950: chown d4/d78/l141 5344661 1 2024-06-03T01:21:17.123 INFO:tasks.workunit.client.0.smithi032.stdout:4/951: readlink d4/d5c/l12c 0 2024-06-03T01:21:17.124 INFO:tasks.workunit.client.0.smithi032.stdout:4/952: unlink d4/db/da/d2d/f13e 0 2024-06-03T01:21:17.124 INFO:tasks.workunit.client.0.smithi032.stdout:4/953: chown d4/db/da/d2d/f9f 1 1 2024-06-03T01:21:17.124 INFO:tasks.workunit.client.0.smithi032.stdout:4/954: dread - d4/db/da/d2d/de1/f138 zero size 2024-06-03T01:21:17.124 INFO:tasks.workunit.client.0.smithi032.stdout:4/955: chown d4/db/da/d37/lbf 451507 1 2024-06-03T01:21:17.125 INFO:tasks.workunit.client.0.smithi032.stdout:4/956: truncate d4/db/da/d37/d90/da1/f14c 435013 0 2024-06-03T01:21:17.125 INFO:tasks.workunit.client.0.smithi032.stdout:4/957: mkdir d4/db/da/d37/d90/da1/d11e/d153 0 2024-06-03T01:21:17.125 INFO:tasks.workunit.client.0.smithi032.stdout:4/958: write d4/db/da/d37/f7b [1772840,115277] 0 2024-06-03T01:21:17.129 INFO:tasks.workunit.client.0.smithi032.stdout:4/959: rmdir d4/db/da/d5d/dc5 39 2024-06-03T01:21:17.132 INFO:tasks.workunit.client.0.smithi032.stdout:4/960: link d4/db/da/d37/d90/da1/l83 d4/db/da/d2d/d27/d53/dea/l154 0 2024-06-03T01:21:17.132 INFO:tasks.workunit.client.0.smithi032.stdout:4/961: stat d4/f6 0 2024-06-03T01:21:17.133 INFO:tasks.workunit.client.0.smithi032.stdout:4/962: mknod d4/db/da/d2d/d27/d53/dea/c155 0 2024-06-03T01:21:17.136 INFO:tasks.workunit.client.0.smithi032.stdout:4/963: link d4/db/da/d37/l45 d4/db/da/d37/dab/dec/l156 0 2024-06-03T01:21:17.136 INFO:tasks.workunit.client.0.smithi032.stdout:4/964: write d4/d5c/f10c [880298,120444] 0 2024-06-03T01:21:17.136 INFO:tasks.workunit.client.0.smithi032.stdout:4/965: chown d4/db/f68 47 1 2024-06-03T01:21:17.137 INFO:tasks.workunit.client.0.smithi032.stdout:4/966: symlink d4/db/da/d5d/d8e/l157 0 2024-06-03T01:21:17.151 INFO:tasks.workunit.client.0.smithi032.stdout:4/967: dwrite d4/db/da/f82 [0,4194304] 0 2024-06-03T01:21:17.154 INFO:tasks.workunit.client.0.smithi032.stdout:4/968: symlink d4/d57/d6c/l158 0 2024-06-03T01:21:17.178 INFO:tasks.workunit.client.0.smithi032.stdout:4/969: dwrite d4/d57/d6c/fcf [0,4194304] 0 2024-06-03T01:21:17.181 INFO:tasks.workunit.client.0.smithi032.stdout:4/970: getdents d4/db/da/d37/dab 0 2024-06-03T01:21:17.183 INFO:tasks.workunit.client.0.smithi032.stdout:4/971: dread d4/db/da/d2d/d27/d8c/db5/fef [0,4194304] 0 2024-06-03T01:21:17.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/972: dwrite d4/db/da/d1d/d6b/f7c [4194304,4194304] 0 2024-06-03T01:21:17.201 INFO:tasks.workunit.client.0.smithi032.stdout:4/973: getdents d4/db/da/d37/d90/da1 0 2024-06-03T01:21:17.204 INFO:tasks.workunit.client.0.smithi032.stdout:4/974: creat d4/db/da/d37/dab/f159 x:0 0 0 2024-06-03T01:21:17.206 INFO:tasks.workunit.client.0.smithi032.stdout:4/975: link d4/l6d d4/d5c/l15a 0 2024-06-03T01:21:17.214 INFO:tasks.workunit.client.0.smithi032.stdout:4/976: dread d4/db/da/d2d/d27/f4f [4194304,4194304] 0 2024-06-03T01:21:17.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/977: dread d4/d57/d6c/f6e [0,4194304] 0 2024-06-03T01:21:17.235 INFO:tasks.workunit.client.0.smithi032.stdout:4/978: dwrite d4/db/da/d2d/d27/f135 [0,4194304] 0 2024-06-03T01:21:17.238 INFO:tasks.workunit.client.0.smithi032.stdout:4/979: creat d4/db/da/d5d/dc5/f15b x:0 0 0 2024-06-03T01:21:17.238 INFO:tasks.workunit.client.0.smithi032.stdout:4/980: truncate d4/db/da/d5d/d8e/f106 326373 0 2024-06-03T01:21:17.240 INFO:tasks.workunit.client.0.smithi032.stdout:4/981: symlink d4/db/da/l15c 0 2024-06-03T01:21:17.240 INFO:tasks.workunit.client.0.smithi032.stdout:4/982: write d4/db/da/d2d/f114 [716863,14521] 0 2024-06-03T01:21:17.244 INFO:tasks.workunit.client.0.smithi032.stdout:4/983: rmdir d4/db/da/d37/d90/da1/d12e 0 2024-06-03T01:21:17.244 INFO:tasks.workunit.client.0.smithi032.stdout:4/984: fsync d4/d57/f9c 0 2024-06-03T01:21:17.265 INFO:tasks.workunit.client.0.smithi032.stdout:4/985: dwrite d4/db/da/d1d/d6b/feb [0,4194304] 0 2024-06-03T01:21:17.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/986: symlink d4/db/da/d2d/d27/l15d 0 2024-06-03T01:21:17.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/987: truncate d4/db/f128 862191 0 2024-06-03T01:21:17.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/988: write d4/db/da/d37/dab/ff5 [156134,37793] 0 2024-06-03T01:21:17.278 INFO:tasks.workunit.client.0.smithi032.stdout:4/989: creat d4/db/da/d37/d90/d13b/d13f/f15e x:0 0 0 2024-06-03T01:21:17.278 INFO:tasks.workunit.client.0.smithi032.stdout:4/990: dread - d4/db/da/d37/dab/dec/d131/f133 zero size 2024-06-03T01:21:17.278 INFO:tasks.workunit.client.0.smithi032.stdout:4/991: chown d4/db/da/d1d/d6b/la7 5450703 1 2024-06-03T01:21:17.279 INFO:tasks.workunit.client.0.smithi032.stdout:4/992: rmdir d4/db/da/d5d/dc5 39 2024-06-03T01:21:17.280 INFO:tasks.workunit.client.0.smithi032.stdout:4/993: write d4/db/da/d1d/d6b/fba [9266290,82860] 0 2024-06-03T01:21:17.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/994: rmdir d4/db/da/d37/d90/da1/d11e 39 2024-06-03T01:21:17.282 INFO:tasks.workunit.client.0.smithi032.stdout:4/995: truncate d4/db/da/f96 642039 0 2024-06-03T01:21:17.285 INFO:tasks.workunit.client.0.smithi032.stdout:4/996: symlink d4/db/da/l15f 0 2024-06-03T01:21:17.285 INFO:tasks.workunit.client.0.smithi032.stdout:4/997: fsync d4/db/da/d2d/d27/f4f 0 2024-06-03T01:21:17.293 INFO:tasks.workunit.client.0.smithi032.stdout:4/998: dread d4/db/f36 [0,4194304] 0 2024-06-03T01:21:17.294 INFO:tasks.workunit.client.0.smithi032.stdout:4/999: mkdir d4/db/da/d1d/d6b/d160 0 2024-06-03T01:21:17.439 INFO:tasks.workunit.client.0.smithi032.stdout:8/918: sync 2024-06-03T01:21:17.455 INFO:tasks.workunit.client.0.smithi032.stdout:8/919: dwrite dc/d25/d28/d58/d8d/d94/ffe [0,4194304] 0 2024-06-03T01:21:17.455 INFO:tasks.workunit.client.0.smithi032.stdout:8/920: read - dc/de/d1e/d43/d9b/dd7/f106 zero size 2024-06-03T01:21:17.463 INFO:tasks.workunit.client.0.smithi032.stdout:8/921: rename dc/d25/d28/d58/d8d/d94 to dc/d25/d28/dea/d11b 0 2024-06-03T01:21:17.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:17 smithi044 ceph-mon[36956]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 126 MiB/s rd, 156 MiB/s wr, 309 op/s 2024-06-03T01:21:17.480 INFO:tasks.workunit.client.0.smithi032.stdout:8/922: dwrite dc/de/d1e/d43/d9b/dfc/fa4 [0,4194304] 0 2024-06-03T01:21:17.481 INFO:tasks.workunit.client.0.smithi032.stdout:8/923: truncate dc/de/d1e/d43/f10a 811628 0 2024-06-03T01:21:17.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/924: link dc/c72 dc/d14/dee/c11c 0 2024-06-03T01:21:17.484 INFO:tasks.workunit.client.0.smithi032.stdout:8/925: creat dc/de/d1e/d43/d9b/dfc/f11d x:0 0 0 2024-06-03T01:21:17.485 INFO:tasks.workunit.client.0.smithi032.stdout:6/822: sync 2024-06-03T01:21:17.485 INFO:tasks.workunit.client.0.smithi032.stdout:1/855: sync 2024-06-03T01:21:17.485 INFO:tasks.workunit.client.0.smithi032.stdout:7/864: sync 2024-06-03T01:21:17.485 INFO:tasks.workunit.client.0.smithi032.stdout:9/828: sync 2024-06-03T01:21:17.485 INFO:tasks.workunit.client.0.smithi032.stdout:9/829: dread - d0/d23/fb5 zero size 2024-06-03T01:21:17.486 INFO:tasks.workunit.client.0.smithi032.stdout:9/830: dread - d0/f9a zero size 2024-06-03T01:21:17.486 INFO:tasks.workunit.client.0.smithi032.stdout:3/758: sync 2024-06-03T01:21:17.486 INFO:tasks.workunit.client.0.smithi032.stdout:8/926: mknod dc/d25/d28/dea/d11b/d59/c11e 0 2024-06-03T01:21:17.487 INFO:tasks.workunit.client.0.smithi032.stdout:6/823: mknod d8/d11/d20/d2b/d39/d19/c108 0 2024-06-03T01:21:17.488 INFO:tasks.workunit.client.0.smithi032.stdout:6/824: read d8/d11/d20/d2b/d39/d19/f40 [4591714,38421] 0 2024-06-03T01:21:17.488 INFO:tasks.workunit.client.0.smithi032.stdout:6/825: creat d8/d4c/daa/f109 x:0 0 0 2024-06-03T01:21:17.490 INFO:tasks.workunit.client.0.smithi032.stdout:7/865: rename dc/d12/d24/c3c to dc/d12/d24/c110 0 2024-06-03T01:21:17.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/759: symlink de/d29/d2e/d31/d23/d53/d69/lf8 0 2024-06-03T01:21:17.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/760: fdatasync de/f10 0 2024-06-03T01:21:17.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/761: write de/d29/d2e/d31/d24/f2f [556989,105089] 0 2024-06-03T01:21:17.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/762: chown de/d29/d2e/d31/d8d/dc3/l83 0 1 2024-06-03T01:21:17.493 INFO:tasks.workunit.client.0.smithi032.stdout:3/763: dread - de/d29/d2e/d31/d3c/d50/ff4 zero size 2024-06-03T01:21:17.493 INFO:tasks.workunit.client.0.smithi032.stdout:6/826: mkdir d8/dd5/da5/d10a 0 2024-06-03T01:21:17.494 INFO:tasks.workunit.client.0.smithi032.stdout:7/866: unlink dc/f30 0 2024-06-03T01:21:17.494 INFO:tasks.workunit.client.0.smithi032.stdout:1/856: read d18/d32/f52 [1287584,46749] 0 2024-06-03T01:21:17.494 INFO:tasks.workunit.client.0.smithi032.stdout:1/857: readlink d18/d27/l79 0 2024-06-03T01:21:17.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/858: truncate d18/d3f/d5f/d6b/d78/f5c 4911982 0 2024-06-03T01:21:17.495 INFO:tasks.workunit.client.0.smithi032.stdout:7/867: symlink dc/d12/d24/dd3/l111 0 2024-06-03T01:21:17.495 INFO:tasks.workunit.client.0.smithi032.stdout:7/868: readlink dc/d12/lcc 0 2024-06-03T01:21:17.495 INFO:tasks.workunit.client.0.smithi032.stdout:7/869: chown dc/d12/d22 99 1 2024-06-03T01:21:17.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/827: symlink d8/d55/l10b 0 2024-06-03T01:21:17.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/859: creat d18/d27/da7/ddb/f106 x:0 0 0 2024-06-03T01:21:17.505 INFO:tasks.workunit.client.0.smithi032.stdout:6/828: dread d8/d4c/f58 [0,4194304] 0 2024-06-03T01:21:17.506 INFO:tasks.workunit.client.0.smithi032.stdout:1/860: dread d18/d3f/d5f/d6b/d78/d3d/d47/fb4 [0,4194304] 0 2024-06-03T01:21:17.507 INFO:tasks.workunit.client.0.smithi032.stdout:3/764: dread de/d29/d2e/d31/d8d/dc3/fbd [0,4194304] 0 2024-06-03T01:21:17.507 INFO:tasks.workunit.client.0.smithi032.stdout:3/765: dread - de/d29/d2e/d31/d3c/d50/d62/fb8 zero size 2024-06-03T01:21:17.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/766: getdents de/d63 0 2024-06-03T01:21:17.515 INFO:tasks.workunit.client.0.smithi032.stdout:3/767: write de/d29/d2e/d31/d23/d53/d69/d8c/fa9 [3912268,19154] 0 2024-06-03T01:21:17.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/768: creat de/d29/d2e/d31/ff9 x:0 0 0 2024-06-03T01:21:17.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/769: fdatasync de/d29/d2e/d31/d3c/d50/d56/f60 0 2024-06-03T01:21:17.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/831: dwrite d0/d23/f85 [0,4194304] 0 2024-06-03T01:21:17.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/770: rename de/d29/d2e/d31/d23/d53/f6c to de/d29/ffa 0 2024-06-03T01:21:17.520 INFO:tasks.workunit.client.0.smithi032.stdout:8/927: dwrite dc/f15 [0,4194304] 0 2024-06-03T01:21:17.520 INFO:tasks.workunit.client.0.smithi032.stdout:8/928: read dc/de/d1e/d43/d9b/dfc/f89 [2752298,87087] 0 2024-06-03T01:21:17.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/870: dwrite dc/d12/d22/d42/d102/f10a [0,4194304] 0 2024-06-03T01:21:17.525 INFO:tasks.workunit.client.0.smithi032.stdout:8/929: truncate dc/d25/d28/d58/f86 2653347 0 2024-06-03T01:21:17.525 INFO:tasks.workunit.client.0.smithi032.stdout:8/930: chown dc/d25/d28/dea/d11b/fe2 31758 1 2024-06-03T01:21:17.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/871: mkdir dc/d12/d24/d33/d112 0 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:7/872: mkdir dc/d12/d6a/d86/dc8/d105/d113 0 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:7/873: dread - dc/d12/d24/d7b/d9d/fa8 zero size 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:7/874: chown dc/d12/d22/d42/d61/ddd/f60 2 1 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:3/771: dread de/d29/d2e/d31/d23/f59 [0,4194304] 0 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:3/772: stat de/f10 0 2024-06-03T01:21:17.532 INFO:tasks.workunit.client.0.smithi032.stdout:8/931: dread dc/d25/d28/d58/dad/fd0 [0,4194304] 0 2024-06-03T01:21:17.533 INFO:tasks.workunit.client.0.smithi032.stdout:3/773: write de/d29/d2e/d31/d8d/dc3/f89 [642382,25860] 0 2024-06-03T01:21:17.533 INFO:tasks.workunit.client.0.smithi032.stdout:7/875: symlink dc/d12/d2a/d2f/d63/de0/l114 0 2024-06-03T01:21:17.537 INFO:tasks.workunit.client.0.smithi032.stdout:3/774: creat de/d29/ffb x:0 0 0 2024-06-03T01:21:17.537 INFO:tasks.workunit.client.0.smithi032.stdout:3/775: chown de/d29/d2e/d31/ff2 63463 1 2024-06-03T01:21:17.540 INFO:tasks.workunit.client.0.smithi032.stdout:9/832: dwrite d0/d23/fce [0,4194304] 0 2024-06-03T01:21:17.546 INFO:tasks.workunit.client.0.smithi032.stdout:1/861: write d18/d27/d2b/f7f [1757718,59330] 0 2024-06-03T01:21:17.547 INFO:tasks.workunit.client.0.smithi032.stdout:9/833: creat d0/d23/f10e x:0 0 0 2024-06-03T01:21:17.548 INFO:tasks.workunit.client.0.smithi032.stdout:9/834: fsync d0/d11/d1d/f1e 0 2024-06-03T01:21:17.548 INFO:tasks.workunit.client.0.smithi032.stdout:9/835: write d0/d11/d1d/f49 [3959650,91953] 0 2024-06-03T01:21:17.549 INFO:tasks.workunit.client.0.smithi032.stdout:1/862: mkdir d18/d27/da7/d107 0 2024-06-03T01:21:17.552 INFO:tasks.workunit.client.0.smithi032.stdout:1/863: mknod d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/c108 0 2024-06-03T01:21:17.558 INFO:tasks.workunit.client.0.smithi032.stdout:1/864: dread d18/d2d/f50 [0,4194304] 0 2024-06-03T01:21:17.558 INFO:tasks.workunit.client.0.smithi032.stdout:1/865: write d18/d3f/d5f/d6b/d78/d3d/d6a/fdc [427061,92411] 0 2024-06-03T01:21:17.560 INFO:tasks.workunit.client.0.smithi032.stdout:9/836: dread d0/d23/d25/f8d [0,4194304] 0 2024-06-03T01:21:17.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/837: mknod d0/da4/dc1/c10f 0 2024-06-03T01:21:17.563 INFO:tasks.workunit.client.0.smithi032.stdout:7/876: dwrite dc/f14 [0,4194304] 0 2024-06-03T01:21:17.563 INFO:tasks.workunit.client.0.smithi032.stdout:8/932: dwrite dc/de/d1e/fa0 [0,4194304] 0 2024-06-03T01:21:17.563 INFO:tasks.workunit.client.0.smithi032.stdout:8/933: readlink dc/d25/d28/dea/d11b/l81 0 2024-06-03T01:21:17.563 INFO:tasks.workunit.client.0.smithi032.stdout:8/934: read - dc/de/d1e/ff3 zero size 2024-06-03T01:21:17.564 INFO:tasks.workunit.client.0.smithi032.stdout:3/776: dwrite de/d29/d2e/d31/d24/fc8 [0,4194304] 0 2024-06-03T01:21:17.565 INFO:tasks.workunit.client.0.smithi032.stdout:7/877: rename dc/d12/d2a/f43 to dc/d12/f115 0 2024-06-03T01:21:17.565 INFO:tasks.workunit.client.0.smithi032.stdout:7/878: read - dc/d12/d22/d42/f10e zero size 2024-06-03T01:21:17.570 INFO:tasks.workunit.client.0.smithi032.stdout:8/935: link dc/d25/d28/dea/d11b/da6/f10c dc/de/d1e/d43/d9b/d76/f11f 0 2024-06-03T01:21:17.570 INFO:tasks.workunit.client.0.smithi032.stdout:8/936: chown dc/d25/d28/l67 23080 1 2024-06-03T01:21:17.576 INFO:tasks.workunit.client.0.smithi032.stdout:3/777: read de/d29/d2e/d31/f21 [2173954,89507] 0 2024-06-03T01:21:17.576 INFO:tasks.workunit.client.0.smithi032.stdout:3/778: chown de/d29/d2e/d31/d23/fed 166 1 2024-06-03T01:21:17.577 INFO:tasks.workunit.client.0.smithi032.stdout:7/879: mknod dc/d12/d24/d7b/d108/c116 0 2024-06-03T01:21:17.577 INFO:tasks.workunit.client.0.smithi032.stdout:7/880: chown dc/d12/d22/d42/d61/ddd/f2c 1 1 2024-06-03T01:21:17.577 INFO:tasks.workunit.client.0.smithi032.stdout:7/881: readlink dc/d12/l64 0 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/779: creat de/d29/d2e/d31/d23/d53/ffc x:0 0 0 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/780: chown de/d29/d2e/d31/c30 547 1 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/781: read - de/d29/d2e/d31/d23/d53/d69/d90/ff3 zero size 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/782: write de/d29/d2e/d31/ff9 [803725,45636] 0 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:3/783: write de/d29/d2e/d31/d8d/f9d [942030,16772] 0 2024-06-03T01:21:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:7/882: creat dc/d12/f117 x:0 0 0 2024-06-03T01:21:17.581 INFO:tasks.workunit.client.0.smithi032.stdout:8/937: dread dc/d25/d28/d3c/f42 [0,4194304] 0 2024-06-03T01:21:17.581 INFO:tasks.workunit.client.0.smithi032.stdout:3/784: write de/d29/d2e/d31/d8d/dc3/f89 [302710,7884] 0 2024-06-03T01:21:17.582 INFO:tasks.workunit.client.0.smithi032.stdout:3/785: write de/d29/d2e/d31/d24/f2f [908746,19943] 0 2024-06-03T01:21:17.582 INFO:tasks.workunit.client.0.smithi032.stdout:8/938: stat dc/de/df0/d119/cd4 0 2024-06-03T01:21:17.583 INFO:tasks.workunit.client.0.smithi032.stdout:8/939: rename dc/de/d1e/c26 to dc/de/d1e/d43/d9b/d76/dae/c120 0 2024-06-03T01:21:17.583 INFO:tasks.workunit.client.0.smithi032.stdout:8/940: truncate dc/d25/d28/d33/fe5 873026 0 2024-06-03T01:21:17.584 INFO:tasks.workunit.client.0.smithi032.stdout:8/941: mknod dc/d25/d28/d32/c121 0 2024-06-03T01:21:17.584 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:17 smithi032 ceph-mon[29360]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 126 MiB/s rd, 156 MiB/s wr, 309 op/s 2024-06-03T01:21:17.585 INFO:tasks.workunit.client.0.smithi032.stdout:7/883: dread dc/d12/d2a/d2f/d63/d69/f5f [0,4194304] 0 2024-06-03T01:21:17.597 INFO:tasks.workunit.client.0.smithi032.stdout:8/942: truncate dc/de/d1e/d43/f10a 248214 0 2024-06-03T01:21:17.598 INFO:tasks.workunit.client.0.smithi032.stdout:7/884: dwrite dc/d12/d22/d42/d61/ddd/f5b [4194304,4194304] 0 2024-06-03T01:21:17.599 INFO:tasks.workunit.client.0.smithi032.stdout:7/885: fdatasync dc/d12/d24/d33/f5c 0 2024-06-03T01:21:17.599 INFO:tasks.workunit.client.0.smithi032.stdout:8/943: rename dc/d25/d28/d58/d8d/l117 to dc/de/d1e/d43/d9b/d76/dae/df2/l122 0 2024-06-03T01:21:17.607 INFO:tasks.workunit.client.0.smithi032.stdout:7/886: dread dc/d12/d22/f85 [0,4194304] 0 2024-06-03T01:21:17.607 INFO:tasks.workunit.client.0.smithi032.stdout:7/887: truncate dc/d12/d22/d42/d61/ddd/fcb 684772 0 2024-06-03T01:21:17.608 INFO:tasks.workunit.client.0.smithi032.stdout:7/888: mknod dc/d12/d24/d33/d10b/c118 0 2024-06-03T01:21:17.622 INFO:tasks.workunit.client.0.smithi032.stdout:8/944: dwrite dc/d25/d28/f103 [0,4194304] 0 2024-06-03T01:21:17.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/945: symlink dc/de/d1e/d43/d9b/dd7/l123 0 2024-06-03T01:21:17.625 INFO:tasks.workunit.client.0.smithi032.stdout:8/946: stat dc/d25/d28/dea/d11b/da6/cd6 0 2024-06-03T01:21:17.631 INFO:tasks.workunit.client.0.smithi032.stdout:7/889: dwrite dc/dbe/fc7 [4194304,4194304] 0 2024-06-03T01:21:17.640 INFO:tasks.workunit.client.0.smithi032.stdout:8/947: dwrite dc/de/df0/d119/f71 [0,4194304] 0 2024-06-03T01:21:17.641 INFO:tasks.workunit.client.0.smithi032.stdout:8/948: rename f2 to dc/d14/f124 0 2024-06-03T01:21:17.643 INFO:tasks.workunit.client.0.smithi032.stdout:8/949: mkdir dc/d14/d31/d3b/d10b/d125 0 2024-06-03T01:21:17.644 INFO:tasks.workunit.client.0.smithi032.stdout:8/950: link f8 dc/d25/d28/d33/f126 0 2024-06-03T01:21:17.645 INFO:tasks.workunit.client.0.smithi032.stdout:7/890: dwrite dc/d12/d6a/d86/d88/d8b/fb5 [0,4194304] 0 2024-06-03T01:21:17.645 INFO:tasks.workunit.client.0.smithi032.stdout:8/951: rmdir dc/de/d1e/d43 39 2024-06-03T01:21:17.646 INFO:tasks.workunit.client.0.smithi032.stdout:7/891: creat dc/d12/d6a/d86/dc8/d105/f119 x:0 0 0 2024-06-03T01:21:17.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/952: unlink dc/de/d1e/l5b 0 2024-06-03T01:21:17.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/953: write dc/d25/d28/d3c/f42 [4186088,32918] 0 2024-06-03T01:21:17.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/954: chown dc/d14/d31/d3b 2074649 1 2024-06-03T01:21:17.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/955: read - dc/d93/ff4 zero size 2024-06-03T01:21:17.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/956: fdatasync dc/d25/d28/dea/d11b/da6/fed 0 2024-06-03T01:21:17.727 INFO:tasks.workunit.client.0.smithi032.stdout:7/892: mkdir dc/d11a 0 2024-06-03T01:21:17.729 INFO:tasks.workunit.client.0.smithi032.stdout:8/957: rename dc/de/d1e/ff3 to dc/d14/dee/f127 0 2024-06-03T01:21:17.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/958: rename dc/de/d1e/d43/d9b/f9c to dc/d14/d31/d3b/f128 0 2024-06-03T01:21:17.739 INFO:tasks.workunit.client.0.smithi032.stdout:7/893: rmdir dc 39 2024-06-03T01:21:17.741 INFO:tasks.workunit.client.0.smithi032.stdout:8/959: creat dc/d14/de3/d11a/f129 x:0 0 0 2024-06-03T01:21:17.749 INFO:tasks.workunit.client.0.smithi032.stdout:8/960: unlink dc/d25/l2f 0 2024-06-03T01:21:17.759 INFO:tasks.workunit.client.0.smithi032.stdout:7/894: dwrite dc/d12/d24/d33/d10b/fe3 [0,4194304] 0 2024-06-03T01:21:17.759 INFO:tasks.workunit.client.0.smithi032.stdout:7/895: stat dc/d12/d22/d42/d61/d82 0 2024-06-03T01:21:17.760 INFO:tasks.workunit.client.0.smithi032.stdout:7/896: getdents dc/d12/d24/dd3 0 2024-06-03T01:21:17.762 INFO:tasks.workunit.client.0.smithi032.stdout:7/897: symlink dc/d12/l11b 0 2024-06-03T01:21:17.763 INFO:tasks.workunit.client.0.smithi032.stdout:7/898: creat dc/d12/d24/d33/daa/f11c x:0 0 0 2024-06-03T01:21:17.763 INFO:tasks.workunit.client.0.smithi032.stdout:7/899: fsync dc/d12/d22/d42/d61/ddd/f60 0 2024-06-03T01:21:17.763 INFO:tasks.workunit.client.0.smithi032.stdout:8/961: dread dc/de/d1e/d43/d9b/d76/fb7 [0,4194304] 0 2024-06-03T01:21:17.778 INFO:tasks.workunit.client.0.smithi032.stdout:7/900: dwrite dc/d12/d2a/d81/ff2 [0,4194304] 0 2024-06-03T01:21:17.780 INFO:tasks.workunit.client.0.smithi032.stdout:7/901: mkdir dc/d12/ddb/d11d 0 2024-06-03T01:21:17.780 INFO:tasks.workunit.client.0.smithi032.stdout:7/902: chown dc/d12/f83 9 1 2024-06-03T01:21:18.177 INFO:tasks.workunit.client.0.smithi032.stdout:6/829: sync 2024-06-03T01:21:18.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/830: dwrite d8/ff [0,4194304] 0 2024-06-03T01:21:18.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/831: chown d8/d11/d20/d3b/fc2 32064 1 2024-06-03T01:21:18.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/832: readlink d8/dd5/da5/ld6 0 2024-06-03T01:21:18.207 INFO:tasks.workunit.client.0.smithi032.stdout:6/833: truncate d8/d11/d20/d2b/d39/f67 2329127 0 2024-06-03T01:21:18.209 INFO:tasks.workunit.client.0.smithi032.stdout:6/834: write d8/d11/d20/d21/d9a/fba [308883,79320] 0 2024-06-03T01:21:18.209 INFO:tasks.workunit.client.0.smithi032.stdout:6/835: fdatasync d8/d11/d20/d3b/fc2 0 2024-06-03T01:21:18.212 INFO:tasks.workunit.client.0.smithi032.stdout:6/836: mkdir d8/d4c/daa/d10c 0 2024-06-03T01:21:18.212 INFO:tasks.workunit.client.0.smithi032.stdout:6/837: write d8/d11/d20/d2b/fb3 [236072,99288] 0 2024-06-03T01:21:18.222 INFO:tasks.workunit.client.0.smithi032.stdout:9/838: sync 2024-06-03T01:21:18.222 INFO:tasks.workunit.client.0.smithi032.stdout:7/903: sync 2024-06-03T01:21:18.222 INFO:tasks.workunit.client.0.smithi032.stdout:1/866: sync 2024-06-03T01:21:18.222 INFO:tasks.workunit.client.0.smithi032.stdout:8/962: sync 2024-06-03T01:21:18.223 INFO:tasks.workunit.client.0.smithi032.stdout:3/786: sync 2024-06-03T01:21:18.223 INFO:tasks.workunit.client.0.smithi032.stdout:3/787: stat de/d29/d2e/d31/d23/d53/d69/fd6 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:3/788: write de/d63/fb2 [1451584,90595] 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:3/789: readlink de/d29/d2e/d31/d3c/l77 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/839: rmdir d0/d11/d1d/d38/d4f/db0/ddb 39 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/840: read d0/d23/d25/dd6/fe2 [934012,66564] 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/841: readlink d0/d11/l8c 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:9/842: write d0/d23/d24/f72 [497236,99424] 0 2024-06-03T01:21:18.224 INFO:tasks.workunit.client.0.smithi032.stdout:7/904: write dc/d12/d24/d7b/d84/fc2 [577371,79515] 0 2024-06-03T01:21:18.225 INFO:tasks.workunit.client.0.smithi032.stdout:7/905: dread - dc/d12/d24/d33/d10b/fae zero size 2024-06-03T01:21:18.225 INFO:tasks.workunit.client.0.smithi032.stdout:3/790: write de/d29/d2e/d31/ff9 [33978,46281] 0 2024-06-03T01:21:18.226 INFO:tasks.workunit.client.0.smithi032.stdout:3/791: truncate de/d29/d2e/d31/d23/d53/d69/d8c/fcf 863690 0 2024-06-03T01:21:18.226 INFO:tasks.workunit.client.0.smithi032.stdout:1/867: rename d18/d32/cab to d18/d2d/d5e/c109 0 2024-06-03T01:21:18.229 INFO:tasks.workunit.client.0.smithi032.stdout:9/843: mkdir d0/d23/d110 0 2024-06-03T01:21:18.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/906: unlink dc/d12/d24/d7b/ca3 0 2024-06-03T01:21:18.232 INFO:tasks.workunit.client.0.smithi032.stdout:3/792: unlink de/d29/d2e/l6f 0 2024-06-03T01:21:18.232 INFO:tasks.workunit.client.0.smithi032.stdout:3/793: chown de/d29/d2e/d31/d23/l2c 1 1 2024-06-03T01:21:18.233 INFO:tasks.workunit.client.0.smithi032.stdout:3/794: write de/d29/d2e/d31/d23/d53/d69/d8c/fcf [310814,22054] 0 2024-06-03T01:21:18.236 INFO:tasks.workunit.client.0.smithi032.stdout:1/868: mkdir d18/d3f/d5f/da6/de1/d10a 0 2024-06-03T01:21:18.236 INFO:tasks.workunit.client.0.smithi032.stdout:6/838: dwrite d8/d11/d20/d31/d3f/ffe [0,4194304] 0 2024-06-03T01:21:18.236 INFO:tasks.workunit.client.0.smithi032.stdout:8/963: symlink dc/d25/d28/d58/d8d/l12a 0 2024-06-03T01:21:18.237 INFO:tasks.workunit.client.0.smithi032.stdout:8/964: write dc/d14/f2d [981730,13063] 0 2024-06-03T01:21:18.241 INFO:tasks.workunit.client.0.smithi032.stdout:9/844: creat d0/d11/f111 x:0 0 0 2024-06-03T01:21:18.243 INFO:tasks.workunit.client.0.smithi032.stdout:8/965: mkdir dc/d25/d28/d58/d8d/d12b 0 2024-06-03T01:21:18.243 INFO:tasks.workunit.client.0.smithi032.stdout:8/966: fsync dc/d14/d31/d3b/f128 0 2024-06-03T01:21:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:9/845: rename d0/d23/d24/f2a to d0/d11/d1d/dd5/f112 0 2024-06-03T01:21:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:9/846: chown d0/d11/d1d/da6/fa2 1339104688 1 2024-06-03T01:21:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:9/847: fsync d0/da4/fbe 0 2024-06-03T01:21:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:9/848: creat d0/da4/dc1/f113 x:0 0 0 2024-06-03T01:21:18.247 INFO:tasks.workunit.client.0.smithi032.stdout:9/849: dread - d0/da4/ffd zero size 2024-06-03T01:21:18.247 INFO:tasks.workunit.client.0.smithi032.stdout:9/850: dread - d0/d11/d1d/f7e zero size 2024-06-03T01:21:18.248 INFO:tasks.workunit.client.0.smithi032.stdout:6/839: symlink d8/d11/db2/l10d 0 2024-06-03T01:21:18.249 INFO:tasks.workunit.client.0.smithi032.stdout:8/967: mkdir dc/de/df0/d12c 0 2024-06-03T01:21:18.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/869: dread d18/d3f/d5f/f84 [0,4194304] 0 2024-06-03T01:21:18.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/870: chown d18/d3f/d5f/d6b/d78/d3d/d6a/fe9 1094515 1 2024-06-03T01:21:18.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/840: link d8/d11/d20/d2b/d39/d19/la7 d8/d4c/daa/d10c/l10e 0 2024-06-03T01:21:18.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/841: creat d8/d11/d20/d2b/d39/f10f x:0 0 0 2024-06-03T01:21:18.254 INFO:tasks.workunit.client.0.smithi032.stdout:7/907: dwrite dc/d12/d22/f47 [0,4194304] 0 2024-06-03T01:21:18.254 INFO:tasks.workunit.client.0.smithi032.stdout:9/851: rename d0/d11/d1d/da6/l50 to d0/d23/d25/de1/df1/l114 0 2024-06-03T01:21:18.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/795: dwrite de/d29/d2e/d31/d23/f6e [4194304,4194304] 0 2024-06-03T01:21:18.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/796: stat de/f10 0 2024-06-03T01:21:18.262 INFO:tasks.workunit.client.0.smithi032.stdout:8/968: write dc/d14/f83 [752464,108094] 0 2024-06-03T01:21:18.280 INFO:tasks.workunit.client.0.smithi032.stdout:3/797: unlink de/d29/d2e/d31/d24/fa5 0 2024-06-03T01:21:18.284 INFO:tasks.workunit.client.0.smithi032.stdout:6/842: link d8/ff d8/d55/f110 0 2024-06-03T01:21:18.284 INFO:tasks.workunit.client.0.smithi032.stdout:6/843: readlink d8/d11/lf1 0 2024-06-03T01:21:18.284 INFO:tasks.workunit.client.0.smithi032.stdout:8/969: creat dc/d14/d31/d3b/d10b/f12d x:0 0 0 2024-06-03T01:21:18.294 INFO:tasks.workunit.client.0.smithi032.stdout:7/908: symlink dc/d12/d22/d42/d61/ddd/l11e 0 2024-06-03T01:21:18.294 INFO:tasks.workunit.client.0.smithi032.stdout:7/909: write dc/d12/d24/d33/d10b/fae [154683,42900] 0 2024-06-03T01:21:18.295 INFO:tasks.workunit.client.0.smithi032.stdout:9/852: getdents d0/d11/d1d/d38/d4f/db0/d41/d9b 0 2024-06-03T01:21:18.297 INFO:tasks.workunit.client.0.smithi032.stdout:8/970: write dc/d25/f35 [320241,11167] 0 2024-06-03T01:21:18.300 INFO:tasks.workunit.client.0.smithi032.stdout:7/910: mknod dc/d12/d22/c11f 0 2024-06-03T01:21:18.302 INFO:tasks.workunit.client.0.smithi032.stdout:9/853: mkdir d0/d11/d1d/d38/d4f/db0/d41/d9b/d115 0 2024-06-03T01:21:18.305 INFO:tasks.workunit.client.0.smithi032.stdout:8/971: mknod dc/de/dfb/c12e 0 2024-06-03T01:21:18.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/911: dread dc/d12/f25 [0,4194304] 0 2024-06-03T01:21:18.305 INFO:tasks.workunit.client.0.smithi032.stdout:7/912: getdents dc/d11a 0 2024-06-03T01:21:18.310 INFO:tasks.workunit.client.0.smithi032.stdout:3/798: dwrite de/d29/d2e/d31/d78/f86 [0,4194304] 0 2024-06-03T01:21:18.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/844: dwrite d8/d11/d20/d2b/d39/d19/f40 [0,4194304] 0 2024-06-03T01:21:18.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/845: write d8/d11/d20/d2b/d39/d19/f46 [6271904,124493] 0 2024-06-03T01:21:18.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/846: write f4 [5156425,26795] 0 2024-06-03T01:21:18.314 INFO:tasks.workunit.client.0.smithi032.stdout:8/972: mknod dc/de/df0/c12f 0 2024-06-03T01:21:18.317 INFO:tasks.workunit.client.0.smithi032.stdout:7/913: symlink dc/d12/d22/d42/d61/l120 0 2024-06-03T01:21:18.317 INFO:tasks.workunit.client.0.smithi032.stdout:7/914: chown dc/d12/d24/d33/daa/dda/f109 2067749 1 2024-06-03T01:21:18.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/854: write d0/d23/d24/f55 [359290,2694] 0 2024-06-03T01:21:18.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/855: write d0/d11/d1d/f47 [3161018,65131] 0 2024-06-03T01:21:18.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/856: dread - d0/da4/fb2 zero size 2024-06-03T01:21:18.322 INFO:tasks.workunit.client.0.smithi032.stdout:3/799: mknod de/d29/d2e/d31/d8d/cfd 0 2024-06-03T01:21:18.330 INFO:tasks.workunit.client.0.smithi032.stdout:9/857: dread d0/d23/d25/f8d [0,4194304] 0 2024-06-03T01:21:18.334 INFO:tasks.workunit.client.0.smithi032.stdout:7/915: creat dc/d12/d2a/d2f/d63/f121 x:0 0 0 2024-06-03T01:21:18.334 INFO:tasks.workunit.client.0.smithi032.stdout:9/858: mkdir d0/d11/d1d/d38/d4f/db0/ddb/d116 0 2024-06-03T01:21:18.334 INFO:tasks.workunit.client.0.smithi032.stdout:9/859: stat d0/d11/f86 0 2024-06-03T01:21:18.334 INFO:tasks.workunit.client.0.smithi032.stdout:9/860: chown d0/d23/d25/f4c 31 1 2024-06-03T01:21:18.335 INFO:tasks.workunit.client.0.smithi032.stdout:7/916: mknod dc/d12/d6a/c122 0 2024-06-03T01:21:18.337 INFO:tasks.workunit.client.0.smithi032.stdout:9/861: symlink d0/d11/d1d/d38/d4f/db0/d41/d9b/l117 0 2024-06-03T01:21:18.337 INFO:tasks.workunit.client.0.smithi032.stdout:9/862: stat d0/fe3 0 2024-06-03T01:21:18.338 INFO:tasks.workunit.client.0.smithi032.stdout:9/863: fsync d0/d11/d1d/da6/f5d 0 2024-06-03T01:21:18.338 INFO:tasks.workunit.client.0.smithi032.stdout:9/864: fsync d0/d23/d25/de1/fe5 0 2024-06-03T01:21:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:3/800: write de/d29/d2e/d31/d23/d53/d69/f8e [3758871,78838] 0 2024-06-03T01:21:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:3/801: write de/d63/f76 [1106129,81103] 0 2024-06-03T01:21:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:8/973: dwrite dc/d14/d31/d3b/f128 [0,4194304] 0 2024-06-03T01:21:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:8/974: read dc/d25/d28/dea/d11b/f6d [3058435,130252] 0 2024-06-03T01:21:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:8/975: stat dc/d25/d28/dea/d11b/d59/l6e 0 2024-06-03T01:21:18.346 INFO:tasks.workunit.client.0.smithi032.stdout:6/847: dwrite d8/d4c/f58 [0,4194304] 0 2024-06-03T01:21:18.354 INFO:tasks.workunit.client.0.smithi032.stdout:7/917: dwrite dc/d12/d6a/fa7 [0,4194304] 0 2024-06-03T01:21:18.354 INFO:tasks.workunit.client.0.smithi032.stdout:3/802: symlink de/d29/d2e/d31/d23/d53/d69/d8c/db4/lfe 0 2024-06-03T01:21:18.359 INFO:tasks.workunit.client.0.smithi032.stdout:6/848: rmdir d8/d11/d20/d31/dfa 0 2024-06-03T01:21:18.361 INFO:tasks.workunit.client.0.smithi032.stdout:3/803: dread de/d29/d2e/d31/d24/f25 [0,4194304] 0 2024-06-03T01:21:18.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/918: mknod dc/d12/d24/dd4/c123 0 2024-06-03T01:21:18.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/919: read dc/d12/d24/d33/d10b/f7d [984052,102195] 0 2024-06-03T01:21:18.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/920: chown dc/d12/d24/d33/d10b/c8d 666099459 1 2024-06-03T01:21:18.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/921: stat dc/d12/d22/d42/d61/ddd/f60 0 2024-06-03T01:21:18.362 INFO:tasks.workunit.client.0.smithi032.stdout:9/865: write d0/f5 [2497476,42469] 0 2024-06-03T01:21:18.363 INFO:tasks.workunit.client.0.smithi032.stdout:3/804: mknod de/d29/d2e/d31/d24/cff 0 2024-06-03T01:21:18.364 INFO:tasks.workunit.client.0.smithi032.stdout:7/922: creat dc/d12/d6a/d86/dc8/d105/d113/f124 x:0 0 0 2024-06-03T01:21:18.369 INFO:tasks.workunit.client.0.smithi032.stdout:7/923: rename dc/d12/d6a/d86/d88/d8b/laf to dc/d12/d6a/d86/dc8/d105/l125 0 2024-06-03T01:21:18.369 INFO:tasks.workunit.client.0.smithi032.stdout:9/866: dread d0/d11/f22 [4194304,4194304] 0 2024-06-03T01:21:18.372 INFO:tasks.workunit.client.0.smithi032.stdout:9/867: truncate d0/d11/d1d/da6/f91 4426658 0 2024-06-03T01:21:18.373 INFO:tasks.workunit.client.0.smithi032.stdout:8/976: dwrite dc/f97 [0,4194304] 0 2024-06-03T01:21:18.373 INFO:tasks.workunit.client.0.smithi032.stdout:8/977: chown dc/d25/f35 1129111235 1 2024-06-03T01:21:18.373 INFO:tasks.workunit.client.0.smithi032.stdout:7/924: rmdir dc/d12/d24/d33/daa 39 2024-06-03T01:21:18.373 INFO:tasks.workunit.client.0.smithi032.stdout:7/925: chown dc/d12/d22 7980829 1 2024-06-03T01:21:18.375 INFO:tasks.workunit.client.0.smithi032.stdout:8/978: mknod dc/d25/d28/d33/c130 0 2024-06-03T01:21:18.375 INFO:tasks.workunit.client.0.smithi032.stdout:8/979: read dc/fd [8162347,37381] 0 2024-06-03T01:21:18.376 INFO:tasks.workunit.client.0.smithi032.stdout:3/805: dread de/d29/d2e/f45 [0,4194304] 0 2024-06-03T01:21:18.376 INFO:tasks.workunit.client.0.smithi032.stdout:7/926: creat dc/f126 x:0 0 0 2024-06-03T01:21:18.377 INFO:tasks.workunit.client.0.smithi032.stdout:6/849: dwrite d8/d11/d20/d2b/fb3 [0,4194304] 0 2024-06-03T01:21:18.377 INFO:tasks.workunit.client.0.smithi032.stdout:6/850: chown d8/d5f 681664938 1 2024-06-03T01:21:18.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/980: getdents dc/de/d1e/d43/d9b/d76/dae/df2 0 2024-06-03T01:21:18.379 INFO:tasks.workunit.client.0.smithi032.stdout:7/927: rmdir dc/d12/d6a/d86/dc8/d105 39 2024-06-03T01:21:18.390 INFO:tasks.workunit.client.0.smithi032.stdout:8/981: write dc/d25/d28/d33/d78/f82 [2950653,54002] 0 2024-06-03T01:21:18.390 INFO:tasks.workunit.client.0.smithi032.stdout:6/851: creat d8/d11/d20/d31/df2/f111 x:0 0 0 2024-06-03T01:21:18.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/982: chown dc/de/c12 43 1 2024-06-03T01:21:18.393 INFO:tasks.workunit.client.0.smithi032.stdout:6/852: rename d8/d11/db2/l10d to d8/d50/dda/l112 0 2024-06-03T01:21:18.395 INFO:tasks.workunit.client.0.smithi032.stdout:7/928: dread f6 [0,4194304] 0 2024-06-03T01:21:18.398 INFO:tasks.workunit.client.0.smithi032.stdout:7/929: symlink dc/d11a/l127 0 2024-06-03T01:21:18.399 INFO:tasks.workunit.client.0.smithi032.stdout:7/930: symlink dc/d12/d2a/d2f/d63/d69/l128 0 2024-06-03T01:21:18.400 INFO:tasks.workunit.client.0.smithi032.stdout:7/931: rmdir dc/d12/d24/d33 39 2024-06-03T01:21:18.400 INFO:tasks.workunit.client.0.smithi032.stdout:7/932: chown dc/d12/d24/dd3 2255 1 2024-06-03T01:21:18.403 INFO:tasks.workunit.client.0.smithi032.stdout:7/933: rename dc/d12/d22/fc0 to dc/d12/d6a/d86/d88/d8b/f129 0 2024-06-03T01:21:18.405 INFO:tasks.workunit.client.0.smithi032.stdout:7/934: creat dc/d12/d24/d33/f12a x:0 0 0 2024-06-03T01:21:18.405 INFO:tasks.workunit.client.0.smithi032.stdout:7/935: fdatasync dc/d12/ddb/ff0 0 2024-06-03T01:21:18.411 INFO:tasks.workunit.client.0.smithi032.stdout:9/868: dwrite d0/d11/d1d/d38/f3e [0,4194304] 0 2024-06-03T01:21:18.411 INFO:tasks.workunit.client.0.smithi032.stdout:9/869: write d0/d11/d1d/d38/d4f/db0/d41/fd8 [856386,51478] 0 2024-06-03T01:21:18.411 INFO:tasks.workunit.client.0.smithi032.stdout:9/870: write d0/d11/d1d/d38/d4f/db0/ddb/fed [1164742,129083] 0 2024-06-03T01:21:18.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/806: dwrite de/d29/f95 [0,4194304] 0 2024-06-03T01:21:18.413 INFO:tasks.workunit.client.0.smithi032.stdout:6/853: dwrite d8/d11/d20/d2b/d39/d19/ff4 [0,4194304] 0 2024-06-03T01:21:18.416 INFO:tasks.workunit.client.0.smithi032.stdout:3/807: symlink de/d63/l100 0 2024-06-03T01:21:18.418 INFO:tasks.workunit.client.0.smithi032.stdout:6/854: creat d8/d11/d20/d21/f113 x:0 0 0 2024-06-03T01:21:18.419 INFO:tasks.workunit.client.0.smithi032.stdout:6/855: chown d8/dd5/cbe 108280187 1 2024-06-03T01:21:18.421 INFO:tasks.workunit.client.0.smithi032.stdout:3/808: getdents de/d29 0 2024-06-03T01:21:18.421 INFO:tasks.workunit.client.0.smithi032.stdout:3/809: truncate de/d29/d2e/d31/d8d/dc3/fa6 4307097 0 2024-06-03T01:21:18.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/856: dwrite d8/d11/d20/d21/d9a/fdc [4194304,4194304] 0 2024-06-03T01:21:18.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/857: stat d8/d11/d20/d21/d9a/fdc 0 2024-06-03T01:21:18.443 INFO:tasks.workunit.client.0.smithi032.stdout:6/858: creat d8/d4c/f114 x:0 0 0 2024-06-03T01:21:18.444 INFO:tasks.workunit.client.0.smithi032.stdout:3/810: dwrite de/d29/d2e/d31/d8d/dc3/fba [0,4194304] 0 2024-06-03T01:21:18.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/859: rmdir d8/d55 39 2024-06-03T01:21:18.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/860: dread - d8/d11/d20/d2b/dc7/ff5 zero size 2024-06-03T01:21:18.451 INFO:tasks.workunit.client.0.smithi032.stdout:9/871: dwrite d0/d11/d1d/dd5/f112 [0,4194304] 0 2024-06-03T01:21:18.454 INFO:tasks.workunit.client.0.smithi032.stdout:9/872: mkdir d0/d23/d25/dd6/d118 0 2024-06-03T01:21:18.454 INFO:tasks.workunit.client.0.smithi032.stdout:9/873: chown d0/ddd 13630410 1 2024-06-03T01:21:18.455 INFO:tasks.workunit.client.0.smithi032.stdout:9/874: mkdir d0/d11/d1d/d38/d119 0 2024-06-03T01:21:18.456 INFO:tasks.workunit.client.0.smithi032.stdout:9/875: mknod d0/d23/d25/dd6/c11a 0 2024-06-03T01:21:18.456 INFO:tasks.workunit.client.0.smithi032.stdout:9/876: write d0/d11/d1d/d38/f97 [1198214,55592] 0 2024-06-03T01:21:18.459 INFO:tasks.workunit.client.0.smithi032.stdout:3/811: dwrite de/d29/d2e/d31/d3c/d50/d56/f67 [4194304,4194304] 0 2024-06-03T01:21:18.464 INFO:tasks.workunit.client.0.smithi032.stdout:9/877: rename d0/d23 to d0/d11/d9c/dd4/d11b 0 2024-06-03T01:21:18.464 INFO:tasks.workunit.client.0.smithi032.stdout:9/878: fsync d0/d11/d1d/d38/d4f/f7a 0 2024-06-03T01:21:18.465 INFO:tasks.workunit.client.0.smithi032.stdout:9/879: truncate d0/f19 6879777 0 2024-06-03T01:21:18.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/880: truncate d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/fdf 538304 0 2024-06-03T01:21:18.469 INFO:tasks.workunit.client.0.smithi032.stdout:9/881: rename d0/d11/d9c/dd4/d11b/d25/l6f to d0/d11/d1d/d38/d4f/db0/d41/l11c 0 2024-06-03T01:21:18.470 INFO:tasks.workunit.client.0.smithi032.stdout:9/882: write d0/d11/d9c/dd4/d11b/d24/f55 [986159,114075] 0 2024-06-03T01:21:18.470 INFO:tasks.workunit.client.0.smithi032.stdout:9/883: symlink d0/d11/d1d/d38/d4f/db0/l11d 0 2024-06-03T01:21:18.471 INFO:tasks.workunit.client.0.smithi032.stdout:9/884: chown d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/de8 105 1 2024-06-03T01:21:18.472 INFO:tasks.workunit.client.0.smithi032.stdout:9/885: creat d0/d11/d9c/dd4/d11b/d25/de1/f11e x:0 0 0 2024-06-03T01:21:18.472 INFO:tasks.workunit.client.0.smithi032.stdout:9/886: write d0/d11/d1d/d38/d4f/fa1 [4785389,91160] 0 2024-06-03T01:21:18.472 INFO:tasks.workunit.client.0.smithi032.stdout:9/887: chown d0/d11/d79/la8 243 1 2024-06-03T01:21:18.476 INFO:tasks.workunit.client.0.smithi032.stdout:3/812: dwrite de/d29/d2e/d31/d8d/f9d [0,4194304] 0 2024-06-03T01:21:18.479 INFO:tasks.workunit.client.0.smithi032.stdout:3/813: creat de/f101 x:0 0 0 2024-06-03T01:21:18.483 INFO:tasks.workunit.client.0.smithi032.stdout:3/814: mkdir de/d29/d2e/d31/d23/d53/dbf/dee/d102 0 2024-06-03T01:21:18.483 INFO:tasks.workunit.client.0.smithi032.stdout:3/815: fsync de/d29/d2e/d31/ff9 0 2024-06-03T01:21:18.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/816: mknod de/d29/d2e/d31/d8d/dc3/c103 0 2024-06-03T01:21:18.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/817: stat de/d29/f70 0 2024-06-03T01:21:18.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/818: readlink de/d29/l87 0 2024-06-03T01:21:18.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/819: dread - de/d29/d2e/d31/d3c/d50/d66/fc1 zero size 2024-06-03T01:21:18.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/820: dread - de/d29/d2e/d31/d8d/dc3/d7e/fb3 zero size 2024-06-03T01:21:18.486 INFO:tasks.workunit.client.0.smithi032.stdout:3/821: symlink de/d29/d2e/d31/d23/d53/dbf/l104 0 2024-06-03T01:21:18.488 INFO:tasks.workunit.client.0.smithi032.stdout:3/822: symlink de/d29/d2e/d31/l105 0 2024-06-03T01:21:18.489 INFO:tasks.workunit.client.0.smithi032.stdout:3/823: symlink de/d29/d2e/d31/l106 0 2024-06-03T01:21:18.490 INFO:tasks.workunit.client.0.smithi032.stdout:9/888: dwrite d0/d11/d1d/da6/f5d [0,4194304] 0 2024-06-03T01:21:18.493 INFO:tasks.workunit.client.0.smithi032.stdout:9/889: rmdir d0/d11/d9c/dd4/d11b/d25/de1 39 2024-06-03T01:21:18.494 INFO:tasks.workunit.client.0.smithi032.stdout:9/890: rename d0/d11/d9c/dd4/d11b/f10e to d0/d11/d1d/d38/d119/f11f 0 2024-06-03T01:21:18.494 INFO:tasks.workunit.client.0.smithi032.stdout:9/891: getdents d0/d11/d1d/d38/d4f/db0/d41/dc6 0 2024-06-03T01:21:18.494 INFO:tasks.workunit.client.0.smithi032.stdout:9/892: dread - d0/d11/d1d/d38/d4f/db0/ddb/fe7 zero size 2024-06-03T01:21:18.496 INFO:tasks.workunit.client.0.smithi032.stdout:9/893: rename d0/d11/d1d/ldc to d0/d11/d1d/d38/d4f/db0/d41/dc6/l120 0 2024-06-03T01:21:18.497 INFO:tasks.workunit.client.0.smithi032.stdout:9/894: symlink d0/d11/d9c/dd4/d11b/d24/l121 0 2024-06-03T01:21:18.497 INFO:tasks.workunit.client.0.smithi032.stdout:9/895: readlink d0/d11/l7d 0 2024-06-03T01:21:18.499 INFO:tasks.workunit.client.0.smithi032.stdout:9/896: mkdir d0/d11/d1d/d38/d122 0 2024-06-03T01:21:18.499 INFO:tasks.workunit.client.0.smithi032.stdout:9/897: write d0/d11/d1d/d38/d4f/db0/f104 [767273,7355] 0 2024-06-03T01:21:18.499 INFO:tasks.workunit.client.0.smithi032.stdout:9/898: getdents d0/d11/d1d/d38/d4f/db0/d41/d9b/d115 0 2024-06-03T01:21:18.499 INFO:tasks.workunit.client.0.smithi032.stdout:9/899: write d0/ddd/f109 [837194,83338] 0 2024-06-03T01:21:18.501 INFO:tasks.workunit.client.0.smithi032.stdout:9/900: truncate d0/d11/d1d/d38/d4f/f7a 306583 0 2024-06-03T01:21:18.502 INFO:tasks.workunit.client.0.smithi032.stdout:9/901: creat d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/dee/f123 x:0 0 0 2024-06-03T01:21:18.507 INFO:tasks.workunit.client.0.smithi032.stdout:3/824: dwrite f9 [0,4194304] 0 2024-06-03T01:21:18.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/825: fdatasync de/d29/f7f 0 2024-06-03T01:21:18.527 INFO:tasks.workunit.client.0.smithi032.stdout:3/826: dwrite de/d29/d2e/d31/f32 [4194304,4194304] 0 2024-06-03T01:21:18.528 INFO:tasks.workunit.client.0.smithi032.stdout:3/827: stat de/d29/d2e/d31/d23/d53/dbf 0 2024-06-03T01:21:18.528 INFO:tasks.workunit.client.0.smithi032.stdout:3/828: write de/d29/f65 [716305,94198] 0 2024-06-03T01:21:18.531 INFO:tasks.workunit.client.0.smithi032.stdout:3/829: rename de/d29/d2e/d31/d78 to de/d29/d2e/d31/d8d/d107 0 2024-06-03T01:21:18.531 INFO:tasks.workunit.client.0.smithi032.stdout:3/830: fdatasync de/d29/d2e/f45 0 2024-06-03T01:21:18.538 INFO:tasks.workunit.client.0.smithi032.stdout:3/831: dread de/d29/f2b [0,4194304] 0 2024-06-03T01:21:18.790 INFO:tasks.workunit.client.0.smithi032.stdout:1/871: sync 2024-06-03T01:21:18.790 INFO:tasks.workunit.client.0.smithi032.stdout:1/872: chown d18/d3f/d5f/d7b/lad 740432718 1 2024-06-03T01:21:18.790 INFO:tasks.workunit.client.0.smithi032.stdout:1/873: creat d18/d27/f10b x:0 0 0 2024-06-03T01:21:18.797 INFO:tasks.workunit.client.0.smithi032.stdout:1/874: dread d18/d32/fdf [0,4194304] 0 2024-06-03T01:21:18.797 INFO:tasks.workunit.client.0.smithi032.stdout:1/875: readlink d18/d27/d2b/l44 0 2024-06-03T01:21:18.797 INFO:tasks.workunit.client.0.smithi032.stdout:1/876: symlink d18/d32/d39/d60/l10c 0 2024-06-03T01:21:18.815 INFO:tasks.workunit.client.0.smithi032.stdout:1/877: dwrite d18/d3f/d5f/d6b/d9d/f8b [0,4194304] 0 2024-06-03T01:21:18.815 INFO:tasks.workunit.client.0.smithi032.stdout:1/878: chown d18/d3f/d5f/d6b/d78/d3d/d6a/cf2 0 1 2024-06-03T01:21:18.815 INFO:tasks.workunit.client.0.smithi032.stdout:1/879: stat d18/c9a 0 2024-06-03T01:21:18.822 INFO:tasks.workunit.client.0.smithi032.stdout:1/880: dread d18/d3f/d5f/d6b/fd5 [0,4194304] 0 2024-06-03T01:21:18.842 INFO:tasks.workunit.client.0.smithi032.stdout:1/881: dwrite d18/d3f/d5f/d6b/d78/f83 [0,4194304] 0 2024-06-03T01:21:18.842 INFO:tasks.workunit.client.0.smithi032.stdout:1/882: chown f17 56 1 2024-06-03T01:21:18.865 INFO:tasks.workunit.client.0.smithi032.stdout:1/883: dwrite d18/d3f/d5f/d6b/d78/d3d/f8c [8388608,4194304] 0 2024-06-03T01:21:18.877 INFO:tasks.workunit.client.0.smithi032.stdout:1/884: dwrite d18/d3f/d5f/d6b/d78/d3d/feb [0,4194304] 0 2024-06-03T01:21:18.879 INFO:tasks.workunit.client.0.smithi032.stdout:1/885: dread d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/fc9 [0,4194304] 0 2024-06-03T01:21:18.882 INFO:tasks.workunit.client.0.smithi032.stdout:1/886: rename d18/d32/c40 to d18/d3f/d5f/d6b/d78/d3d/d6a/c10d 0 2024-06-03T01:21:18.882 INFO:tasks.workunit.client.0.smithi032.stdout:1/887: fsync d18/d3f/d5f/d6b/d78/f9f 0 2024-06-03T01:21:18.882 INFO:tasks.workunit.client.0.smithi032.stdout:1/888: stat d18/d32/d39 0 2024-06-03T01:21:18.887 INFO:tasks.workunit.client.0.smithi032.stdout:1/889: read d18/d32/d39/d60/fd3 [13667,3048] 0 2024-06-03T01:21:18.888 INFO:tasks.workunit.client.0.smithi032.stdout:1/890: write d18/d32/f4b [258654,51517] 0 2024-06-03T01:21:18.889 INFO:tasks.workunit.client.0.smithi032.stdout:1/891: creat d18/d3f/d5f/d6b/d78/de3/f10e x:0 0 0 2024-06-03T01:21:18.890 INFO:tasks.workunit.client.0.smithi032.stdout:1/892: stat d18/d2d/dff 0 2024-06-03T01:21:19.075 INFO:tasks.workunit.client.0.smithi032.stdout:8/983: sync 2024-06-03T01:21:19.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/984: creat dc/d25/f131 x:0 0 0 2024-06-03T01:21:19.077 INFO:tasks.workunit.client.0.smithi032.stdout:8/985: chown dc/d93/ffa 72534 1 2024-06-03T01:21:19.079 INFO:tasks.workunit.client.0.smithi032.stdout:8/986: link dc/de/df0/d119/c53 dc/d25/d28/d33/c132 0 2024-06-03T01:21:19.092 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-03T01:21:19.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-06-03T01:21:19.095 INFO:tasks.workunit.client.0.smithi032.stdout:8/987: dwrite dc/d25/d28/d33/fe6 [0,4194304] 0 2024-06-03T01:21:19.095 INFO:tasks.workunit.client.0.smithi032.stdout:8/988: stat dc/d93/ff4 0 2024-06-03T01:21:19.118 INFO:tasks.workunit.client.0.smithi032.stdout:8/989: dwrite dc/d25/d28/dea/d11b/f6d [0,4194304] 0 2024-06-03T01:21:19.120 INFO:tasks.workunit.client.0.smithi032.stdout:8/990: mknod dc/de/d1e/d43/c133 0 2024-06-03T01:21:19.122 INFO:tasks.workunit.client.0.smithi032.stdout:8/991: creat dc/de/d1e/d43/d9b/d76/dae/f134 x:0 0 0 2024-06-03T01:21:19.124 INFO:tasks.workunit.client.0.smithi032.stdout:8/992: truncate dc/d25/d28/d58/f79 534304 0 2024-06-03T01:21:19.124 INFO:tasks.workunit.client.0.smithi032.stdout:8/993: chown dc/d14/de3/d11a/f129 51560 1 2024-06-03T01:21:19.127 INFO:tasks.workunit.client.0.smithi032.stdout:8/994: unlink dc/c62 0 2024-06-03T01:21:19.127 INFO:tasks.workunit.client.0.smithi032.stdout:8/995: creat dc/d25/d28/d32/f135 x:0 0 0 2024-06-03T01:21:19.129 INFO:tasks.workunit.client.0.smithi032.stdout:8/996: mknod dc/d93/c136 0 2024-06-03T01:21:19.130 INFO:tasks.workunit.client.0.smithi032.stdout:8/997: mkdir dc/de/d1e/d43/d9b/dfc/d8b/d137 0 2024-06-03T01:21:19.132 INFO:tasks.workunit.client.0.smithi032.stdout:8/998: mknod dc/d25/c138 0 2024-06-03T01:21:19.221 INFO:tasks.workunit.client.0.smithi032.stdout:7/936: sync 2024-06-03T01:21:19.222 INFO:tasks.workunit.client.0.smithi032.stdout:7/937: mknod dc/d12/d24/d33/d112/c12b 0 2024-06-03T01:21:19.223 INFO:tasks.workunit.client.0.smithi032.stdout:7/938: truncate dc/d12/f115 4899789 0 2024-06-03T01:21:19.224 INFO:tasks.workunit.client.0.smithi032.stdout:7/939: symlink dc/d12/d24/d7b/d9d/l12c 0 2024-06-03T01:21:19.225 INFO:tasks.workunit.client.0.smithi032.stdout:7/940: mkdir dc/d12/d6a/d86/d88/d8b/d12d 0 2024-06-03T01:21:19.226 INFO:tasks.workunit.client.0.smithi032.stdout:7/941: chown dc/d12/d22/d42/f51 88 1 2024-06-03T01:21:19.228 INFO:tasks.workunit.client.0.smithi032.stdout:7/942: mkdir dc/d12/d22/d42/d61/df1/d12e 0 2024-06-03T01:21:19.228 INFO:tasks.workunit.client.0.smithi032.stdout:7/943: chown dc/d12/d24/d33 15103 1 2024-06-03T01:21:19.245 INFO:tasks.workunit.client.0.smithi032.stdout:7/944: dwrite dc/d12/d6a/d86/d88/d8b/fcf [0,4194304] 0 2024-06-03T01:21:19.249 INFO:tasks.workunit.client.0.smithi032.stdout:7/945: rename dc/d12/d24/d7b/fbc to dc/d12/d22/d42/d61/d82/f12f 0 2024-06-03T01:21:19.256 INFO:tasks.workunit.client.0.smithi032.stdout:7/946: dread dc/d12/d24/d33/f66 [0,4194304] 0 2024-06-03T01:21:19.258 INFO:tasks.workunit.client.0.smithi032.stdout:7/947: creat dc/d12/d22/d42/d61/d82/f130 x:0 0 0 2024-06-03T01:21:19.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/948: write dc/d12/d24/d7b/f20 [1291753,86958] 0 2024-06-03T01:21:19.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/949: write dc/d12/d22/d42/d61/d82/f130 [570455,65417] 0 2024-06-03T01:21:19.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/950: mkdir dc/d12/d22/d42/d102/d131 0 2024-06-03T01:21:19.267 INFO:tasks.workunit.client.0.smithi032.stdout:7/951: rename dc/d12/d6a/c91 to dc/d12/d22/d42/d61/df1/d12e/c132 0 2024-06-03T01:21:19.275 INFO:tasks.workunit.client.0.smithi032.stdout:7/952: write dc/d12/d22/f47 [2609012,130383] 0 2024-06-03T01:21:19.276 INFO:tasks.workunit.client.0.smithi032.stdout:7/953: truncate dc/d12/d22/d42/f50 4431202 0 2024-06-03T01:21:19.277 INFO:tasks.workunit.client.0.smithi032.stdout:7/954: mknod dc/d12/ddb/d11d/c133 0 2024-06-03T01:21:19.277 INFO:tasks.workunit.client.0.smithi032.stdout:7/955: write dc/d12/d24/d33/f92 [309938,110157] 0 2024-06-03T01:21:19.278 INFO:tasks.workunit.client.0.smithi032.stdout:7/956: getdents dc/d12/d24/d7b/d9d 0 2024-06-03T01:21:19.280 INFO:tasks.workunit.client.0.smithi032.stdout:7/957: rename dc/d12/d24/d7b/d9d/l12c to dc/d12/d22/d42/d61/df1/d12e/l134 0 2024-06-03T01:21:19.284 INFO:tasks.workunit.client.0.smithi032.stdout:7/958: read dc/d12/d24/d33/f66 [2135671,92628] 0 2024-06-03T01:21:19.286 INFO:tasks.workunit.client.0.smithi032.stdout:7/959: rename dc/d12/f13 to dc/d12/d22/d42/d61/f135 0 2024-06-03T01:21:19.287 INFO:tasks.workunit.client.0.smithi032.stdout:7/960: fdatasync dc/d12/d24/d33/d10b/fe3 0 2024-06-03T01:21:19.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/961: truncate dc/d12/d22/d42/d61/ddd/f5b 204259 0 2024-06-03T01:21:19.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/962: chown dc/f14 2969139 1 2024-06-03T01:21:19.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/963: chown dc/d12/d6a 152382 1 2024-06-03T01:21:19.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/964: creat dc/d12/d2a/d2f/d63/f136 x:0 0 0 2024-06-03T01:21:19.292 INFO:tasks.workunit.client.0.smithi032.stdout:7/965: mknod dc/d12/d6a/d86/d88/c137 0 2024-06-03T01:21:19.307 INFO:tasks.workunit.client.0.smithi032.stdout:6/861: sync 2024-06-03T01:21:19.307 INFO:tasks.workunit.client.0.smithi032.stdout:9/902: sync 2024-06-03T01:21:19.307 INFO:tasks.workunit.client.0.smithi032.stdout:9/903: fsync d0/d11/d1d/f99 0 2024-06-03T01:21:19.308 INFO:tasks.workunit.client.0.smithi032.stdout:9/904: mknod d0/d11/d1d/d38/d4f/db0/d41/d9b/d115/c124 0 2024-06-03T01:21:19.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/905: write d0/d11/d9c/dd4/d11b/d25/f62 [1435429,315] 0 2024-06-03T01:21:19.324 INFO:tasks.workunit.client.0.smithi032.stdout:6/862: dwrite d8/d4c/daa/fb7 [0,4194304] 0 2024-06-03T01:21:19.327 INFO:tasks.workunit.client.0.smithi032.stdout:6/863: rename d8/d11/d20/d21/ff3 to d8/dd5/da5/f115 0 2024-06-03T01:21:19.328 INFO:tasks.workunit.client.0.smithi032.stdout:6/864: mknod d8/d11/d20/d7f/c116 0 2024-06-03T01:21:19.328 INFO:tasks.workunit.client.0.smithi032.stdout:6/865: write d8/d11/d20/d2b/d39/d19/f40 [5497167,53165] 0 2024-06-03T01:21:19.329 INFO:tasks.workunit.client.0.smithi032.stdout:9/906: dread d0/d11/d9c/dd4/d11b/fce [0,4194304] 0 2024-06-03T01:21:19.391 INFO:tasks.workunit.client.0.smithi032.stdout:1/893: sync 2024-06-03T01:21:19.391 INFO:tasks.workunit.client.0.smithi032.stdout:3/832: sync 2024-06-03T01:21:19.391 INFO:tasks.workunit.client.0.smithi032.stdout:9/907: dread d0/d11/d1d/d38/f97 [0,4194304] 0 2024-06-03T01:21:19.391 INFO:tasks.workunit.client.0.smithi032.stdout:9/908: dread - d0/da4/dc1/f10b zero size 2024-06-03T01:21:19.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/999: sync 2024-06-03T01:21:19.393 INFO:tasks.workunit.client.0.smithi032.stdout:3/833: rename de/d29/d2e/d31/d3c/d50/d62/lb7 to de/d29/d2e/d31/d3c/d50/d56/dc7/l108 0 2024-06-03T01:21:19.395 INFO:tasks.workunit.client.0.smithi032.stdout:9/909: unlink d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/de8/l107 0 2024-06-03T01:21:19.396 INFO:tasks.workunit.client.0.smithi032.stdout:9/910: read d0/d11/d9c/dd4/d11b/d25/f52 [3298235,122740] 0 2024-06-03T01:21:19.400 INFO:tasks.workunit.client.0.smithi032.stdout:3/834: creat de/d29/d2e/d31/d24/daf/f109 x:0 0 0 2024-06-03T01:21:19.402 INFO:tasks.workunit.client.0.smithi032.stdout:9/911: unlink d0/d11/d1d/d38/d4f/db0/cb1 0 2024-06-03T01:21:19.402 INFO:tasks.workunit.client.0.smithi032.stdout:6/866: write d8/d11/db2/fcc [1407091,56240] 0 2024-06-03T01:21:19.402 INFO:tasks.workunit.client.0.smithi032.stdout:6/867: chown d8/d11/d20/d2b/d39/d19/ff4 47 1 2024-06-03T01:21:19.403 INFO:tasks.workunit.client.0.smithi032.stdout:3/835: creat de/d29/d2e/d31/d23/d53/dbf/dee/d102/f10a x:0 0 0 2024-06-03T01:21:19.404 INFO:tasks.workunit.client.0.smithi032.stdout:9/912: write d0/d11/d1d/da6/fd0 [339048,13189] 0 2024-06-03T01:21:19.404 INFO:tasks.workunit.client.0.smithi032.stdout:9/913: fdatasync d0/d11/d1d/d38/d4f/fa1 0 2024-06-03T01:21:19.406 INFO:tasks.workunit.client.0.smithi032.stdout:6/868: rename d8/d11/d20/d21/d9a/cf6 to d8/dd5/da5/d10a/c117 0 2024-06-03T01:21:19.407 INFO:tasks.workunit.client.0.smithi032.stdout:6/869: readlink d8/d11/d20/d3b/l61 0 2024-06-03T01:21:19.410 INFO:tasks.workunit.client.0.smithi032.stdout:9/914: truncate d0/d11/d1d/f99 1867643 0 2024-06-03T01:21:19.410 INFO:tasks.workunit.client.0.smithi032.stdout:9/915: chown d0/d11/d9c/dd4/d11b/d24/c5b 167 1 2024-06-03T01:21:19.411 INFO:tasks.workunit.client.0.smithi032.stdout:3/836: rename de/cc9 to de/d29/c10b 0 2024-06-03T01:21:19.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/837: chown de/d29/d2e/d31/d24/f25 102082316 1 2024-06-03T01:21:19.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/838: rename de/d29/d2e/d31/da1 to de/d29/d2e/d31/da1/d10c 22 2024-06-03T01:21:19.416 INFO:tasks.workunit.client.0.smithi032.stdout:3/839: symlink de/d29/d2e/d31/d23/d53/d79/dab/l10d 0 2024-06-03T01:21:19.422 INFO:tasks.workunit.client.0.smithi032.stdout:9/916: write d0/d11/d1d/da6/f5d [2774784,30449] 0 2024-06-03T01:21:19.424 INFO:tasks.workunit.client.0.smithi032.stdout:9/917: link d0/c65 d0/d11/d9c/dd4/d11b/d25/dd6/c125 0 2024-06-03T01:21:19.425 INFO:tasks.workunit.client.0.smithi032.stdout:6/870: dwrite d8/d11/d20/d2b/f3c [0,4194304] 0 2024-06-03T01:21:19.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/871: chown d8/d11/d20/d7f/c9b 0 1 2024-06-03T01:21:19.426 INFO:tasks.workunit.client.0.smithi032.stdout:9/918: creat d0/d11/d9c/dd4/d11b/d110/f126 x:0 0 0 2024-06-03T01:21:19.427 INFO:tasks.workunit.client.0.smithi032.stdout:9/919: getdents d0/d11/d9c/dd4/d11b 0 2024-06-03T01:21:19.429 INFO:tasks.workunit.client.0.smithi032.stdout:9/920: creat d0/d11/d1d/d38/d4f/db0/ddb/f127 x:0 0 0 2024-06-03T01:21:19.431 INFO:tasks.workunit.client.0.smithi032.stdout:9/921: creat d0/d11/d1d/d38/d4f/db0/d41/d9b/f128 x:0 0 0 2024-06-03T01:21:19.431 INFO:tasks.workunit.client.0.smithi032.stdout:9/922: readlink d0/d11/d1d/l42 0 2024-06-03T01:21:19.431 INFO:tasks.workunit.client.0.smithi032.stdout:9/923: truncate d0/da4/dc1/f10b 760377 0 2024-06-03T01:21:19.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/924: mkdir d0/d11/d9c/d129 0 2024-06-03T01:21:19.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/925: readlink d0/d11/d79/l98 0 2024-06-03T01:21:19.433 INFO:tasks.workunit.client.0.smithi032.stdout:6/872: dread d8/d55/f86 [0,4194304] 0 2024-06-03T01:21:19.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/873: mknod d8/d11/d20/d31/df2/df8/c118 0 2024-06-03T01:21:19.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/874: write d8/d11/d20/d21/d9a/fba [242776,92042] 0 2024-06-03T01:21:19.437 INFO:tasks.workunit.client.0.smithi032.stdout:6/875: mknod d8/d4c/daa/c119 0 2024-06-03T01:21:19.437 INFO:tasks.workunit.client.0.smithi032.stdout:6/876: chown d8/d50/dda/fe1 6870 1 2024-06-03T01:21:19.439 INFO:tasks.workunit.client.0.smithi032.stdout:6/877: link d8/d11/d20/d31/d3f/c87 d8/dd5/c11a 0 2024-06-03T01:21:19.439 INFO:tasks.workunit.client.0.smithi032.stdout:6/878: write f3 [1351665,75082] 0 2024-06-03T01:21:19.440 INFO:tasks.workunit.client.0.smithi032.stdout:9/926: write d0/d11/d1d/d38/f3e [1522889,114805] 0 2024-06-03T01:21:19.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/879: creat d8/d11/d20/d31/df2/df8/f11b x:0 0 0 2024-06-03T01:21:19.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/880: fsync d8/dd5/fef 0 2024-06-03T01:21:19.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/881: write d8/d11/d20/d31/fe6 [1622070,34691] 0 2024-06-03T01:21:19.442 INFO:tasks.workunit.client.0.smithi032.stdout:6/882: truncate d8/d11/d20/d3b/fad 501741 0 2024-06-03T01:21:19.465 INFO:tasks.workunit.client.0.smithi032.stdout:9/927: dwrite d0/d11/d9c/dd4/d11b/d24/f10d [0,4194304] 0 2024-06-03T01:21:19.465 INFO:tasks.workunit.client.0.smithi032.stdout:9/928: chown d0/d11/d9c/dd4/d11b/d25/de1/fe5 747493 1 2024-06-03T01:21:19.466 INFO:tasks.workunit.client.0.smithi032.stdout:9/929: read d0/d11/d9c/dd4/d11b/d25/de1/df1/dea/fc7 [599475,106009] 0 2024-06-03T01:21:19.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/930: creat d0/d11/d1d/d38/d4f/db0/d41/d9b/d115/f12a x:0 0 0 2024-06-03T01:21:19.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/883: dwrite d8/ff [0,4194304] 0 2024-06-03T01:21:19.469 INFO:tasks.workunit.client.0.smithi032.stdout:9/931: rename d0/d11/d9c/dd4/d11b/d25/dd6/cef to d0/d11/d1d/dd5/c12b 0 2024-06-03T01:21:19.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:19 smithi044 ceph-mon[36956]: pgmap v14: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 195 MiB/s rd, 231 MiB/s wr, 451 op/s 2024-06-03T01:21:19.471 INFO:tasks.workunit.client.0.smithi032.stdout:9/932: symlink d0/d11/d1d/d38/d4f/db0/ddb/l12c 0 2024-06-03T01:21:19.474 INFO:tasks.workunit.client.0.smithi032.stdout:9/933: dread d0/d11/d9c/dd4/d11b/d24/f9d [0,4194304] 0 2024-06-03T01:21:19.475 INFO:tasks.workunit.client.0.smithi032.stdout:6/884: dread d8/dd5/fef [0,4194304] 0 2024-06-03T01:21:19.476 INFO:tasks.workunit.client.0.smithi032.stdout:6/885: write f3 [1055415,23455] 0 2024-06-03T01:21:19.478 INFO:tasks.workunit.client.0.smithi032.stdout:6/886: link d8/d4c/daa/f103 d8/d11/d20/d21/d9a/d102/f11c 0 2024-06-03T01:21:19.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/887: mknod d8/dd5/da5/c11d 0 2024-06-03T01:21:19.481 INFO:tasks.workunit.client.0.smithi032.stdout:6/888: dread - d8/d11/d20/d7f/ffc zero size 2024-06-03T01:21:19.490 INFO:tasks.workunit.client.0.smithi032.stdout:9/934: dwrite d0/d11/d1d/d38/f97 [0,4194304] 0 2024-06-03T01:21:19.490 INFO:tasks.workunit.client.0.smithi032.stdout:9/935: write d0/fe3 [1593406,62781] 0 2024-06-03T01:21:19.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/936: dread d0/d11/d9c/dd4/d11b/d25/f8d [0,4194304] 0 2024-06-03T01:21:19.496 INFO:tasks.workunit.client.0.smithi032.stdout:9/937: mknod d0/d11/d9c/dd4/d11b/d24/c12d 0 2024-06-03T01:21:19.497 INFO:tasks.workunit.client.0.smithi032.stdout:6/889: dwrite d8/d50/f105 [0,4194304] 0 2024-06-03T01:21:19.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/938: link d0/da4/ffd d0/d11/d1d/d38/d122/f12e 0 2024-06-03T01:21:19.502 INFO:tasks.workunit.client.0.smithi032.stdout:6/890: symlink d8/dd5/da5/d10a/l11e 0 2024-06-03T01:21:19.503 INFO:tasks.workunit.client.0.smithi032.stdout:9/939: creat d0/d11/d9c/d129/f12f x:0 0 0 2024-06-03T01:21:19.504 INFO:tasks.workunit.client.0.smithi032.stdout:9/940: chown d0/da4/dc1/c10f 502000 1 2024-06-03T01:21:19.504 INFO:tasks.workunit.client.0.smithi032.stdout:9/941: read - d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/fd3 zero size 2024-06-03T01:21:19.504 INFO:tasks.workunit.client.0.smithi032.stdout:9/942: chown d0/d11/d79/l98 1 1 2024-06-03T01:21:19.505 INFO:tasks.workunit.client.0.smithi032.stdout:6/891: dread d8/d11/d20/d2b/d39/fa4 [0,4194304] 0 2024-06-03T01:21:19.518 INFO:tasks.workunit.client.0.smithi032.stdout:6/892: read d8/d4c/fa0 [555454,93796] 0 2024-06-03T01:21:19.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:19 smithi032 ceph-mon[29360]: pgmap v14: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 195 MiB/s rd, 231 MiB/s wr, 451 op/s 2024-06-03T01:21:19.678 INFO:tasks.workunit.client.0.smithi032.stdout:7/966: sync 2024-06-03T01:21:19.679 INFO:tasks.workunit.client.0.smithi032.stdout:7/967: mknod dc/d12/d24/d33/daa/dda/c138 0 2024-06-03T01:21:19.680 INFO:tasks.workunit.client.0.smithi032.stdout:7/968: creat dc/d12/d24/d33/daa/dda/f139 x:0 0 0 2024-06-03T01:21:19.680 INFO:tasks.workunit.client.0.smithi032.stdout:7/969: fdatasync dc/d12/d24/f6c 0 2024-06-03T01:21:19.680 INFO:tasks.workunit.client.0.smithi032.stdout:7/970: chown dc/d12/d78/c7c 16170862 1 2024-06-03T01:21:19.680 INFO:tasks.workunit.client.0.smithi032.stdout:7/971: getdents dc/d12/d6a/d86/dc8/d105 0 2024-06-03T01:21:19.682 INFO:tasks.workunit.client.0.smithi032.stdout:7/972: dread dc/d12/d24/d33/f92 [0,4194304] 0 2024-06-03T01:21:19.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/973: creat dc/d12/d6a/d86/d88/d8b/d12d/f13a x:0 0 0 2024-06-03T01:21:19.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/974: dread - dc/d12/d24/d33/daa/dda/f109 zero size 2024-06-03T01:21:19.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/975: mkdir dc/d12/d24/d7b/d108/d13b 0 2024-06-03T01:21:19.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/976: write dc/d12/d22/f29 [3980512,116107] 0 2024-06-03T01:21:19.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/977: dread - dc/d12/d24/d33/daa/f11c zero size 2024-06-03T01:21:19.685 INFO:tasks.workunit.client.0.smithi032.stdout:7/978: link dc/d12/d2a/d2f/d9a/cb6 dc/d12/d78/c13c 0 2024-06-03T01:21:19.705 INFO:tasks.workunit.client.0.smithi032.stdout:7/979: dwrite dc/d12/d6a/d86/d88/d8b/fb5 [4194304,4194304] 0 2024-06-03T01:21:19.714 INFO:tasks.workunit.client.0.smithi032.stdout:7/980: dread dc/d12/d6a/fa7 [0,4194304] 0 2024-06-03T01:21:19.730 INFO:tasks.workunit.client.0.smithi032.stdout:7/981: dwrite dc/d12/d22/d42/d102/f10a [0,4194304] 0 2024-06-03T01:21:19.730 INFO:tasks.workunit.client.0.smithi032.stdout:7/982: write dc/d12/d24/d7b/d9d/fb7 [571519,97279] 0 2024-06-03T01:21:19.731 INFO:tasks.workunit.client.0.smithi032.stdout:7/983: symlink dc/d12/d24/d33/l13d 0 2024-06-03T01:21:19.732 INFO:tasks.workunit.client.0.smithi032.stdout:7/984: mkdir dc/d12/d24/d33/daa/dda/d13e 0 2024-06-03T01:21:19.844 INFO:tasks.workunit.client.0.smithi032.stdout:1/894: sync 2024-06-03T01:21:19.844 INFO:tasks.workunit.client.0.smithi032.stdout:9/943: sync 2024-06-03T01:21:19.844 INFO:tasks.workunit.client.0.smithi032.stdout:3/840: sync 2024-06-03T01:21:19.844 INFO:tasks.workunit.client.0.smithi032.stdout:3/841: rename de/d29/d2e/d31 to de/d29/d2e/d31/d3c/d50/d56/dc7/d10e 22 2024-06-03T01:21:19.845 INFO:tasks.workunit.client.0.smithi032.stdout:9/944: getdents d0/d11/d9c/dd4/d11b 0 2024-06-03T01:21:19.845 INFO:tasks.workunit.client.0.smithi032.stdout:1/895: creat d18/d27/da7/ddb/f10f x:0 0 0 2024-06-03T01:21:19.847 INFO:tasks.workunit.client.0.smithi032.stdout:3/842: mknod de/d63/c10f 0 2024-06-03T01:21:19.850 INFO:tasks.workunit.client.0.smithi032.stdout:9/945: getdents d0/d11/d1d/d38/d4f/db0/d41 0 2024-06-03T01:21:19.851 INFO:tasks.workunit.client.0.smithi032.stdout:1/896: creat d18/f110 x:0 0 0 2024-06-03T01:21:19.851 INFO:tasks.workunit.client.0.smithi032.stdout:1/897: stat d18/d3f/d5f/d7b 0 2024-06-03T01:21:19.853 INFO:tasks.workunit.client.0.smithi032.stdout:1/898: rename d18/d2d/d5e/f64 to d18/d32/d39/d60/f111 0 2024-06-03T01:21:19.854 INFO:tasks.workunit.client.0.smithi032.stdout:1/899: chown d18/d32/d39/d60/l10c 1712416 1 2024-06-03T01:21:19.855 INFO:tasks.workunit.client.0.smithi032.stdout:1/900: symlink d18/d3f/d5f/d6b/d78/d3d/df3/dfc/l112 0 2024-06-03T01:21:19.859 INFO:tasks.workunit.client.0.smithi032.stdout:9/946: read d0/f1 [8362094,94583] 0 2024-06-03T01:21:19.866 INFO:tasks.workunit.client.0.smithi032.stdout:1/901: read d18/d32/d39/fc5 [2545880,55335] 0 2024-06-03T01:21:19.866 INFO:tasks.workunit.client.0.smithi032.stdout:3/843: dwrite de/d29/d2e/f37 [0,4194304] 0 2024-06-03T01:21:19.867 INFO:tasks.workunit.client.0.smithi032.stdout:9/947: dread d0/d11/d9c/fa3 [0,4194304] 0 2024-06-03T01:21:19.868 INFO:tasks.workunit.client.0.smithi032.stdout:1/902: rename d18/d3f/d5f/d6b/d78/f5c to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/f113 0 2024-06-03T01:21:19.869 INFO:tasks.workunit.client.0.smithi032.stdout:9/948: getdents d0/d11/d9c 0 2024-06-03T01:21:19.870 INFO:tasks.workunit.client.0.smithi032.stdout:1/903: creat d18/d3f/f114 x:0 0 0 2024-06-03T01:21:19.871 INFO:tasks.workunit.client.0.smithi032.stdout:9/949: mknod d0/d11/d9c/dd4/d11b/d110/c130 0 2024-06-03T01:21:19.872 INFO:tasks.workunit.client.0.smithi032.stdout:1/904: creat d18/d27/da7/d107/f115 x:0 0 0 2024-06-03T01:21:19.872 INFO:tasks.workunit.client.0.smithi032.stdout:1/905: stat d18/d3f/d5f/d6b/d78/d3d/c7d 0 2024-06-03T01:21:19.872 INFO:tasks.workunit.client.0.smithi032.stdout:1/906: dread - d18/d32/d3c/fb0 zero size 2024-06-03T01:21:19.873 INFO:tasks.workunit.client.0.smithi032.stdout:1/907: fdatasync d18/d32/d3c/fd2 0 2024-06-03T01:21:19.874 INFO:tasks.workunit.client.0.smithi032.stdout:9/950: unlink d0/d11/d1d/d38/d4f/db0/l11d 0 2024-06-03T01:21:19.875 INFO:tasks.workunit.client.0.smithi032.stdout:1/908: truncate d18/d3f/f55 3396778 0 2024-06-03T01:21:19.875 INFO:tasks.workunit.client.0.smithi032.stdout:1/909: chown d18/d3f/d5f/d7b 25847 1 2024-06-03T01:21:19.876 INFO:tasks.workunit.client.0.smithi032.stdout:1/910: dread d18/f1a [0,4194304] 0 2024-06-03T01:21:19.878 INFO:tasks.workunit.client.0.smithi032.stdout:9/951: write d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/fdf [122346,77278] 0 2024-06-03T01:21:19.883 INFO:tasks.workunit.client.0.smithi032.stdout:3/844: dwrite de/d29/d2e/d31/da1/ff0 [0,4194304] 0 2024-06-03T01:21:19.886 INFO:tasks.workunit.client.0.smithi032.stdout:1/911: dread d18/d3f/fcc [0,4194304] 0 2024-06-03T01:21:19.891 INFO:tasks.workunit.client.0.smithi032.stdout:9/952: read d0/d11/d9c/dd4/d11b/d24/f10d [3444371,127946] 0 2024-06-03T01:21:19.891 INFO:tasks.workunit.client.0.smithi032.stdout:9/953: readlink d0/d11/d1d/d38/d4f/db0/d41/dc6/l120 0 2024-06-03T01:21:19.891 INFO:tasks.workunit.client.0.smithi032.stdout:3/845: mkdir de/d29/d2e/d31/d3c/d50/d66/d110 0 2024-06-03T01:21:19.893 INFO:tasks.workunit.client.0.smithi032.stdout:1/912: truncate d18/d32/d39/d60/f111 3149449 0 2024-06-03T01:21:19.893 INFO:tasks.workunit.client.0.smithi032.stdout:1/913: fsync d18/d3f/d5f/d6b/d78/d3d/d47/f77 0 2024-06-03T01:21:19.899 INFO:tasks.workunit.client.0.smithi032.stdout:6/893: sync 2024-06-03T01:21:19.900 INFO:tasks.workunit.client.0.smithi032.stdout:7/985: sync 2024-06-03T01:21:19.900 INFO:tasks.workunit.client.0.smithi032.stdout:9/954: read d0/d11/d9c/dd4/d11b/d25/de1/df1/dea/f96 [3845750,33759] 0 2024-06-03T01:21:19.903 INFO:tasks.workunit.client.0.smithi032.stdout:6/894: truncate d8/d5f/fd8 721682 0 2024-06-03T01:21:19.903 INFO:tasks.workunit.client.0.smithi032.stdout:6/895: fdatasync d8/f6f 0 2024-06-03T01:21:19.904 INFO:tasks.workunit.client.0.smithi032.stdout:7/986: truncate dc/d12/d22/d42/d61/ddd/f36 990861 0 2024-06-03T01:21:19.905 INFO:tasks.workunit.client.0.smithi032.stdout:3/846: rename de/d29/d2e/d31/d3c/d50/d56/dc7 to de/d29/d2e/d31/d3c/d111 0 2024-06-03T01:21:19.907 INFO:tasks.workunit.client.0.smithi032.stdout:6/896: mkdir d8/d11/d20/d31/df2/df8/d11f 0 2024-06-03T01:21:19.909 INFO:tasks.workunit.client.0.smithi032.stdout:6/897: unlink d8/d11/d20/d2b/d39/c2d 0 2024-06-03T01:21:19.909 INFO:tasks.workunit.client.0.smithi032.stdout:6/898: stat d8/d11/d20/d21/d9a/d102/f11c 0 2024-06-03T01:21:19.911 INFO:tasks.workunit.client.0.smithi032.stdout:6/899: symlink d8/dd5/da5/d10a/l120 0 2024-06-03T01:21:19.915 INFO:tasks.workunit.client.0.smithi032.stdout:1/914: dwrite d18/d32/d39/d60/fd3 [0,4194304] 0 2024-06-03T01:21:19.916 INFO:tasks.workunit.client.0.smithi032.stdout:6/900: rename d8/d4c/daa/f109 to d8/d11/d20/f121 0 2024-06-03T01:21:19.927 INFO:tasks.workunit.client.0.smithi032.stdout:9/955: dwrite d0/d11/f86 [0,4194304] 0 2024-06-03T01:21:19.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/956: readlink d0/d11/d9c/dd4/d11b/lbf 0 2024-06-03T01:21:19.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/957: chown d0/d11/d1d 1 1 2024-06-03T01:21:19.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/958: truncate d0/d11/fa5 23428 0 2024-06-03T01:21:19.929 INFO:tasks.workunit.client.0.smithi032.stdout:9/959: mknod d0/da4/c131 0 2024-06-03T01:21:19.941 INFO:tasks.workunit.client.0.smithi032.stdout:1/915: dwrite d18/d32/d39/d60/f72 [0,4194304] 0 2024-06-03T01:21:19.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/960: dread d0/d11/d1d/d38/f97 [0,4194304] 0 2024-06-03T01:21:19.941 INFO:tasks.workunit.client.0.smithi032.stdout:1/916: creat d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/f116 x:0 0 0 2024-06-03T01:21:19.944 INFO:tasks.workunit.client.0.smithi032.stdout:9/961: creat d0/d11/d1d/d38/f132 x:0 0 0 2024-06-03T01:21:19.945 INFO:tasks.workunit.client.0.smithi032.stdout:7/987: dwrite dc/d12/d22/fad [0,4194304] 0 2024-06-03T01:21:19.947 INFO:tasks.workunit.client.0.smithi032.stdout:3/847: dwrite de/d29/d2e/d31/f1c [0,4194304] 0 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:1/917: unlink d18/d27/d89/c9e 0 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:1/918: chown d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b 1002 1 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/901: dwrite d8/d55/f86 [0,4194304] 0 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:6/902: write f3 [1535389,45508] 0 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:1/919: write d18/d27/da7/d107/f115 [331499,112906] 0 2024-06-03T01:21:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:1/920: dread - d18/d32/d3c/fb0 zero size 2024-06-03T01:21:19.951 INFO:tasks.workunit.client.0.smithi032.stdout:1/921: fsync d18/d3f/d5f/f84 0 2024-06-03T01:21:19.951 INFO:tasks.workunit.client.0.smithi032.stdout:1/922: chown d18/d3f/d5f/d6b/d78/d3d/d6a/l100 1 1 2024-06-03T01:21:19.951 INFO:tasks.workunit.client.0.smithi032.stdout:9/962: symlink d0/l133 0 2024-06-03T01:21:19.952 INFO:tasks.workunit.client.0.smithi032.stdout:3/848: creat de/d29/d2e/d31/d3c/d50/d56/de7/f112 x:0 0 0 2024-06-03T01:21:19.954 INFO:tasks.workunit.client.0.smithi032.stdout:6/903: rmdir d8/d11/d20/d21/d9a 39 2024-06-03T01:21:19.955 INFO:tasks.workunit.client.0.smithi032.stdout:1/923: mkdir d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d117 0 2024-06-03T01:21:19.955 INFO:tasks.workunit.client.0.smithi032.stdout:1/924: creat d18/d3f/d5f/d6b/d78/de3/f118 x:0 0 0 2024-06-03T01:21:19.956 INFO:tasks.workunit.client.0.smithi032.stdout:9/963: mknod d0/d11/d9c/dd4/d11b/c134 0 2024-06-03T01:21:19.958 INFO:tasks.workunit.client.0.smithi032.stdout:3/849: mkdir de/d29/d113 0 2024-06-03T01:21:19.958 INFO:tasks.workunit.client.0.smithi032.stdout:3/850: chown de/d29/c34 530111312 1 2024-06-03T01:21:19.958 INFO:tasks.workunit.client.0.smithi032.stdout:6/904: symlink d8/d50/l122 0 2024-06-03T01:21:19.961 INFO:tasks.workunit.client.0.smithi032.stdout:1/925: link d18/d27/d89/cca d18/d32/d39/c119 0 2024-06-03T01:21:19.962 INFO:tasks.workunit.client.0.smithi032.stdout:9/964: link d0/d11/d1d/d38/d4f/db0/fa0 d0/d11/d1d/f135 0 2024-06-03T01:21:19.963 INFO:tasks.workunit.client.0.smithi032.stdout:9/965: write d0/fe3 [2469194,100448] 0 2024-06-03T01:21:19.963 INFO:tasks.workunit.client.0.smithi032.stdout:9/966: write d0/d11/d1d/d38/d4f/db0/ddb/f127 [146892,64566] 0 2024-06-03T01:21:19.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/988: dwrite dc/d12/d6a/d86/d88/d8b/f129 [0,4194304] 0 2024-06-03T01:21:19.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/989: stat dc/d12/d2a/d2f/d63/de0/l114 0 2024-06-03T01:21:19.964 INFO:tasks.workunit.client.0.smithi032.stdout:6/905: creat d8/d11/d20/d31/df2/df8/d107/f123 x:0 0 0 2024-06-03T01:21:19.964 INFO:tasks.workunit.client.0.smithi032.stdout:6/906: write d8/d4c/f114 [1027615,90113] 0 2024-06-03T01:21:19.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/926: rmdir d18/d3f/d5f/d6b/d78/d3d/df3/dfc 39 2024-06-03T01:21:19.968 INFO:tasks.workunit.client.0.smithi032.stdout:3/851: mkdir de/d29/d2e/d31/d23/d53/d114 0 2024-06-03T01:21:19.970 INFO:tasks.workunit.client.0.smithi032.stdout:7/990: mkdir dc/d12/d2a/d2f/d9a/d13f 0 2024-06-03T01:21:19.971 INFO:tasks.workunit.client.0.smithi032.stdout:6/907: read - d8/d11/fa8 zero size 2024-06-03T01:21:19.971 INFO:tasks.workunit.client.0.smithi032.stdout:1/927: symlink d18/d32/d3c/l11a 0 2024-06-03T01:21:19.973 INFO:tasks.workunit.client.0.smithi032.stdout:7/991: creat dc/d12/f140 x:0 0 0 2024-06-03T01:21:19.975 INFO:tasks.workunit.client.0.smithi032.stdout:6/908: mkdir d8/d4c/daa/d10c/d124 0 2024-06-03T01:21:19.978 INFO:tasks.workunit.client.0.smithi032.stdout:3/852: read de/d29/d2e/d31/d3c/f44 [2581689,64907] 0 2024-06-03T01:21:19.981 INFO:tasks.workunit.client.0.smithi032.stdout:6/909: rename d8/d11/d20/c57 to d8/d11/d20/d21/d9a/c125 0 2024-06-03T01:21:19.983 INFO:tasks.workunit.client.0.smithi032.stdout:3/853: dread de/d29/f3b [0,4194304] 0 2024-06-03T01:21:19.984 INFO:tasks.workunit.client.0.smithi032.stdout:6/910: symlink d8/d11/db2/l126 0 2024-06-03T01:21:19.987 INFO:tasks.workunit.client.0.smithi032.stdout:3/854: rename de/d63/lb9 to de/d29/l115 0 2024-06-03T01:21:19.988 INFO:tasks.workunit.client.0.smithi032.stdout:3/855: chown de/d29/d2e/d31/f1c 29302 1 2024-06-03T01:21:19.991 INFO:tasks.workunit.client.0.smithi032.stdout:6/911: unlink d8/d11/d20/c68 0 2024-06-03T01:21:19.996 INFO:tasks.workunit.client.0.smithi032.stdout:1/928: dwrite d18/d3f/d5f/d7b/da8/fc2 [4194304,4194304] 0 2024-06-03T01:21:19.997 INFO:tasks.workunit.client.0.smithi032.stdout:1/929: fsync d18/d3f/d5f/d98/fa2 0 2024-06-03T01:21:19.997 INFO:tasks.workunit.client.0.smithi032.stdout:1/930: creat d18/d32/d3c/f11b x:0 0 0 2024-06-03T01:21:19.997 INFO:tasks.workunit.client.0.smithi032.stdout:1/931: chown d18/d27/da7/ddb 81944437 1 2024-06-03T01:21:19.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/967: write d0/f15 [2956699,11286] 0 2024-06-03T01:21:19.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/968: stat d0/d11/d79/la8 0 2024-06-03T01:21:19.998 INFO:tasks.workunit.client.0.smithi032.stdout:7/992: write - open dc/d12/d22/d42/f51 failed 14 2024-06-03T01:21:19.998 INFO:tasks.workunit.client.0.smithi032.stdout:7/993: write dc/d12/d24/d33/de5/fd9 [655892,67209] 0 2024-06-03T01:21:20.001 INFO:tasks.workunit.client.0.smithi032.stdout:7/994: read dc/d12/d22/f52 [832933,82569] 0 2024-06-03T01:21:20.001 INFO:tasks.workunit.client.0.smithi032.stdout:7/995: chown dc/d12/d24/d33/de5/fce 277 1 2024-06-03T01:21:20.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/932: creat d18/d32/d39/d60/f11c x:0 0 0 2024-06-03T01:21:20.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/933: creat d18/d3f/f11d x:0 0 0 2024-06-03T01:21:20.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/934: fsync d18/d3f/d5f/d7b/da8/fc2 0 2024-06-03T01:21:20.009 INFO:tasks.workunit.client.0.smithi032.stdout:3/856: dwrite de/d29/f65 [0,4194304] 0 2024-06-03T01:21:20.009 INFO:tasks.workunit.client.0.smithi032.stdout:3/857: fsync de/d63/f7a 0 2024-06-03T01:21:20.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/969: mknod d0/d11/d76/c136 0 2024-06-03T01:21:20.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/970: write d0/d11/d1d/d38/d4f/db0/d41/faa [162187,100891] 0 2024-06-03T01:21:20.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/971: chown d0/d11/f63 1070655253 1 2024-06-03T01:21:20.016 INFO:tasks.workunit.client.0.smithi032.stdout:1/935: link d18/d3f/cb5 d18/d27/da7/d107/c11e 0 2024-06-03T01:21:20.019 INFO:tasks.workunit.client.0.smithi032.stdout:3/858: dread de/d29/d2e/d31/f21 [0,4194304] 0 2024-06-03T01:21:20.021 INFO:tasks.workunit.client.0.smithi032.stdout:3/859: mknod de/d29/d2e/d31/d23/d53/d69/dca/c116 0 2024-06-03T01:21:20.021 INFO:tasks.workunit.client.0.smithi032.stdout:3/860: creat de/d29/d2e/d31/f117 x:0 0 0 2024-06-03T01:21:20.022 INFO:tasks.workunit.client.0.smithi032.stdout:7/996: dwrite dc/d12/d24/d33/d10b/ffd [0,4194304] 0 2024-06-03T01:21:20.022 INFO:tasks.workunit.client.0.smithi032.stdout:6/912: dwrite d8/fb [0,4194304] 0 2024-06-03T01:21:20.028 INFO:tasks.workunit.client.0.smithi032.stdout:9/972: dread d0/fe [0,4194304] 0 2024-06-03T01:21:20.028 INFO:tasks.workunit.client.0.smithi032.stdout:7/997: creat dc/d12/d2a/f141 x:0 0 0 2024-06-03T01:21:20.032 INFO:tasks.workunit.client.0.smithi032.stdout:7/998: symlink dc/d12/d22/d42/l142 0 2024-06-03T01:21:20.037 INFO:tasks.workunit.client.0.smithi032.stdout:7/999: read f6 [207267,12547] 0 2024-06-03T01:21:20.043 INFO:tasks.workunit.client.0.smithi032.stdout:1/936: dwrite d18/d3f/f53 [0,4194304] 0 2024-06-03T01:21:20.043 INFO:tasks.workunit.client.0.smithi032.stdout:1/937: creat d18/d27/f11f x:0 0 0 2024-06-03T01:21:20.044 INFO:tasks.workunit.client.0.smithi032.stdout:3/861: dwrite de/d29/f65 [0,4194304] 0 2024-06-03T01:21:20.044 INFO:tasks.workunit.client.0.smithi032.stdout:3/862: fsync de/d29/d2e/d31/d3c/f44 0 2024-06-03T01:21:20.053 INFO:tasks.workunit.client.0.smithi032.stdout:1/938: unlink d18/l21 0 2024-06-03T01:21:20.056 INFO:tasks.workunit.client.0.smithi032.stdout:3/863: mknod de/d63/c118 0 2024-06-03T01:21:20.059 INFO:tasks.workunit.client.0.smithi032.stdout:1/939: creat d18/d3f/d5f/d6b/d78/d3d/df3/dfc/f120 x:0 0 0 2024-06-03T01:21:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:9/973: write d0/d11/d1d/da6/fba [1004392,67282] 0 2024-06-03T01:21:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:1/940: link d18/d2d/d5e/l76 d18/d3f/d5f/d6b/d78/d3d/df3/dfc/l121 0 2024-06-03T01:21:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:1/941: readlink d18/d32/daa/lc8 0 2024-06-03T01:21:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:3/864: read de/f1a [3343980,81569] 0 2024-06-03T01:21:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:3/865: dread - de/d29/d2e/d31/d8d/d107/fbe zero size 2024-06-03T01:21:20.065 INFO:tasks.workunit.client.0.smithi032.stdout:9/974: creat d0/d11/d9c/dd4/d11b/f137 x:0 0 0 2024-06-03T01:21:20.066 INFO:tasks.workunit.client.0.smithi032.stdout:1/942: getdents d18/d2d/d5e 0 2024-06-03T01:21:20.066 INFO:tasks.workunit.client.0.smithi032.stdout:3/866: getdents de/d29/d2e/d31/d23/d53 0 2024-06-03T01:21:20.068 INFO:tasks.workunit.client.0.smithi032.stdout:9/975: creat d0/d11/d9c/dd4/d11b/d24/f138 x:0 0 0 2024-06-03T01:21:20.068 INFO:tasks.workunit.client.0.smithi032.stdout:9/976: write d0/da4/fb2 [318651,128433] 0 2024-06-03T01:21:20.068 INFO:tasks.workunit.client.0.smithi032.stdout:9/977: chown d0/cb 314610 1 2024-06-03T01:21:20.069 INFO:tasks.workunit.client.0.smithi032.stdout:1/943: creat d18/d2d/f122 x:0 0 0 2024-06-03T01:21:20.072 INFO:tasks.workunit.client.0.smithi032.stdout:3/867: dread de/d29/d2e/d31/d24/f39 [0,4194304] 0 2024-06-03T01:21:20.075 INFO:tasks.workunit.client.0.smithi032.stdout:3/868: getdents de/d29/d2e/d31/d23/d53/dbf 0 2024-06-03T01:21:20.079 INFO:tasks.workunit.client.0.smithi032.stdout:9/978: dread d0/d11/d1d/dd5/f112 [0,4194304] 0 2024-06-03T01:21:20.079 INFO:tasks.workunit.client.0.smithi032.stdout:9/979: readlink d0/d11/d1d/d38/d4f/db0/d41/l11c 0 2024-06-03T01:21:20.079 INFO:tasks.workunit.client.0.smithi032.stdout:9/980: truncate d0/d11/d1d/da6/fd0 473308 0 2024-06-03T01:21:20.093 INFO:tasks.workunit.client.0.smithi032.stdout:3/869: dwrite de/d29/d2e/d31/f21 [4194304,4194304] 0 2024-06-03T01:21:20.101 INFO:tasks.workunit.client.0.smithi032.stdout:3/870: dread de/d29/f65 [0,4194304] 0 2024-06-03T01:21:20.101 INFO:tasks.workunit.client.0.smithi032.stdout:3/871: chown de/d29/d2e/d31/d3c/d50/d62/fb8 57843 1 2024-06-03T01:21:20.101 INFO:tasks.workunit.client.0.smithi032.stdout:3/872: chown de/d29/d2e/d31/d3c/d111/ff1 92250 1 2024-06-03T01:21:20.102 INFO:tasks.workunit.client.0.smithi032.stdout:9/981: dwrite d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1/dee/f123 [0,4194304] 0 2024-06-03T01:21:20.104 INFO:tasks.workunit.client.0.smithi032.stdout:3/873: mkdir de/d29/d2e/d31/d23/d53/dbf/d119 0 2024-06-03T01:21:20.104 INFO:tasks.workunit.client.0.smithi032.stdout:3/874: truncate de/f101 351896 0 2024-06-03T01:21:20.104 INFO:tasks.workunit.client.0.smithi032.stdout:3/875: fdatasync de/d29/d2e/d31/d8d/d107/ff6 0 2024-06-03T01:21:20.111 INFO:tasks.workunit.client.0.smithi032.stdout:9/982: rmdir d0/d11/d1d/d38/d4f/db0/d41/dc6/dd1 39 2024-06-03T01:21:20.113 INFO:tasks.workunit.client.0.smithi032.stdout:3/876: write de/d29/d2e/f37 [2267760,3646] 0 2024-06-03T01:21:20.113 INFO:tasks.workunit.client.0.smithi032.stdout:3/877: readlink de/d29/d2e/d31/d24/l91 0 2024-06-03T01:21:20.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/983: mknod d0/d11/d1d/da6/c139 0 2024-06-03T01:21:20.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/984: stat d0/d11/d1d/d38/f132 0 2024-06-03T01:21:20.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/985: fdatasync d0/d11/d1d/d38/d4f/db0/d41/d9b/d115/f12a 0 2024-06-03T01:21:20.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/986: chown d0/d11/d1d/da6/fa2 905318 1 2024-06-03T01:21:20.132 INFO:tasks.workunit.client.0.smithi032.stdout:3/878: dwrite de/d29/d2e/d31/d8d/de1/fec [0,4194304] 0 2024-06-03T01:21:20.132 INFO:tasks.workunit.client.0.smithi032.stdout:3/879: readlink de/d29/d2e/d31/l105 0 2024-06-03T01:21:20.132 INFO:tasks.workunit.client.0.smithi032.stdout:3/880: fsync de/d29/d2e/f37 0 2024-06-03T01:21:20.135 INFO:tasks.workunit.client.0.smithi032.stdout:3/881: stat de/l26 0 2024-06-03T01:21:20.135 INFO:tasks.workunit.client.0.smithi032.stdout:3/882: dread - de/d29/d2e/d31/d3c/d50/d56/fae zero size 2024-06-03T01:21:20.135 INFO:tasks.workunit.client.0.smithi032.stdout:3/883: chown de/d29/d2e/d31/d8d/dc3 29371 1 2024-06-03T01:21:20.136 INFO:tasks.workunit.client.0.smithi032.stdout:3/884: truncate de/d29/f3f 448778 0 2024-06-03T01:21:20.142 INFO:tasks.workunit.client.0.smithi032.stdout:3/885: dread de/d29/d2e/d31/d3c/f44 [0,4194304] 0 2024-06-03T01:21:20.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/886: dwrite de/f8b [0,4194304] 0 2024-06-03T01:21:20.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/887: chown de/d29/d2e/d31/d3c/d50/d62/fb1 5047 1 2024-06-03T01:21:20.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/888: stat de/d63/c10f 0 2024-06-03T01:21:20.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/889: truncate de/f8b 4839945 0 2024-06-03T01:21:20.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/890: write de/d29/d2e/d31/d23/d53/ffc [994538,110834] 0 2024-06-03T01:21:20.159 INFO:tasks.workunit.client.0.smithi032.stdout:3/891: symlink de/d29/d113/l11a 0 2024-06-03T01:21:20.159 INFO:tasks.workunit.client.0.smithi032.stdout:3/892: chown de/d29/f3f 4684 1 2024-06-03T01:21:20.173 INFO:tasks.workunit.client.0.smithi032.stdout:3/893: write de/d29/d2e/d31/f1c [684985,61858] 0 2024-06-03T01:21:20.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/894: mknod de/d29/d2e/d31/da1/c11b 0 2024-06-03T01:21:20.176 INFO:tasks.workunit.client.0.smithi032.stdout:3/895: getdents de/d29/d2e/d31/d23/d53/d79 0 2024-06-03T01:21:20.176 INFO:tasks.workunit.client.0.smithi032.stdout:3/896: read f8 [4378540,26531] 0 2024-06-03T01:21:20.176 INFO:tasks.workunit.client.0.smithi032.stdout:3/897: dread - de/d29/d2e/d31/d8d/d107/ff6 zero size 2024-06-03T01:21:20.177 INFO:tasks.workunit.client.0.smithi032.stdout:3/898: truncate de/d29/d2e/d31/d8d/de1/fec 2666125 0 2024-06-03T01:21:20.178 INFO:tasks.workunit.client.0.smithi032.stdout:3/899: mkdir de/d29/d2e/d31/d3c/d50/d11c 0 2024-06-03T01:21:20.179 INFO:tasks.workunit.client.0.smithi032.stdout:3/900: creat de/d29/d2e/d31/d3c/d111/f11d x:0 0 0 2024-06-03T01:21:20.212 INFO:tasks.workunit.client.0.smithi032.stdout:3/901: rename de/d29/d2e/d31/d23/d53/d79/dab/le3 to de/d29/d2e/d31/d3c/d111/l11e 0 2024-06-03T01:21:20.212 INFO:tasks.workunit.client.0.smithi032.stdout:3/902: fsync de/d29/d2e/d31/d3c/d50/ff4 0 2024-06-03T01:21:20.212 INFO:tasks.workunit.client.0.smithi032.stdout:3/903: write de/d29/d2e/d31/d3c/d50/d62/fb8 [103406,37145] 0 2024-06-03T01:21:20.213 INFO:tasks.workunit.client.0.smithi032.stdout:3/904: write de/d29/fa7 [653700,93267] 0 2024-06-03T01:21:20.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/905: unlink de/ce4 0 2024-06-03T01:21:20.464 INFO:tasks.workunit.client.0.smithi032.stdout:3/906: creat de/d29/d2e/d31/d3c/d50/d56/f11f x:0 0 0 2024-06-03T01:21:20.466 INFO:tasks.workunit.client.0.smithi032.stdout:3/907: dread de/d29/d2e/d31/d23/f4a [0,4194304] 0 2024-06-03T01:21:20.466 INFO:tasks.workunit.client.0.smithi032.stdout:3/908: readlink de/d29/l115 0 2024-06-03T01:21:20.483 INFO:tasks.workunit.client.0.smithi032.stdout:3/909: getdents de/d29/d2e/d31/d23/d53/d69/d8c 0 2024-06-03T01:21:20.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/910: symlink de/d29/d2e/d31/da1/l120 0 2024-06-03T01:21:20.485 INFO:tasks.workunit.client.0.smithi032.stdout:3/911: creat de/d29/d2e/d31/da1/f121 x:0 0 0 2024-06-03T01:21:20.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/912: dwrite de/d29/d2e/d31/d23/d53/d69/d8c/fcf [0,4194304] 0 2024-06-03T01:21:20.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/913: chown de/d29/d2e/d31/d3c/d50/d62/ld2 15 1 2024-06-03T01:21:20.718 INFO:tasks.workunit.client.0.smithi032.stdout:1/944: sync 2024-06-03T01:21:20.718 INFO:tasks.workunit.client.0.smithi032.stdout:9/987: sync 2024-06-03T01:21:20.718 INFO:tasks.workunit.client.0.smithi032.stdout:6/913: sync 2024-06-03T01:21:20.718 INFO:tasks.workunit.client.0.smithi032.stdout:9/988: chown d0/d11/d1d/dd5/f112 26 1 2024-06-03T01:21:20.719 INFO:tasks.workunit.client.0.smithi032.stdout:1/945: write d18/faf [489336,76584] 0 2024-06-03T01:21:20.724 INFO:tasks.workunit.client.0.smithi032.stdout:1/946: dread d18/d3f/d5f/f84 [0,4194304] 0 2024-06-03T01:21:20.724 INFO:tasks.workunit.client.0.smithi032.stdout:1/947: write d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/fc9 [611863,47424] 0 2024-06-03T01:21:20.725 INFO:tasks.workunit.client.0.smithi032.stdout:1/948: chown d18/d27/d2b/c75 6 1 2024-06-03T01:21:20.725 INFO:tasks.workunit.client.0.smithi032.stdout:1/949: truncate d18/d3f/d5f/d94/f103 386387 0 2024-06-03T01:21:20.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/914: write d8/d11/d20/d31/d3f/ffe [3825430,57300] 0 2024-06-03T01:21:20.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/915: write d8/d55/fe9 [1167184,29526] 0 2024-06-03T01:21:20.732 INFO:tasks.workunit.client.0.smithi032.stdout:6/916: truncate d8/d11/d20/d2b/d39/f67 1097489 0 2024-06-03T01:21:20.732 INFO:tasks.workunit.client.0.smithi032.stdout:6/917: creat d8/d11/d20/d31/df2/df8/f127 x:0 0 0 2024-06-03T01:21:20.733 INFO:tasks.workunit.client.0.smithi032.stdout:9/989: dread d0/d11/d1d/f47 [0,4194304] 0 2024-06-03T01:21:20.752 INFO:tasks.workunit.client.0.smithi032.stdout:1/950: dwrite d18/d32/d39/fc5 [0,4194304] 0 2024-06-03T01:21:20.753 INFO:tasks.workunit.client.0.smithi032.stdout:6/918: dread d8/d4c/fa0 [0,4194304] 0 2024-06-03T01:21:20.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/951: getdents d18/d32/daa 0 2024-06-03T01:21:20.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/952: readlink d18/d3f/d5f/d98/le7 0 2024-06-03T01:21:20.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/953: fdatasync d18/d3f/d5f/d94/f97 0 2024-06-03T01:21:20.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/954: chown d18/d3f/d5f/d6b/fd5 32381 1 2024-06-03T01:21:20.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/955: creat d18/d3f/d5f/da6/f123 x:0 0 0 2024-06-03T01:21:20.756 INFO:tasks.workunit.client.0.smithi032.stdout:6/919: dread d8/d11/d20/d3b/fad [0,4194304] 0 2024-06-03T01:21:20.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/956: truncate d18/d32/fdf 1854862 0 2024-06-03T01:21:20.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/957: read - d18/d3f/d5f/d6b/d78/d3d/df3/dfc/f120 zero size 2024-06-03T01:21:20.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/958: truncate d18/d32/d39/d60/fd6 602521 0 2024-06-03T01:21:20.796 INFO:tasks.workunit.client.0.smithi032.stdout:1/959: chown d18/d3f/d5f/d6b/d78/d3d/df3/dfc/l121 47 1 2024-06-03T01:21:20.797 INFO:tasks.workunit.client.0.smithi032.stdout:6/920: mknod d8/d11/d20/c128 0 2024-06-03T01:21:20.797 INFO:tasks.workunit.client.0.smithi032.stdout:6/921: write d8/dd5/f64 [667339,34371] 0 2024-06-03T01:21:20.798 INFO:tasks.workunit.client.0.smithi032.stdout:6/922: link d8/d11/d20/d7f/l9d d8/d11/d20/d7f/l129 0 2024-06-03T01:21:20.800 INFO:tasks.workunit.client.0.smithi032.stdout:6/923: creat d8/d11/d20/d2b/d39/d7e/f12a x:0 0 0 2024-06-03T01:21:20.800 INFO:tasks.workunit.client.0.smithi032.stdout:6/924: write d8/d11/d20/d7f/f91 [41962,93004] 0 2024-06-03T01:21:20.803 INFO:tasks.workunit.client.0.smithi032.stdout:6/925: mkdir d8/dd5/da5/d10a/d12b 0 2024-06-03T01:21:20.803 INFO:tasks.workunit.client.0.smithi032.stdout:6/926: write d8/d11/d20/d7f/ffc [799679,113973] 0 2024-06-03T01:21:20.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/927: dread f4 [4194304,4194304] 0 2024-06-03T01:21:20.814 INFO:tasks.workunit.client.0.smithi032.stdout:6/928: unlink d8/d11/fe8 0 2024-06-03T01:21:20.814 INFO:tasks.workunit.client.0.smithi032.stdout:6/929: dread - d8/d11/d20/d21/d9a/d102/f11c zero size 2024-06-03T01:21:20.815 INFO:tasks.workunit.client.0.smithi032.stdout:6/930: write d8/d11/d20/d7f/ffc [677721,31342] 0 2024-06-03T01:21:20.817 INFO:tasks.workunit.client.0.smithi032.stdout:6/931: rename d8/dd5/c11a to d8/d50/dda/dfd/c12c 0 2024-06-03T01:21:20.817 INFO:tasks.workunit.client.0.smithi032.stdout:1/960: dwrite d18/d32/d39/fc5 [0,4194304] 0 2024-06-03T01:21:20.823 INFO:tasks.workunit.client.0.smithi032.stdout:1/961: rmdir d18/d2d 39 2024-06-03T01:21:20.825 INFO:tasks.workunit.client.0.smithi032.stdout:1/962: getdents d18/d3f/d5f/d6b/d78/d3d 0 2024-06-03T01:21:20.827 INFO:tasks.workunit.client.0.smithi032.stdout:1/963: rename d18/d3f/d5f/d6b/cce to d18/d3f/d5f/d6b/d9d/c124 0 2024-06-03T01:21:20.830 INFO:tasks.workunit.client.0.smithi032.stdout:1/964: link d18/d3f/d5f/d6b/f81 d18/d3f/d5f/d6b/d78/f125 0 2024-06-03T01:21:20.847 INFO:tasks.workunit.client.0.smithi032.stdout:1/965: dwrite d18/d3f/d5f/d6b/d78/d3d/d47/d6e/f9b [0,4194304] 0 2024-06-03T01:21:20.848 INFO:tasks.workunit.client.0.smithi032.stdout:1/966: rmdir d18/d32/daa 39 2024-06-03T01:21:20.849 INFO:tasks.workunit.client.0.smithi032.stdout:1/967: mkdir d18/d32/d126 0 2024-06-03T01:21:20.852 INFO:tasks.workunit.client.0.smithi032.stdout:1/968: rmdir d18/d27/d2b 39 2024-06-03T01:21:20.856 INFO:tasks.workunit.client.0.smithi032.stdout:1/969: rmdir d18/d3f/d5f/d6b/d78/de3/df9 0 2024-06-03T01:21:20.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/970: fdatasync d18/d3f/d5f/d7b/ffb 0 2024-06-03T01:21:20.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/971: fdatasync d18/d27/fbe 0 2024-06-03T01:21:20.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/972: stat d18/d3f/d5f/d7b/ld1 0 2024-06-03T01:21:20.879 INFO:tasks.workunit.client.0.smithi032.stdout:1/973: dwrite d18/d27/fbe [0,4194304] 0 2024-06-03T01:21:20.879 INFO:tasks.workunit.client.0.smithi032.stdout:1/974: truncate d18/d32/d39/d60/fd6 662620 0 2024-06-03T01:21:20.880 INFO:tasks.workunit.client.0.smithi032.stdout:1/975: mkdir d18/d3f/d5f/da6/de1/d127 0 2024-06-03T01:21:20.895 INFO:tasks.workunit.client.0.smithi032.stdout:1/976: dread d18/f8e [0,4194304] 0 2024-06-03T01:21:20.897 INFO:tasks.workunit.client.0.smithi032.stdout:1/977: link d18/d2d/d5e/c109 d18/d3f/d5f/da6/de1/d10a/c128 0 2024-06-03T01:21:20.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/978: rename d18/d3f/d5f/d98/fa2 to d18/d27/d2b/f129 0 2024-06-03T01:21:20.900 INFO:tasks.workunit.client.0.smithi032.stdout:1/979: creat d18/d3f/d5f/d6b/d78/d3d/d47/d6e/dfa/de6/f12a x:0 0 0 2024-06-03T01:21:20.904 INFO:tasks.workunit.client.0.smithi032.stdout:1/980: creat d18/d3f/d5f/da6/de1/d10a/f12b x:0 0 0 2024-06-03T01:21:20.907 INFO:tasks.workunit.client.0.smithi032.stdout:1/981: unlink d18/c9a 0 2024-06-03T01:21:20.910 INFO:tasks.workunit.client.0.smithi032.stdout:1/982: rename d18/d3f/d5f/d6b/d78/d3d/d6a/l100 to d18/d32/d39/d60/l12c 0 2024-06-03T01:21:20.911 INFO:tasks.workunit.client.0.smithi032.stdout:1/983: symlink d18/d32/daa/l12d 0 2024-06-03T01:21:20.914 INFO:tasks.workunit.client.0.smithi032.stdout:1/984: creat d18/d32/d126/f12e x:0 0 0 2024-06-03T01:21:20.937 INFO:tasks.workunit.client.0.smithi032.stdout:3/914: sync 2024-06-03T01:21:20.939 INFO:tasks.workunit.client.0.smithi032.stdout:3/915: rename de/d29/d2e/d31/d23/d53/dbf/l104 to de/d29/d2e/d31/d23/d53/dbf/dee/l122 0 2024-06-03T01:21:20.939 INFO:tasks.workunit.client.0.smithi032.stdout:1/985: dwrite d18/d27/da7/ddb/f106 [0,4194304] 0 2024-06-03T01:21:20.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/916: write de/d29/d2e/d31/d3c/d50/d62/fb1 [3098170,105065] 0 2024-06-03T01:21:20.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/917: dread - de/d29/d2e/d31/d8d/d107/ff6 zero size 2024-06-03T01:21:20.949 INFO:tasks.workunit.client.0.smithi032.stdout:3/918: rename de/d29/d2e/d31/d24/f25 to de/d29/d2e/d31/d23/d53/dbf/d119/f123 0 2024-06-03T01:21:20.950 INFO:tasks.workunit.client.0.smithi032.stdout:3/919: truncate de/d29/d2e/d31/d8d/dc3/f89 1284282 0 2024-06-03T01:21:20.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/986: dwrite d18/d32/d39/d60/fd6 [0,4194304] 0 2024-06-03T01:21:20.966 INFO:tasks.workunit.client.0.smithi032.stdout:3/920: dwrite de/f8b [0,4194304] 0 2024-06-03T01:21:20.968 INFO:tasks.workunit.client.0.smithi032.stdout:1/987: rename d18/d3f/d5f/d7b/ffb to d18/d3f/d5f/d6b/d78/d3d/d47/d6e/d91/df8/f12f 0 2024-06-03T01:21:20.970 INFO:tasks.workunit.client.0.smithi032.stdout:1/988: unlink d18/d27/l3a 0 2024-06-03T01:21:20.970 INFO:tasks.workunit.client.0.smithi032.stdout:1/989: creat d18/d3f/d5f/d94/f130 x:0 0 0 2024-06-03T01:21:20.975 INFO:tasks.workunit.client.0.smithi032.stdout:3/921: dread de/d29/d2e/d31/d23/d53/d69/d8c/fa9 [0,4194304] 0 2024-06-03T01:21:20.977 INFO:tasks.workunit.client.0.smithi032.stdout:3/922: dread de/fdb [0,4194304] 0 2024-06-03T01:21:20.978 INFO:tasks.workunit.client.0.smithi032.stdout:3/923: creat de/d29/d2e/d31/d3c/d50/f124 x:0 0 0 2024-06-03T01:21:20.978 INFO:tasks.workunit.client.0.smithi032.stdout:3/924: chown de/d29/f3f 86909079 1 2024-06-03T01:21:20.980 INFO:tasks.workunit.client.0.smithi032.stdout:3/925: creat de/d29/d2e/d31/d23/d53/f125 x:0 0 0 2024-06-03T01:21:20.980 INFO:tasks.workunit.client.0.smithi032.stdout:3/926: write de/d29/d2e/d31/d3c/d50/d62/fd9 [480209,59622] 0 2024-06-03T01:21:20.981 INFO:tasks.workunit.client.0.smithi032.stdout:1/990: dread d18/d3f/d5f/d6b/d9d/f8b [0,4194304] 0 2024-06-03T01:21:20.981 INFO:tasks.workunit.client.0.smithi032.stdout:3/927: unlink de/d29/d2e/d31/d3c/l94 0 2024-06-03T01:21:20.982 INFO:tasks.workunit.client.0.smithi032.stdout:1/991: write d18/f1a [1039212,125820] 0 2024-06-03T01:21:20.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/928: link de/d29/d2e/d31/d23/d53/d69/f8e de/d29/d2e/d31/d23/d53/d79/f126 0 2024-06-03T01:21:20.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/929: chown de/d29/d2e/d31/d23/d53 7 1 2024-06-03T01:21:20.985 INFO:tasks.workunit.client.0.smithi032.stdout:1/992: creat d18/d3f/d5f/d7b/f131 x:0 0 0 2024-06-03T01:21:20.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/993: getdents d18/d3f/d5f/d6b/d78/d3d/d47 0 2024-06-03T01:21:20.988 INFO:tasks.workunit.client.0.smithi032.stdout:1/994: mknod d18/d3f/d5f/c132 0 2024-06-03T01:21:20.989 INFO:tasks.workunit.client.0.smithi032.stdout:1/995: symlink d18/d3f/d5f/d94/l133 0 2024-06-03T01:21:20.989 INFO:tasks.workunit.client.0.smithi032.stdout:1/996: chown d18/d3f/c43 62 1 2024-06-03T01:21:20.993 INFO:tasks.workunit.client.0.smithi032.stdout:1/997: write d18/d32/fdd [946750,75741] 0 2024-06-03T01:21:20.994 INFO:tasks.workunit.client.0.smithi032.stdout:3/930: fsync de/d29/d2e/d31/d23/d53/d69/f8e 0 2024-06-03T01:21:20.994 INFO:tasks.workunit.client.0.smithi032.stdout:3/931: dread - de/d29/d2e/d31/d23/d53/d69/fcb zero size 2024-06-03T01:21:20.996 INFO:tasks.workunit.client.0.smithi032.stdout:3/932: creat de/d29/d2e/d31/d23/d53/dbf/f127 x:0 0 0 2024-06-03T01:21:21.003 INFO:tasks.workunit.client.0.smithi032.stdout:1/998: dread d18/d3f/d5f/d6b/d78/d3d/d6a/fa1 [0,4194304] 0 2024-06-03T01:21:21.003 INFO:tasks.workunit.client.0.smithi032.stdout:3/933: dread de/d29/d2e/d31/d8d/de1/fec [0,4194304] 0 2024-06-03T01:21:21.005 INFO:tasks.workunit.client.0.smithi032.stdout:3/934: link de/d29/d2e/d31/da1/l120 de/d29/d2e/d31/d8d/d107/l128 0 2024-06-03T01:21:21.005 INFO:tasks.workunit.client.0.smithi032.stdout:3/935: getdents de/d29/d2e/d31/d3c/d50/d66/d110 0 2024-06-03T01:21:21.005 INFO:tasks.workunit.client.0.smithi032.stdout:3/936: creat de/d29/d2e/d31/d3c/d50/d56/de7/f129 x:0 0 0 2024-06-03T01:21:21.024 INFO:tasks.workunit.client.0.smithi032.stdout:1/999: dwrite d18/d32/fdf [0,4194304] 0 2024-06-03T01:21:21.024 INFO:tasks.workunit.client.0.smithi032.stdout:3/937: dwrite de/d29/d2e/d31/d8d/faa [0,4194304] 0 2024-06-03T01:21:21.024 INFO:tasks.workunit.client.0.smithi032.stdout:3/938: read de/d29/d2e/d31/d3c/f44 [3963115,78723] 0 2024-06-03T01:21:21.026 INFO:tasks.workunit.client.0.smithi032.stdout:3/939: link de/d29/d2e/d31/d23/d53/dbf/dee/l122 de/d29/d2e/d31/d3c/d50/d11c/l12a 0 2024-06-03T01:21:21.034 INFO:tasks.workunit.client.0.smithi032.stdout:3/940: read de/d29/d2e/d31/d3c/d50/d56/fef [1225184,33451] 0 2024-06-03T01:21:21.035 INFO:tasks.workunit.client.0.smithi032.stdout:3/941: link de/d29/d2e/d31/d23/d53/d69/dca/c116 de/d29/d2e/d31/d3c/d50/d66/c12b 0 2024-06-03T01:21:21.036 INFO:tasks.workunit.client.0.smithi032.stdout:3/942: read de/fdb [197294,123565] 0 2024-06-03T01:21:21.038 INFO:tasks.workunit.client.0.smithi032.stdout:3/943: creat de/d29/d2e/d31/d23/d53/d69/f12c x:0 0 0 2024-06-03T01:21:21.039 INFO:tasks.workunit.client.0.smithi032.stdout:3/944: write de/d29/d2e/d31/d3c/d50/f9b [1028111,114989] 0 2024-06-03T01:21:21.039 INFO:tasks.workunit.client.0.smithi032.stdout:3/945: chown de/d29/d2e/d31/d24 54 1 2024-06-03T01:21:21.040 INFO:tasks.workunit.client.0.smithi032.stdout:3/946: dread de/d29/d2e/d31/d3c/d50/d66/f9a [0,4194304] 0 2024-06-03T01:21:21.043 INFO:tasks.workunit.client.0.smithi032.stdout:3/947: symlink de/d29/d2e/d31/d3c/d50/d11c/l12d 0 2024-06-03T01:21:21.049 INFO:tasks.workunit.client.0.smithi032.stdout:3/948: rmdir de/d29/d2e/d31/d23/d53/d69 39 2024-06-03T01:21:21.049 INFO:tasks.workunit.client.0.smithi032.stdout:3/949: fdatasync de/d29/d2e/d31/d3c/d50/d62/fb1 0 2024-06-03T01:21:21.049 INFO:tasks.workunit.client.0.smithi032.stdout:3/950: stat de/d29/d2e/d31/d23/d53/d69/d8c/db4/lfe 0 2024-06-03T01:21:21.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/951: symlink de/d29/d2e/d31/d23/d53/dbf/d119/l12e 0 2024-06-03T01:21:21.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/952: dread - de/d29/d2e/d31/d3c/d50/ff7 zero size 2024-06-03T01:21:21.067 INFO:tasks.workunit.client.0.smithi032.stdout:3/953: dwrite de/d29/d2e/d31/d24/f2f [0,4194304] 0 2024-06-03T01:21:21.068 INFO:tasks.workunit.client.0.smithi032.stdout:3/954: creat de/d29/d2e/d31/d23/d53/dbf/d119/f12f x:0 0 0 2024-06-03T01:21:21.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/955: mknod de/d29/d2e/d31/d23/d53/dbf/d119/c130 0 2024-06-03T01:21:21.084 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:20 smithi032 ceph-mon[29360]: pgmap v15: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 119 MiB/s wr, 269 op/s 2024-06-03T01:21:21.087 INFO:tasks.workunit.client.0.smithi032.stdout:3/956: dwrite de/d29/d2e/d31/d23/d53/d69/f8e [4194304,4194304] 0 2024-06-03T01:21:21.087 INFO:tasks.workunit.client.0.smithi032.stdout:3/957: stat de/d29/d2e/d31/d24/l42 0 2024-06-03T01:21:21.087 INFO:tasks.workunit.client.0.smithi032.stdout:3/958: chown de/d29/d2e/d31/d8d/dc3/c103 352 1 2024-06-03T01:21:21.091 INFO:tasks.workunit.client.0.smithi032.stdout:3/959: unlink de/d29/fc6 0 2024-06-03T01:21:21.100 INFO:tasks.workunit.client.0.smithi032.stdout:3/960: write de/d29/f7f [955993,117918] 0 2024-06-03T01:21:21.103 INFO:tasks.workunit.client.0.smithi032.stdout:3/961: mkdir de/d131 0 2024-06-03T01:21:21.103 INFO:tasks.workunit.client.0.smithi032.stdout:3/962: creat de/d29/d2e/d31/d3c/d50/d66/f132 x:0 0 0 2024-06-03T01:21:21.103 INFO:tasks.workunit.client.0.smithi032.stdout:3/963: chown de/d29/d2e/d31/d23/d53/d69/fd6 2 1 2024-06-03T01:21:21.103 INFO:tasks.workunit.client.0.smithi032.stdout:6/932: sync 2024-06-03T01:21:21.104 INFO:tasks.workunit.client.0.smithi032.stdout:6/933: chown d8/d11/d20/d21/l85 192980817 1 2024-06-03T01:21:21.104 INFO:tasks.workunit.client.0.smithi032.stdout:9/990: sync 2024-06-03T01:21:21.104 INFO:tasks.workunit.client.0.smithi032.stdout:9/991: chown d0/d11/d1d/d38/d4f/db0/d41/dc6/dc9/feb 26100 1 2024-06-03T01:21:21.104 INFO:tasks.workunit.client.0.smithi032.stdout:9/992: truncate d0/d11/d9c/dd4/d11b/d25/dd6/fe2 4797094 0 2024-06-03T01:21:21.104 INFO:tasks.workunit.client.0.smithi032.stdout:9/993: stat d0/d11/d1d/d38/d4f/db0 0 2024-06-03T01:21:21.105 INFO:tasks.workunit.client.0.smithi032.stdout:9/994: read d0/d11/d1d/da6/fd0 [464650,131032] 0 2024-06-03T01:21:21.106 INFO:tasks.workunit.client.0.smithi032.stdout:6/934: creat d8/d11/d20/d3b/d66/f12d x:0 0 0 2024-06-03T01:21:21.108 INFO:tasks.workunit.client.0.smithi032.stdout:6/935: creat d8/d4c/daa/d10c/f12e x:0 0 0 2024-06-03T01:21:21.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/964: dread de/f8b [0,4194304] 0 2024-06-03T01:21:21.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/965: chown de/d29/d2e/d31/d8d/dc3/fa6 104 1 2024-06-03T01:21:21.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/966: creat de/d29/d2e/d31/d23/d53/dbf/dee/d102/f133 x:0 0 0 2024-06-03T01:21:21.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/967: fdatasync de/d29/d2e/d31/d23/d53/dbf/dee/d102/f133 0 2024-06-03T01:21:21.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/968: readlink de/d29/d2e/lc2 0 2024-06-03T01:21:21.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/995: write d0/d11/d1d/d38/d4f/db0/f39 [1442322,17514] 0 2024-06-03T01:21:21.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/996: write d0/d11/d9c/dd4/d11b/d24/f138 [942656,66350] 0 2024-06-03T01:21:21.116 INFO:tasks.workunit.client.0.smithi032.stdout:3/969: link de/d29/d2e/d31/d3c/c47 de/d29/d2e/d31/d8d/dc3/c134 0 2024-06-03T01:21:21.116 INFO:tasks.workunit.client.0.smithi032.stdout:3/970: fsync de/d29/d2e/d31/d3c/d50/ff4 0 2024-06-03T01:21:21.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/971: truncate de/d29/d2e/d31/d23/f4a 663384 0 2024-06-03T01:21:21.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/972: write de/d29/d2e/d31/d3c/d50/ff7 [879909,112248] 0 2024-06-03T01:21:21.123 INFO:tasks.workunit.client.0.smithi032.stdout:3/973: dread de/d29/d2e/d31/d8d/de1/fec [0,4194304] 0 2024-06-03T01:21:21.125 INFO:tasks.workunit.client.0.smithi032.stdout:3/974: read de/d29/d2e/d31/d3c/d50/d62/fd9 [47951,42138] 0 2024-06-03T01:21:21.125 INFO:tasks.workunit.client.0.smithi032.stdout:3/975: write de/d63/f7a [638042,96134] 0 2024-06-03T01:21:21.143 INFO:tasks.workunit.client.0.smithi032.stdout:9/997: dwrite d0/d11/d9c/dd4/d11b/d25/f62 [4194304,4194304] 0 2024-06-03T01:21:21.145 INFO:tasks.workunit.client.0.smithi032.stdout:9/998: mkdir d0/d11/d9c/d13a 0 2024-06-03T01:21:21.145 INFO:tasks.workunit.client.0.smithi032.stdout:3/976: dwrite de/d29/d2e/d31/d8d/d107/ff6 [0,4194304] 0 2024-06-03T01:21:21.145 INFO:tasks.workunit.client.0.smithi032.stdout:9/999: dread - d0/d11/d9c/dd4/d11b/d25/de1/fe5 zero size 2024-06-03T01:21:21.169 INFO:tasks.workunit.client.0.smithi032.stdout:3/977: dwrite de/f101 [0,4194304] 0 2024-06-03T01:21:21.169 INFO:tasks.workunit.client.0.smithi032.stdout:3/978: write de/d29/d2e/d31/d3c/d50/d56/de7/f112 [743917,28663] 0 2024-06-03T01:21:21.169 INFO:tasks.workunit.client.0.smithi032.stdout:3/979: creat de/d29/d2e/d31/d8d/de1/f135 x:0 0 0 2024-06-03T01:21:21.173 INFO:tasks.workunit.client.0.smithi032.stdout:3/980: mknod de/d29/d2e/d31/d23/d53/d69/d90/c136 0 2024-06-03T01:21:21.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/981: truncate de/d29/d2e/d31/d8d/faa 1285803 0 2024-06-03T01:21:21.177 INFO:tasks.workunit.client.0.smithi032.stdout:3/982: rename de/d29/d2e/d31/d3c/d50/d62/fd9 to de/d29/d2e/d31/d23/f137 0 2024-06-03T01:21:21.193 INFO:tasks.workunit.client.0.smithi032.stdout:3/983: dwrite de/d29/d2e/d31/d3c/d50/d56/de7/f129 [0,4194304] 0 2024-06-03T01:21:21.197 INFO:tasks.workunit.client.0.smithi032.stdout:3/984: unlink de/d29/d2e/d31/d8d/dc3/fa6 0 2024-06-03T01:21:21.200 INFO:tasks.workunit.client.0.smithi032.stdout:3/985: rename de/c40 to de/d29/d2e/d31/d23/d53/d69/dca/c138 0 2024-06-03T01:21:21.200 INFO:tasks.workunit.client.0.smithi032.stdout:3/986: write de/d29/f2b [4034807,119653] 0 2024-06-03T01:21:21.208 INFO:tasks.workunit.client.0.smithi032.stdout:3/987: dread de/d29/d2e/d31/d3c/d50/d56/de7/f129 [0,4194304] 0 2024-06-03T01:21:21.210 INFO:tasks.workunit.client.0.smithi032.stdout:3/988: symlink de/d29/d2e/d31/d23/d53/d69/dca/l139 0 2024-06-03T01:21:21.216 INFO:tasks.workunit.client.0.smithi032.stdout:3/989: write de/d29/f65 [2848705,122717] 0 2024-06-03T01:21:21.217 INFO:tasks.workunit.client.0.smithi032.stdout:3/990: symlink de/d63/l13a 0 2024-06-03T01:21:21.217 INFO:tasks.workunit.client.0.smithi032.stdout:3/991: chown de/d29/d2e/d31/d24/daf 59 1 2024-06-03T01:21:21.218 INFO:tasks.workunit.client.0.smithi032.stdout:3/992: mknod de/d29/d2e/d31/d23/d53/dbf/d119/c13b 0 2024-06-03T01:21:21.218 INFO:tasks.workunit.client.0.smithi032.stdout:3/993: mkdir de/d29/d2e/d31/d3c/d50/d13c 0 2024-06-03T01:21:21.219 INFO:tasks.workunit.client.0.smithi032.stdout:3/994: creat de/d29/d2e/d31/d23/d53/d69/f13d x:0 0 0 2024-06-03T01:21:21.220 INFO:tasks.workunit.client.0.smithi032.stdout:3/995: unlink de/d29/d113/l11a 0 2024-06-03T01:21:21.221 INFO:tasks.workunit.client.0.smithi032.stdout:3/996: stat de/d29/d2e/d31/d3c/d50/d56/c82 0 2024-06-03T01:21:21.221 INFO:tasks.workunit.client.0.smithi032.stdout:3/997: stat de/d29/d2e/d31/d23/d53/d69/lf8 0 2024-06-03T01:21:21.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:20 smithi044 ceph-mon[36956]: pgmap v15: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 119 MiB/s wr, 269 op/s 2024-06-03T01:21:21.221 INFO:tasks.workunit.client.0.smithi032.stdout:3/998: truncate f5 12105430 0 2024-06-03T01:21:21.222 INFO:tasks.workunit.client.0.smithi032.stdout:3/999: creat de/d29/d2e/d31/f13e x:0 0 0 2024-06-03T01:21:21.331 INFO:tasks.workunit.client.0.smithi032.stdout:6/936: sync 2024-06-03T01:21:21.332 INFO:tasks.workunit.client.0.smithi032.stdout:6/937: mknod d8/dd5/da5/d10a/c12f 0 2024-06-03T01:21:21.332 INFO:tasks.workunit.client.0.smithi032.stdout:6/938: dread - d8/d11/d20/d21/d9a/d102/f11c zero size 2024-06-03T01:21:21.332 INFO:tasks.workunit.client.0.smithi032.stdout:6/939: getdents d8/d11/d20/d31/df2/df8/d11f 0 2024-06-03T01:21:21.332 INFO:tasks.workunit.client.0.smithi032.stdout:6/940: truncate d8/d11/d20/d31/df2/df8/f127 878628 0 2024-06-03T01:21:21.333 INFO:tasks.workunit.client.0.smithi032.stdout:6/941: creat d8/d11/d20/d21/d9a/d102/f130 x:0 0 0 2024-06-03T01:21:21.340 INFO:tasks.workunit.client.0.smithi032.stdout:6/942: dread d8/d11/d20/d3b/f80 [0,4194304] 0 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/943: dread d8/d11/d20/d2b/f4a [0,4194304] 0 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/944: truncate d8/d11/d20/f121 897162 0 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/945: truncate d8/d50/fd3 1040805 0 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/946: chown d8 208173 1 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/947: getdents d8/d5f 0 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/948: chown d8/d11/d20/d21/d9a/fdc 1040800490 1 2024-06-03T01:21:21.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/949: creat d8/d4c/daa/f131 x:0 0 0 2024-06-03T01:21:21.349 INFO:tasks.workunit.client.0.smithi032.stdout:6/950: mknod d8/d11/d20/d31/df2/df8/d107/c132 0 2024-06-03T01:21:21.349 INFO:tasks.workunit.client.0.smithi032.stdout:6/951: write d8/d11/d20/d31/df2/df8/d107/f123 [998540,60813] 0 2024-06-03T01:21:21.349 INFO:tasks.workunit.client.0.smithi032.stdout:6/952: creat d8/d11/d20/d21/f133 x:0 0 0 2024-06-03T01:21:21.356 INFO:tasks.workunit.client.0.smithi032.stdout:6/953: dread d8/d11/d20/d2b/d39/fed [0,4194304] 0 2024-06-03T01:21:21.357 INFO:tasks.workunit.client.0.smithi032.stdout:6/954: chown d8/d11/d20/d21/f113 16669628 1 2024-06-03T01:21:21.371 INFO:tasks.workunit.client.0.smithi032.stdout:6/955: dwrite d8/dd5/fef [0,4194304] 0 2024-06-03T01:21:21.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/956: mkdir d8/d11/d20/d31/df2/df8/d134 0 2024-06-03T01:21:21.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/957: write d8/d11/d20/d31/d3f/f69 [531099,125608] 0 2024-06-03T01:21:21.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/958: fdatasync d8/d11/d20/d3b/fd1 0 2024-06-03T01:21:21.394 INFO:tasks.workunit.client.0.smithi032.stdout:6/959: dwrite d8/d11/d20/d2b/d39/d19/f46 [4194304,4194304] 0 2024-06-03T01:21:21.408 INFO:tasks.workunit.client.0.smithi032.stdout:6/960: dwrite d8/d55/f110 [4194304,4194304] 0 2024-06-03T01:21:21.424 INFO:tasks.workunit.client.0.smithi032.stdout:6/961: dwrite d8/d11/d20/d3b/f41 [0,4194304] 0 2024-06-03T01:21:21.425 INFO:tasks.workunit.client.0.smithi032.stdout:6/962: symlink d8/d11/d20/d21/d9a/l135 0 2024-06-03T01:21:21.425 INFO:tasks.workunit.client.0.smithi032.stdout:6/963: readlink d8/d55/l5e 0 2024-06-03T01:21:21.425 INFO:tasks.workunit.client.0.smithi032.stdout:6/964: chown d8/d11/d20/d2b/d39/d19/l98 8504 1 2024-06-03T01:21:21.425 INFO:tasks.workunit.client.0.smithi032.stdout:6/965: getdents d8/dd5/da5/d10a/d12b 0 2024-06-03T01:21:21.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/966: creat d8/dd5/f136 x:0 0 0 2024-06-03T01:21:21.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/967: chown d8/d55/l5c 1068123184 1 2024-06-03T01:21:21.432 INFO:tasks.workunit.client.0.smithi032.stdout:6/968: read d8/d4c/f58 [2663407,112142] 0 2024-06-03T01:21:21.434 INFO:tasks.workunit.client.0.smithi032.stdout:6/969: rename d8/d11/d20/d3b/fad to d8/d5f/f137 0 2024-06-03T01:21:21.434 INFO:tasks.workunit.client.0.smithi032.stdout:6/970: fsync d8/fb 0 2024-06-03T01:21:21.434 INFO:tasks.workunit.client.0.smithi032.stdout:6/971: truncate d8/d11/d20/d2b/d39/d19/ff4 2974333 0 2024-06-03T01:21:21.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/972: creat d8/dd5/da5/d10a/f138 x:0 0 0 2024-06-03T01:21:21.445 INFO:tasks.workunit.client.0.smithi032.stdout:6/973: dread d8/d4c/daa/fb7 [0,4194304] 0 2024-06-03T01:21:21.466 INFO:tasks.workunit.client.0.smithi032.stdout:6/974: dwrite d8/dd5/f38 [0,4194304] 0 2024-06-03T01:21:21.922 INFO:tasks.workunit.client.0.smithi032.stdout:6/975: sync 2024-06-03T01:21:22.147 INFO:tasks.workunit.client.0.smithi032.stdout:6/976: sync 2024-06-03T01:21:22.149 INFO:tasks.workunit.client.0.smithi032.stdout:6/977: mknod d8/d4c/c139 0 2024-06-03T01:21:22.150 INFO:tasks.workunit.client.0.smithi032.stdout:6/978: symlink d8/dd5/da5/dc8/l13a 0 2024-06-03T01:21:22.150 INFO:tasks.workunit.client.0.smithi032.stdout:6/979: creat d8/d11/d20/d2b/d39/d7e/f13b x:0 0 0 2024-06-03T01:21:22.152 INFO:tasks.workunit.client.0.smithi032.stdout:6/980: getdents d8/d11/db2 0 2024-06-03T01:21:22.178 INFO:tasks.workunit.client.0.smithi032.stdout:6/981: dwrite d8/d11/d20/d2b/d39/f10f [0,4194304] 0 2024-06-03T01:21:22.179 INFO:tasks.workunit.client.0.smithi032.stdout:6/982: creat d8/d50/f13c x:0 0 0 2024-06-03T01:21:22.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/983: creat d8/d11/d20/d2b/d39/f13d x:0 0 0 2024-06-03T01:21:22.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/984: dread - d8/d11/d20/d21/d9a/d102/f11c zero size 2024-06-03T01:21:22.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/985: creat d8/d11/d20/d31/df2/df8/d107/f13e x:0 0 0 2024-06-03T01:21:22.197 INFO:tasks.workunit.client.0.smithi032.stdout:6/986: rmdir d8/d11/d20 39 2024-06-03T01:21:22.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/987: mknod d8/d11/d20/d31/df2/df8/d107/c13f 0 2024-06-03T01:21:22.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/988: write d8/ff [7726726,61991] 0 2024-06-03T01:21:22.222 INFO:tasks.workunit.client.0.smithi032.stdout:6/989: dread d8/d11/d20/d2b/fb3 [0,4194304] 0 2024-06-03T01:21:22.224 INFO:tasks.workunit.client.0.smithi032.stdout:6/990: creat d8/d11/d20/d2b/d39/f140 x:0 0 0 2024-06-03T01:21:22.225 INFO:tasks.workunit.client.0.smithi032.stdout:6/991: mkdir d8/d11/d20/d2b/d141 0 2024-06-03T01:21:22.226 INFO:tasks.workunit.client.0.smithi032.stdout:6/992: write d8/d50/dda/fe1 [8653483,86451] 0 2024-06-03T01:21:22.227 INFO:tasks.workunit.client.0.smithi032.stdout:6/993: mknod d8/d11/d20/d2b/d39/d19/c142 0 2024-06-03T01:21:22.248 INFO:tasks.workunit.client.0.smithi032.stdout:6/994: dwrite d8/d11/d20/d21/d9a/fba [0,4194304] 0 2024-06-03T01:21:22.248 INFO:tasks.workunit.client.0.smithi032.stdout:6/995: fsync d8/d11/d20/d31/d3f/f69 0 2024-06-03T01:21:22.248 INFO:tasks.workunit.client.0.smithi032.stdout:6/996: write d8/d11/d20/d31/df2/df8/f11b [362308,71022] 0 2024-06-03T01:21:22.250 INFO:tasks.workunit.client.0.smithi032.stdout:6/997: write d8/d11/d20/d31/d3f/f69 [234359,86887] 0 2024-06-03T01:21:22.250 INFO:tasks.workunit.client.0.smithi032.stdout:6/998: chown d8/d11/d20/d21/d9a/fdc 384008 1 2024-06-03T01:21:22.259 INFO:tasks.workunit.client.0.smithi032.stdout:6/999: dread d8/d11/d20/d3b/f6d [0,4194304] 0 2024-06-03T01:21:22.267 INFO:tasks.workunit.client.0.smithi032.stderr:+ rm -rf -- ./tmp.TgdXWrVXgM 2024-06-03T01:21:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:22 smithi032 ceph-mon[29360]: pgmap v16: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 119 MiB/s wr, 269 op/s 2024-06-03T01:21:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:22 smithi044 ceph-mon[36956]: pgmap v16: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 119 MiB/s wr, 269 op/s 2024-06-03T01:21:24.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:24 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:24.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:24 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:24 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:24 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: pgmap v17: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 139 MiB/s rd, 158 MiB/s wr, 398 op/s 2024-06-03T01:21:25.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:25.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:25 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:21:25.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: pgmap v17: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 139 MiB/s rd, 158 MiB/s wr, 398 op/s 2024-06-03T01:21:25.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:25.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:25 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: Adjusting osd_memory_target on smithi032 to 1217M 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:26 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: Adjusting osd_memory_target on smithi032 to 1217M 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:26.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:26 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: pgmap v18: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 102 MiB/s rd, 114 MiB/s wr, 270 op/s 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.756 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.757 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.757 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:27 smithi032 ceph-mon[29360]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:21:27.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:21:27.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: pgmap v18: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 102 MiB/s rd, 114 MiB/s wr, 270 op/s 2024-06-03T01:21:27.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:21:27.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:21:27.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:21:27.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:27.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:27 smithi044 ceph-mon[36956]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:21:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:28 smithi032 ceph-mon[29360]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:21:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:28 smithi044 ceph-mon[36956]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:21:29.765 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:29 smithi032 ceph-mon[29360]: pgmap v19: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 103 MiB/s rd, 116 MiB/s wr, 436 op/s 2024-06-03T01:21:29.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:29 smithi044 ceph-mon[36956]: pgmap v19: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 103 MiB/s rd, 116 MiB/s wr, 436 op/s 2024-06-03T01:21:31.019 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:30 smithi032 ceph-mon[29360]: pgmap v20: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 293 op/s 2024-06-03T01:21:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:30 smithi044 ceph-mon[36956]: pgmap v20: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 293 op/s 2024-06-03T01:21:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:32 smithi044 ceph-mon[36956]: pgmap v21: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 293 op/s 2024-06-03T01:21:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:32 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:32 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:32 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:21:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:32 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:32 smithi032 ceph-mon[29360]: pgmap v21: 65 pgs: 65 active+clean; 1.7 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 293 op/s 2024-06-03T01:21:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:32 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:32 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:32 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:21:33.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:32 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:33.722 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-03T01:21:33.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T01:21:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:33 smithi032 ceph-mon[29360]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:21:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:33 smithi032 ceph-mon[29360]: Upgrade: Need to upgrade myself (mgr.smithi044.wszjlw) 2024-06-03T01:21:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:33 smithi032 ceph-mon[29360]: Upgrade: Need to upgrade myself (mgr.smithi044.wszjlw) 2024-06-03T01:21:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:33 smithi044 ceph-mon[36956]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:21:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:33 smithi044 ceph-mon[36956]: Upgrade: Need to upgrade myself (mgr.smithi044.wszjlw) 2024-06-03T01:21:34.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:33 smithi044 ceph-mon[36956]: Upgrade: Need to upgrade myself (mgr.smithi044.wszjlw) 2024-06-03T01:21:34.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.924+0000 7f659b57c700 1 -- 172.21.15.32:0/2921266625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659410a6b0 msgr2=0x7f659410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:34.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.924+0000 7f659b57c700 1 --2- 172.21.15.32:0/2921266625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659410a6b0 0x7f659410ab10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6584009870 tx=0x7f6584009b80 comp rx=0 tx=0).stop 2024-06-03T01:21:34.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 -- 172.21.15.32:0/2921266625 shutdown_connections 2024-06-03T01:21:34.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 --2- 172.21.15.32:0/2921266625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659410a6b0 0x7f659410ab10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:34.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 --2- 172.21.15.32:0/2921266625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6594109500 0x7f6594109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 -- 172.21.15.32:0/2921266625 >> 172.21.15.32:0/2921266625 conn(0x7f659406fc60 msgr2=0x7f65940720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 -- 172.21.15.32:0/2921266625 shutdown_connections 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.925+0000 7f659b57c700 1 -- 172.21.15.32:0/2921266625 wait complete. 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 Processor -- start 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 -- start start 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 0x7f659406a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941a28d0 con 0x7f6594109500 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f659b57c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941a2a10 con 0x7f659410a6b0 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f6599318700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f6599318700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36200/0 (socket says 172.21.15.32:36200) 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f6599318700 1 -- 172.21.15.32:0/4009546053 learned_addr learned my addr 172.21.15.32:0/4009546053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:34.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.926+0000 7f6598b17700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 0x7f659406a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f6598b17700 1 -- 172.21.15.32:0/4009546053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 msgr2=0x7f659406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f6598b17700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f6598b17700 1 -- 172.21.15.32:0/4009546053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900095f0 con 0x7f659410a6b0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f6599318700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f6598b17700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 0x7f659406a100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6584009230 tx=0x7f658400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658401c070 con 0x7f659410a6b0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f658400b4e0 con 0x7f659410a6b0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6584022d90 con 0x7f659410a6b0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.927+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6584009520 con 0x7f659410a6b0 2024-06-03T01:21:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.928+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659406a920 con 0x7f659410a6b0 2024-06-03T01:21:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.929+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594112010 con 0x7f659410a6b0 2024-06-03T01:21:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.930+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91286+0+0 (secure 0 0 0) 0x7f6584020070 con 0x7f659410a6b0 2024-06-03T01:21:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.930+0000 7f658a7fc700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 0x7f658006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.930+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f658408dd80 con 0x7f659410a6b0 2024-06-03T01:21:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.930+0000 7f6599318700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 0x7f658006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.931+0000 7f6599318700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 0x7f658006f7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6590005950 tx=0x7f6590009310 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:34.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:34.935+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f658405b190 con 0x7f659410a6b0 2024-06-03T01:21:35.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: pgmap v22: 65 pgs: 65 active+clean; 864 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 42 MiB/s wr, 477 op/s 2024-06-03T01:21:35.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: Upgrade: Updating mgr.smithi032.kfcfaa 2024-06-03T01:21:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:21:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:21:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:21:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:34 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:35.251 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: pgmap v22: 65 pgs: 65 active+clean; 864 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 42 MiB/s wr, 477 op/s 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: Upgrade: Updating mgr.smithi032.kfcfaa 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:21:35.252 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:34 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:35.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.332+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f659410f000 con 0x7f658006d2e0 2024-06-03T01:21:35.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.334+0000 7f658a7fc700 1 -- 172.21.15.32:0/4009546053 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f659410f000 con 0x7f658006d2e0 2024-06-03T01:21:35.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 msgr2=0x7f658006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 0x7f658006f7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6590005950 tx=0x7f6590009310 comp rx=0 tx=0).stop 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 msgr2=0x7f659406a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 0x7f659406a100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6584009230 tx=0x7f658400f690 comp rx=0 tx=0).stop 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 shutdown_connections 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.338+0000 7f659b57c700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f658006d2e0 0x7f658006f7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.339+0000 7f659b57c700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6594109500 0x7f659406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.339+0000 7f659b57c700 1 --2- 172.21.15.32:0/4009546053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659410a6b0 0x7f659406a100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.339+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 >> 172.21.15.32:0/4009546053 conn(0x7f659406fc60 msgr2=0x7f659410d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:35.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.339+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 shutdown_connections 2024-06-03T01:21:35.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.339+0000 7f659b57c700 1 -- 172.21.15.32:0/4009546053 wait complete. 2024-06-03T01:21:35.353 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 -- 172.21.15.32:0/1995098674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 msgr2=0x7f217c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 --2- 172.21.15.32:0/1995098674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c0fffc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f216c009910 tx=0x7f216c009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 -- 172.21.15.32:0/1995098674 shutdown_connections 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 --2- 172.21.15.32:0/1995098674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 0x7f217c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 --2- 172.21.15.32:0/1995098674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c0fffc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 -- 172.21.15.32:0/1995098674 >> 172.21.15.32:0/1995098674 conn(0x7f217c0fb120 msgr2=0x7f217c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 -- 172.21.15.32:0/1995098674 shutdown_connections 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.479+0000 7f218251a700 1 -- 172.21.15.32:0/1995098674 wait complete. 2024-06-03T01:21:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.480+0000 7f218251a700 1 Processor -- start 2024-06-03T01:21:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.480+0000 7f218251a700 1 -- start start 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.480+0000 7f218251a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.480+0000 7f218251a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 0x7f217c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.480+0000 7f218251a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c197030 con 0x7f217c0ffba0 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f218251a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c197170 con 0x7f217c100da0 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36224/0 (socket says 172.21.15.32:36224) 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 -- 172.21.15.32:0/3865855082 learned_addr learned my addr 172.21.15.32:0/3865855082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217b7fe700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 0x7f217c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 -- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 msgr2=0x7f217c196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 0x7f217c196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 -- 172.21.15.32:0/3865855082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0095f0 con 0x7f217c0ffba0 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f217bfff700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f216c00bda0 tx=0x7f216c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216c01d070 con 0x7f217c0ffba0 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c19bbc0 con 0x7f217c0ffba0 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f216c022470 con 0x7f217c0ffba0 2024-06-03T01:21:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.481+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216c00f650 con 0x7f217c0ffba0 2024-06-03T01:21:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.482+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c19c0b0 con 0x7f217c0ffba0 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.483+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c108700 con 0x7f217c0ffba0 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.483+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) v1 ==== 91286+0+0 (secure 0 0 0) 0x7f216c004af0 con 0x7f217c0ffba0 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.484+0000 7f21797fa700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 0x7f216406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.484+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f216c08e270 con 0x7f217c0ffba0 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.484+0000 7f217b7fe700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 0x7f216406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.485+0000 7f217b7fe700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 0x7f216406f7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2170005950 tx=0x7f217000a300 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.489+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f216c05b590 con 0x7f217c0ffba0 2024-06-03T01:21:35.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.786+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f217c069520 con 0x7f216406d2f0 2024-06-03T01:21:35.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.788+0000 7f21797fa700 1 -- 172.21.15.32:0/3865855082 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f217c069520 con 0x7f216406d2f0 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 msgr2=0x7f216406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 0x7f216406f7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2170005950 tx=0x7f217000a300 comp rx=0 tx=0).stop 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 msgr2=0x7f217c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f216c00bda0 tx=0x7f216c00bdd0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 shutdown_connections 2024-06-03T01:21:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f216406d2f0 0x7f216406f7b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f217c0ffba0 0x7f217c1964d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 --2- 172.21.15.32:0/3865855082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c100da0 0x7f217c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 >> 172.21.15.32:0/3865855082 conn(0x7f217c0fb120 msgr2=0x7f217c068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 shutdown_connections 2024-06-03T01:21:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.790+0000 7f218251a700 1 -- 172.21.15.32:0/3865855082 wait complete. 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.910+0000 7f3da9473700 1 -- 172.21.15.32:0/2486466042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4101800 msgr2=0x7f3da4103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.910+0000 7f3da9473700 1 --2- 172.21.15.32:0/2486466042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4101800 0x7f3da4103bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009910 tx=0x7f3d94009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 -- 172.21.15.32:0/2486466042 shutdown_connections 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 --2- 172.21.15.32:0/2486466042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4104130 0x7f3da4106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 --2- 172.21.15.32:0/2486466042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4101800 0x7f3da4103bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 -- 172.21.15.32:0/2486466042 >> 172.21.15.32:0/2486466042 conn(0x7f3da40fb190 msgr2=0x7f3da40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 -- 172.21.15.32:0/2486466042 shutdown_connections 2024-06-03T01:21:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 -- 172.21.15.32:0/2486466042 wait complete. 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 Processor -- start 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.911+0000 7f3da9473700 1 -- start start 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da9473700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da9473700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da9473700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4196f70 con 0x7f3da4104130 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da9473700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da41970b0 con 0x7f3da4101800 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36236/0 (socket says 172.21.15.32:36236) 2024-06-03T01:21:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da27fc700 1 -- 172.21.15.32:0/507591689 learned_addr learned my addr 172.21.15.32:0/507591689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:37060/0 (socket says 172.21.15.32:37060) 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 -- 172.21.15.32:0/507591689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 msgr2=0x7f3da4196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 -- 172.21.15.32:0/507591689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940095f0 con 0x7f3da4101800 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da27fc700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.912+0000 7f3da2ffd700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009910 tx=0x7f3d9400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.913+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d9401d070 con 0x7f3da4101800 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.913+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da419bb00 con 0x7f3da4101800 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.913+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d9400f460 con 0x7f3da4101800 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.913+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d94021620 con 0x7f3da4101800 2024-06-03T01:21:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.913+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da419bff0 con 0x7f3da4101800 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.914+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91286+0+0 (secure 0 0 0) 0x7f3d9400fa40 con 0x7f3da4101800 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.914+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da4190610 con 0x7f3da4101800 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.915+0000 7f3d8bfff700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 0x7f3d8c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.915+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3d9408de90 con 0x7f3da4101800 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.915+0000 7f3da27fc700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 0x7f3d8c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.916+0000 7f3da27fc700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 0x7f3d8c06f7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005950 tx=0x7f3d9800b500 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:35.920+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f3d9405b060 con 0x7f3da4101800 2024-06-03T01:21:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:35 smithi032 ceph-mon[29360]: Deploying daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:21:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:35 smithi032 ceph-mon[29360]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:36.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.199+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3da40611d0 con 0x7f3d8c06d2f0 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.210+0000 7f3d8bfff700 1 -- 172.21.15.32:0/507591689 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3da40611d0 con 0x7f3d8c06d2f0 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 12s ago 8m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (9m) 12s ago 9m 7721k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (7m) 28s ago 7m 7554k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:21:36.210 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (9m) 12s ago 9m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (7m) 28s ago 7m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 12s ago 8m 87.6M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (3m) 12s ago 3m 143M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (3m) 12s ago 3m 133M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (3m) 28s ago 3m 13.2M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (3m) 28s ago 3m 15.5M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:9283,8765,8443 running (10m) 12s ago 10m 323M - 18.2.1 5be31c24972a 5db22291b59c 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (60s) 28s ago 7m 495M - 19.0.0-2946-g3d89accf b045ac545f82 57506042450c 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (10m) 12s ago 10m 48.5M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (7m) 28s ago 7m 40.8M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 12s ago 8m 14.1M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 28s ago 7m 16.3M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) 12s ago 6m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:21:36.211 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (6m) 12s ago 6m 255M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:21:36.212 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (6m) 12s ago 6m 192M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:21:36.212 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (5m) 28s ago 5m 279M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:21:36.212 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (5m) 28s ago 5m 248M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:21:36.212 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (4m) 28s ago 4m 299M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:21:36.212 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 starting - - - - 2024-06-03T01:21:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 msgr2=0x7f3d8c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 0x7f3d8c06f7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005950 tx=0x7f3d9800b500 comp rx=0 tx=0).stop 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 msgr2=0x7f3da4196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009910 tx=0x7f3d9400bd70 comp rx=0 tx=0).stop 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 shutdown_connections 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f3d8c06d2f0 0x7f3d8c06f7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4101800 0x7f3da4196410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 --2- 172.21.15.32:0/507591689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da4104130 0x7f3da4196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.213+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 >> 172.21.15.32:0/507591689 conn(0x7f3da40fb190 msgr2=0x7f3da40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.214+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 shutdown_connections 2024-06-03T01:21:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.214+0000 7f3da9473700 1 -- 172.21.15.32:0/507591689 wait complete. 2024-06-03T01:21:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:35 smithi044 ceph-mon[36956]: Deploying daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:21:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:35 smithi044 ceph-mon[36956]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.395+0000 7f60fd01d700 1 -- 172.21.15.32:0/1459839334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80ffcd0 msgr2=0x7f60f8100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.395+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1459839334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80ffcd0 0x7f60f8100150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009910 tx=0x7f60ec009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.396+0000 7f60fd01d700 1 -- 172.21.15.32:0/1459839334 shutdown_connections 2024-06-03T01:21:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.396+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1459839334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80ffcd0 0x7f60f8100150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.396+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1459839334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80fef60 0x7f60f80ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.396+0000 7f60fd01d700 1 -- 172.21.15.32:0/1459839334 >> 172.21.15.32:0/1459839334 conn(0x7f60f80fa8c0 msgr2=0x7f60f80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.397+0000 7f60fd01d700 1 -- 172.21.15.32:0/1459839334 shutdown_connections 2024-06-03T01:21:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.397+0000 7f60fd01d700 1 -- 172.21.15.32:0/1459839334 wait complete. 2024-06-03T01:21:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.398+0000 7f60fd01d700 1 Processor -- start 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60fd01d700 1 -- start start 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60fd01d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60fd01d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80ffcd0 0x7f60f819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60fd01d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f819d5c0 con 0x7f60f80ffcd0 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60fd01d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f819d700 con 0x7f60f80fef60 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80ffcd0 0x7f60f819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:37076/0 (socket says 172.21.15.32:37076) 2024-06-03T01:21:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.399+0000 7f60f6d9d700 1 -- 172.21.15.32:0/1067693515 learned_addr learned my addr 172.21.15.32:0/1067693515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60f6d9d700 1 -- 172.21.15.32:0/1067693515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80ffcd0 msgr2=0x7f60f819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60f6d9d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80ffcd0 0x7f60f819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60f6d9d700 1 -- 172.21.15.32:0/1067693515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0095f0 con 0x7f60f80fef60 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60f6d9d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d8d0 tx=0x7f60e800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e80098a0 con 0x7f60f80fef60 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60e8010460 con 0x7f60f80fef60 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.400+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e800f5f0 con 0x7f60f80fef60 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.401+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f81a21b0 con 0x7f60f80fef60 2024-06-03T01:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.401+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f81a2640 con 0x7f60f80fef60 2024-06-03T01:21:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.403+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f810fe10 con 0x7f60f80fef60 2024-06-03T01:21:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.403+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91286+0+0 (secure 0 0 0) 0x7f60e8009a00 con 0x7f60f80fef60 2024-06-03T01:21:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.404+0000 7f60dffff700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 0x7f60e006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.404+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f60e808ca70 con 0x7f60f80fef60 2024-06-03T01:21:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.405+0000 7f60f659c700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 0x7f60e006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.405+0000 7f60f659c700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 0x7f60e006f830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b5c0 tx=0x7f60ec005fb0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.411+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f60e8059d40 con 0x7f60f80fef60 2024-06-03T01:21:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.937+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60f81a2d30 con 0x7f60f80fef60 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:21:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:21:36.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.938+0000 7f60dffff700 1 -- 172.21.15.32:0/1067693515 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+764 (secure 0 0 0) 0x7f60e8059d40 con 0x7f60f80fef60 2024-06-03T01:21:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 msgr2=0x7f60e006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 0x7f60e006f830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b5c0 tx=0x7f60ec005fb0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 msgr2=0x7f60f819ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d8d0 tx=0x7f60e800dbe0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 shutdown_connections 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f60e006d370 0x7f60e006f830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60f80fef60 0x7f60f819ca60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 --2- 172.21.15.32:0/1067693515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60f80ffcd0 0x7f60f819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 >> 172.21.15.32:0/1067693515 conn(0x7f60f80fa8c0 msgr2=0x7f60f8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 shutdown_connections 2024-06-03T01:21:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:36.942+0000 7f60fd01d700 1 -- 172.21.15.32:0/1067693515 wait complete. 2024-06-03T01:21:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:36 smithi032 ceph-mon[29360]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:36 smithi032 ceph-mon[29360]: pgmap v23: 65 pgs: 65 active+clean; 864 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-06-03T01:21:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:36 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:21:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:36 smithi032 ceph-mon[29360]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.120+0000 7fe06df2d700 1 -- 172.21.15.32:0/74573567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068104050 msgr2=0x7fe0681044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:37.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.120+0000 7fe06df2d700 1 --2- 172.21.15.32:0/74573567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068104050 0x7fe0681044d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009910 tx=0x7fe05c009c20 comp rx=0 tx=0).stop 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.120+0000 7fe06df2d700 1 -- 172.21.15.32:0/74573567 shutdown_connections 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.120+0000 7fe06df2d700 1 --2- 172.21.15.32:0/74573567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068104050 0x7fe0681044d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.121+0000 7fe06df2d700 1 --2- 172.21.15.32:0/74573567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068102e60 0x7fe068103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.121+0000 7fe06df2d700 1 -- 172.21.15.32:0/74573567 >> 172.21.15.32:0/74573567 conn(0x7fe0680fe470 msgr2=0x7fe068100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.122+0000 7fe06df2d700 1 -- 172.21.15.32:0/74573567 shutdown_connections 2024-06-03T01:21:37.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.122+0000 7fe06df2d700 1 -- 172.21.15.32:0/74573567 wait complete. 2024-06-03T01:21:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 Processor -- start 2024-06-03T01:21:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 -- start start 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068102e60 0x7fe06810f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068110190 con 0x7fe068102e60 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.123+0000 7fe06df2d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681102d0 con 0x7fe068104050 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:47748/0 (socket says 172.21.15.32:47748) 2024-06-03T01:21:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 -- 172.21.15.32:0/748585373 learned_addr learned my addr 172.21.15.32:0/748585373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:21:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 -- 172.21.15.32:0/748585373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068102e60 msgr2=0x7fe06810f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068102e60 0x7fe06810f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.124+0000 7fe066ffd700 1 -- 172.21.15.32:0/748585373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0095f0 con 0x7fe068104050 2024-06-03T01:21:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.125+0000 7fe066ffd700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.126+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe05c01d070 con 0x7fe068104050 2024-06-03T01:21:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.126+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe05c022470 con 0x7fe068104050 2024-06-03T01:21:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.126+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe05c00f670 con 0x7fe068104050 2024-06-03T01:21:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.126+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe068110d00 con 0x7fe068104050 2024-06-03T01:21:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.126+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0681111f0 con 0x7fe068104050 2024-06-03T01:21:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.128+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91286+0+0 (secure 0 0 0) 0x7fe05c0225e0 con 0x7fe068104050 2024-06-03T01:21:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.128+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe068076e40 con 0x7fe068104050 2024-06-03T01:21:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.129+0000 7fe064ff9700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 0x7fe05006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.129+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe05c08e260 con 0x7fe068104050 2024-06-03T01:21:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.130+0000 7fe0677fe700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 0x7fe05006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:37.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.131+0000 7fe0677fe700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 0x7fe05006f900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe068103bc0 tx=0x7fe058006d20 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.137+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe05c05b5e0 con 0x7fe068104050 2024-06-03T01:21:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:36 smithi044 ceph-mon[36956]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:36 smithi044 ceph-mon[36956]: pgmap v23: 65 pgs: 65 active+clean; 864 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-06-03T01:21:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:36 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:21:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:36 smithi044 ceph-mon[36956]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.568+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0681114d0 con 0x7fe05006d440 2024-06-03T01:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.570+0000 7fe064ff9700 1 -- 172.21.15.32:0/748585373 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe0681114d0 con 0x7fe05006d440 2024-06-03T01:21:37.569 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "1/2 daemons upgraded", 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mgr daemons", 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:21:37.570 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:21:37.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 msgr2=0x7fe05006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:37.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 0x7fe05006f900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe068103bc0 tx=0x7fe058006d20 comp rx=0 tx=0).stop 2024-06-03T01:21:37.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 msgr2=0x7fe06810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c00be20 comp rx=0 tx=0).stop 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 shutdown_connections 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fe05006d440 0x7fe05006f900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe068102e60 0x7fe06810f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 --2- 172.21.15.32:0/748585373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe068104050 0x7fe06810fb70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.572+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 >> 172.21.15.32:0/748585373 conn(0x7fe0680fe470 msgr2=0x7fe0680ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.573+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 shutdown_connections 2024-06-03T01:21:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:21:37.573+0000 7fe06df2d700 1 -- 172.21.15.32:0/748585373 wait complete. 2024-06-03T01:21:38.119 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:37 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1067693515' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:21:38.119 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:37 smithi044 ceph-mon[36956]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:38.144 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:37 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1067693515' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:21:38.144 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:37 smithi032 ceph-mon[29360]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:21:39.160 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:38 smithi032 ceph-mon[29360]: pgmap v24: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2024-06-03T01:21:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:38 smithi044 ceph-mon[36956]: pgmap v24: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2024-06-03T01:21:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:40 smithi032 ceph-mon[29360]: pgmap v25: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-06-03T01:21:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:40 smithi044 ceph-mon[36956]: pgmap v25: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-06-03T01:21:41.806 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:41 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:41 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:42 smithi032 ceph-mon[29360]: pgmap v26: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-06-03T01:21:43.163 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:42 smithi044 ceph-mon[36956]: pgmap v26: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-06-03T01:21:45.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:44 smithi044 ceph-mon[36956]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s rd, 2.6 MiB/s wr, 528 op/s 2024-06-03T01:21:45.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:44 smithi032 ceph-mon[29360]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s rd, 2.6 MiB/s wr, 528 op/s 2024-06-03T01:21:46.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:46 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:46.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:46 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:46.940 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:46 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:46.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:46 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:46.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:46 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:21:46.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:46 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:47.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:47 smithi044 ceph-mon[36956]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 344 op/s 2024-06-03T01:21:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:47 smithi032 ceph-mon[29360]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 344 op/s 2024-06-03T01:21:49.694 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:49 smithi032 ceph-mon[29360]: pgmap v29: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.8 MiB/s wr, 484 op/s 2024-06-03T01:21:49.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:49 smithi044 ceph-mon[36956]: pgmap v29: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.8 MiB/s wr, 484 op/s 2024-06-03T01:21:50.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:50 smithi032 ceph-mon[29360]: pgmap v30: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-06-03T01:21:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:50 smithi044 ceph-mon[36956]: pgmap v30: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-06-03T01:21:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: pgmap v31: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-06-03T01:21:53.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: Standby manager daemon smithi032.kfcfaa restarted 2024-06-03T01:21:53.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: Standby manager daemon smithi032.kfcfaa started 2024-06-03T01:21:53.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/crt"}]: dispatch 2024-06-03T01:21:53.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:21:53.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/key"}]: dispatch 2024-06-03T01:21:53.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:52 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: pgmap v31: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.1 MiB/s wr, 356 op/s 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: Standby manager daemon smithi032.kfcfaa restarted 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: Standby manager daemon smithi032.kfcfaa started 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/crt"}]: dispatch 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.kfcfaa/key"}]: dispatch 2024-06-03T01:21:53.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:52 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:21:54.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:53 smithi044 ceph-mon[36956]: mgrmap e25: smithi044.wszjlw(active, since 57s), standbys: smithi032.kfcfaa 2024-06-03T01:21:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:53 smithi032 ceph-mon[29360]: mgrmap e25: smithi044.wszjlw(active, since 57s), standbys: smithi032.kfcfaa 2024-06-03T01:21:55.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:54 smithi044 ceph-mon[36956]: pgmap v32: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-06-03T01:21:55.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:54 smithi032 ceph-mon[29360]: pgmap v32: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-06-03T01:21:57.149 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:56 smithi032 ceph-mon[29360]: pgmap v33: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-06-03T01:21:57.149 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:56 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:57.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:56 smithi044 ceph-mon[36956]: pgmap v33: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-06-03T01:21:57.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:56 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:21:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:21:58 smithi044 ceph-mon[36956]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.9 MiB/s wr, 359 op/s 2024-06-03T01:21:59.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:21:58 smithi032 ceph-mon[29360]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.9 MiB/s wr, 359 op/s 2024-06-03T01:22:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:00 smithi032 ceph-mon[29360]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-06-03T01:22:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:00 smithi044 ceph-mon[36956]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-06-03T01:22:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:02 smithi044 ceph-mon[36956]: pgmap v36: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-06-03T01:22:03.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:02 smithi032 ceph-mon[29360]: pgmap v36: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-06-03T01:22:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:04 smithi044 ceph-mon[36956]: pgmap v37: 65 pgs: 65 active+clean; 328 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.3 MiB/s wr, 318 op/s 2024-06-03T01:22:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:04 smithi032 ceph-mon[29360]: pgmap v37: 65 pgs: 65 active+clean; 328 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.3 MiB/s wr, 318 op/s 2024-06-03T01:22:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:06 smithi044 ceph-mon[36956]: pgmap v38: 65 pgs: 65 active+clean; 328 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 234 op/s 2024-06-03T01:22:07.276 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:06 smithi032 ceph-mon[29360]: pgmap v38: 65 pgs: 65 active+clean; 328 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 234 op/s 2024-06-03T01:22:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.747+0000 7f0d6dec8700 1 -- 172.21.15.32:0/267355343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 msgr2=0x7f0d68101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.747+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/267355343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68101960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009f90 tx=0x7f0d6000b330 comp rx=0 tx=0).stop 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 -- 172.21.15.32:0/267355343 shutdown_connections 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/267355343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/267355343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68101960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 -- 172.21.15.32:0/267355343 >> 172.21.15.32:0/267355343 conn(0x7f0d680fb150 msgr2=0x7f0d680fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 -- 172.21.15.32:0/267355343 shutdown_connections 2024-06-03T01:22:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.749+0000 7f0d6dec8700 1 -- 172.21.15.32:0/267355343 wait complete. 2024-06-03T01:22:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.750+0000 7f0d6dec8700 1 Processor -- start 2024-06-03T01:22:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.750+0000 7f0d6dec8700 1 -- start start 2024-06-03T01:22:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d6dec8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d6dec8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d6dec8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68072920 con 0x7f0d680ff530 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d6dec8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d681aa800 con 0x7f0d68101ea0 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d66ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d66ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:46254/0 (socket says 172.21.15.32:46254) 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d66ffd700 1 -- 172.21.15.32:0/1094366504 learned_addr learned my addr 172.21.15.32:0/1094366504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.751+0000 7f0d677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.752+0000 7f0d677fe700 1 -- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 msgr2=0x7f0d68072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.752+0000 7f0d677fe700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.752+0000 7f0d677fe700 1 -- 172.21.15.32:0/1094366504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009c40 con 0x7f0d680ff530 2024-06-03T01:22:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.752+0000 7f0d677fe700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68071dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009f60 tx=0x7f0d6000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.754+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d6000e070 con 0x7f0d680ff530 2024-06-03T01:22:07.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.754+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d681aa9a0 con 0x7f0d680ff530 2024-06-03T01:22:07.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.754+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0d60003d80 con 0x7f0d680ff530 2024-06-03T01:22:07.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.754+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d681aaea0 con 0x7f0d680ff530 2024-06-03T01:22:07.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.754+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d60012660 con 0x7f0d680ff530 2024-06-03T01:22:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.756+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d54005320 con 0x7f0d680ff530 2024-06-03T01:22:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.757+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f0d600127c0 con 0x7f0d680ff530 2024-06-03T01:22:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.757+0000 7f0d64ff9700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 0x7f0d50070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.757+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0d6008f020 con 0x7f0d680ff530 2024-06-03T01:22:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.758+0000 7f0d66ffd700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 0x7f0d50070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.758+0000 7f0d66ffd700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 0x7f0d50070580 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00bd90 tx=0x7f0d5c00b4e0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:07.766+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f0d600c3590 con 0x7f0d680ff530 2024-06-03T01:22:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:07 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:07 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.169+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d54000bf0 con 0x7f0d5006e0c0 2024-06-03T01:22:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.171+0000 7f0d64ff9700 1 -- 172.21.15.32:0/1094366504 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0d54000bf0 con 0x7f0d5006e0c0 2024-06-03T01:22:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.175+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 msgr2=0x7f0d50070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.175+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 0x7f0d50070580 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00bd90 tx=0x7f0d5c00b4e0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.175+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 msgr2=0x7f0d68071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.175+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68071dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009f60 tx=0x7f0d6000bef0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 shutdown_connections 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0d5006e0c0 0x7f0d50070580 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d680ff530 0x7f0d68071dc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 --2- 172.21.15.32:0/1094366504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d68101ea0 0x7f0d68072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 >> 172.21.15.32:0/1094366504 conn(0x7f0d680fb150 msgr2=0x7f0d680fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 shutdown_connections 2024-06-03T01:22:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.176+0000 7f0d6dec8700 1 -- 172.21.15.32:0/1094366504 wait complete. 2024-06-03T01:22:08.195 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:22:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:07 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:07 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:08.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 -- 172.21.15.32:0/9809674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff298107f60 msgr2=0x7ff29810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/9809674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff298107f60 0x7ff29810a350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00fa70 tx=0x7ff28c00fd80 comp rx=0 tx=0).stop 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 -- 172.21.15.32:0/9809674 shutdown_connections 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/9809674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29810a890 0x7ff29810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/9809674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff298107f60 0x7ff29810a350 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 -- 172.21.15.32:0/9809674 >> 172.21.15.32:0/9809674 conn(0x7ff29806deb0 msgr2=0x7ff2980702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.351+0000 7ff29cbd0700 1 -- 172.21.15.32:0/9809674 shutdown_connections 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.352+0000 7ff29cbd0700 1 -- 172.21.15.32:0/9809674 wait complete. 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.352+0000 7ff29cbd0700 1 Processor -- start 2024-06-03T01:22:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.353+0000 7ff29cbd0700 1 -- start start 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.353+0000 7ff29cbd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff298107f60 0x7ff2981a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.353+0000 7ff29cbd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.353+0000 7ff29cbd0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2981a5de0 con 0x7ff29810a890 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.353+0000 7ff29cbd0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2981a5f20 con 0x7ff298107f60 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37980/0 (socket says 172.21.15.32:37980) 2024-06-03T01:22:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 -- 172.21.15.32:0/2898601074 learned_addr learned my addr 172.21.15.32:0/2898601074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 -- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff298107f60 msgr2=0x7ff2981a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff29659c700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff298107f60 0x7ff2981a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff298107f60 0x7ff2981a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.354+0000 7ff295d9b700 1 -- 172.21.15.32:0/2898601074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c00f750 con 0x7ff29810a890 2024-06-03T01:22:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.355+0000 7ff295d9b700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff288007a00 tx=0x7ff288007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.355+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff288010070 con 0x7ff29810a890 2024-06-03T01:22:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.356+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff28800a8d0 con 0x7ff29810a890 2024-06-03T01:22:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.356+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff288003d30 con 0x7ff29810a890 2024-06-03T01:22:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.356+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2981aa9d0 con 0x7ff29810a890 2024-06-03T01:22:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.356+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2981aaf20 con 0x7ff29810a890 2024-06-03T01:22:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.357+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278005320 con 0x7ff29810a890 2024-06-03T01:22:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.360+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 92158+0+0 (secure 0 0 0) 0x7ff2880041f0 con 0x7ff29810a890 2024-06-03T01:22:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.360+0000 7ff2877fe700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 0x7ff280070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.360+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff28808dd40 con 0x7ff29810a890 2024-06-03T01:22:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.361+0000 7ff29659c700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 0x7ff280070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.361+0000 7ff29659c700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 0x7ff280070580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff28c004ff0 tx=0x7ff28c011040 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.367+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff28805b010 con 0x7ff29810a890 2024-06-03T01:22:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.836+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff278000bf0 con 0x7ff28006e0c0 2024-06-03T01:22:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.838+0000 7ff2877fe700 1 -- 172.21.15.32:0/2898601074 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff278000bf0 con 0x7ff28006e0c0 2024-06-03T01:22:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.844+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 msgr2=0x7ff280070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.844+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 0x7ff280070580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff28c004ff0 tx=0x7ff28c011040 comp rx=0 tx=0).stop 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.844+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 msgr2=0x7ff2981a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.844+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff288007a00 tx=0x7ff288007dc0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 shutdown_connections 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7ff28006e0c0 0x7ff280070580 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff298107f60 0x7ff2981a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 --2- 172.21.15.32:0/2898601074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29810a890 0x7ff2981a57c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 >> 172.21.15.32:0/2898601074 conn(0x7ff29806deb0 msgr2=0x7ff2980702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.845+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 shutdown_connections 2024-06-03T01:22:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:08.846+0000 7ff29cbd0700 1 -- 172.21.15.32:0/2898601074 wait complete. 2024-06-03T01:22:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/4148610650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 msgr2=0x7fa8d4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/4148610650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4101220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009910 tx=0x7fa8c4009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/4148610650 shutdown_connections 2024-06-03T01:22:09.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/4148610650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4101220 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/4148610650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.037+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/4148610650 >> 172.21.15.32:0/4148610650 conn(0x7fa8d40fb160 msgr2=0x7fa8d40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.038+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/4148610650 shutdown_connections 2024-06-03T01:22:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.038+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/4148610650 wait complete. 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 Processor -- start 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 -- start start 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4196220 con 0x7fa8d4100dc0 2024-06-03T01:22:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.039+0000 7fa8dbcaf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4196360 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:46292/0 (socket says 172.21.15.32:46292) 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 -- 172.21.15.32:0/3839126012 learned_addr learned my addr 172.21.15.32:0/3839126012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d924a700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 -- 172.21.15.32:0/3839126012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 msgr2=0x7fa8d4195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.040+0000 7fa8d9a4b700 1 -- 172.21.15.32:0/3839126012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40095f0 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8d9a4b700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000d900 tx=0x7fa8d000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8d000b920 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0010460 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d419ae10 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8d000f590 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.041+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d419b280 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.043+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d418f530 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.043+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 92158+0+0 (secure 0 0 0) 0x7fa8d000ba80 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.044+0000 7fa8caffd700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 0x7fa8c0070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.044+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa8d008d940 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.044+0000 7fa8d924a700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 0x7fa8c0070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.045+0000 7fa8d924a700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 0x7fa8c0070480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b5c0 tx=0x7fa8c4005ee0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.050+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fa8d005abb0 con 0x7fa8d40ffbc0 2024-06-03T01:22:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.436+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8d40611d0 con 0x7fa8c006dfc0 2024-06-03T01:22:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.448+0000 7fa8caffd700 1 -- 172.21.15.32:0/3839126012 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa8d40611d0 con 0x7fa8c006dfc0 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 2s ago 9m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (9m) 2s ago 9m 7780k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (8m) 61s ago 8m 7554k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (9m) 2s ago 9m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (8m) 61s ago 8m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 2s ago 9m 87.6M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (3m) 2s ago 3m 151M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (3m) 2s ago 3m 181M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:22:09.450 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (3m) 61s ago 3m 13.2M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (3m) 61s ago 3m 15.5M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (28s) 2s ago 10m 160M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (94s) 61s ago 8m 495M - 19.0.0-2946-g3d89accf b045ac545f82 57506042450c 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (10m) 2s ago 10m 49.4M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (8m) 61s ago 8m 40.8M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (9m) 2s ago 9m 13.9M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 61s ago 8m 16.3M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 2s ago 7m 219M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (7m) 2s ago 7m 218M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (6m) 2s ago 6m 179M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (6m) 61s ago 6m 279M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (5m) 61s ago 5m 248M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:22:09.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (5m) 61s ago 5m 299M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:22:09.452 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (37s) 2s ago 8m 44.8M - 2.43.0 a07b618ecd1d 80107eab4dbc 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 msgr2=0x7fa8c0070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 0x7fa8c0070480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b5c0 tx=0x7fa8c4005ee0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 msgr2=0x7fa8d41956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000d900 tx=0x7fa8d000dc10 comp rx=0 tx=0).stop 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 shutdown_connections 2024-06-03T01:22:09.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7fa8c006dfc0 0x7fa8c0070480 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b5c0 tx=0x7fa8c4005ee0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8d40ffbc0 0x7fa8d41956c0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000d900 tx=0x7fa8d000dc10 comp rx=0 tx=0).stop 2024-06-03T01:22:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 --2- 172.21.15.32:0/3839126012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d4100dc0 0x7fa8d4195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.452+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 >> 172.21.15.32:0/3839126012 conn(0x7fa8d40fb160 msgr2=0x7fa8d4104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.453+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 shutdown_connections 2024-06-03T01:22:09.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.453+0000 7fa8dbcaf700 1 -- 172.21.15.32:0/3839126012 wait complete. 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.631+0000 7f2b53da7700 1 -- 172.21.15.32:0/2869538770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c1086f0 msgr2=0x7f2b4c108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.631+0000 7f2b53da7700 1 --2- 172.21.15.32:0/2869538770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c108b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009910 tx=0x7f2b3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.632+0000 7f2b53da7700 1 -- 172.21.15.32:0/2869538770 shutdown_connections 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.632+0000 7f2b53da7700 1 --2- 172.21.15.32:0/2869538770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c108b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.632+0000 7f2b53da7700 1 --2- 172.21.15.32:0/2869538770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c107d90 0x7f2b4c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.632+0000 7f2b53da7700 1 -- 172.21.15.32:0/2869538770 >> 172.21.15.32:0/2869538770 conn(0x7f2b4c06deb0 msgr2=0x7f2b4c0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.633+0000 7f2b53da7700 1 -- 172.21.15.32:0/2869538770 shutdown_connections 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.633+0000 7f2b53da7700 1 -- 172.21.15.32:0/2869538770 wait complete. 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.633+0000 7f2b53da7700 1 Processor -- start 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b53da7700 1 -- start start 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b53da7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c107d90 0x7f2b4c1132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b53da7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b53da7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c10fdc0 con 0x7f2b4c107d90 2024-06-03T01:22:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b53da7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c10ff30 con 0x7f2b4c1086f0 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b51342700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b51342700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:46312/0 (socket says 172.21.15.32:46312) 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.634+0000 7f2b51342700 1 -- 172.21.15.32:0/1792221266 learned_addr learned my addr 172.21.15.32:0/1792221266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.635+0000 7f2b51b43700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c107d90 0x7f2b4c1132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.635+0000 7f2b51342700 1 -- 172.21.15.32:0/1792221266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c107d90 msgr2=0x7f2b4c1132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.635+0000 7f2b51342700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c107d90 0x7f2b4c1132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.635+0000 7f2b51342700 1 -- 172.21.15.32:0/1792221266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0095f0 con 0x7f2b4c1086f0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b51342700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005850 tx=0x7f2b3c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c01d070 con 0x7f2b4c1086f0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c1101b0 con 0x7f2b4c1086f0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b3c00f460 con 0x7f2b4c1086f0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c017630 con 0x7f2b4c1086f0 2024-06-03T01:22:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.636+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c1106a0 con 0x7f2b4c1086f0 2024-06-03T01:22:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.639+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f2b3c004b20 con 0x7f2b4c1086f0 2024-06-03T01:22:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.639+0000 7f2b42ffd700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 0x7f2b38070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.639+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2b3c08f290 con 0x7f2b4c1086f0 2024-06-03T01:22:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.639+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30005320 con 0x7f2b4c1086f0 2024-06-03T01:22:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.640+0000 7f2b51b43700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 0x7f2b38070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.640+0000 7f2b51b43700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 0x7f2b38070510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b48005fd0 tx=0x7f2b4800a560 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:09.647+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f2b3c05c4e0 con 0x7f2b4c1086f0 2024-06-03T01:22:09.702 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:09 smithi032 ceph-mon[29360]: pgmap v39: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 431 op/s 2024-06-03T01:22:09.702 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:09 smithi032 ceph-mon[29360]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:09.702 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:09 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:09.702 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:09 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:09 smithi044 ceph-mon[36956]: pgmap v39: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 431 op/s 2024-06-03T01:22:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:09 smithi044 ceph-mon[36956]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:09 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:09 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.166+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2b30005cc0 con 0x7f2b4c1086f0 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.167+0000 7f2b42ffd700 1 -- 172.21.15.32:0/1792221266 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f2b3c05c070 con 0x7f2b4c1086f0 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:22:10.169 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:22:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 msgr2=0x7f2b38070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 0x7f2b38070510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b48005fd0 tx=0x7f2b4800a560 comp rx=0 tx=0).stop 2024-06-03T01:22:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 msgr2=0x7f2b4c113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005850 tx=0x7f2b3c00be20 comp rx=0 tx=0).stop 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 shutdown_connections 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f2b3806e050 0x7f2b38070510 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b4c107d90 0x7f2b4c1132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 --2- 172.21.15.32:0/1792221266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b4c1086f0 0x7f2b4c113810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 >> 172.21.15.32:0/1792221266 conn(0x7f2b4c06deb0 msgr2=0x7f2b4c10d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 shutdown_connections 2024-06-03T01:22:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.169+0000 7f2b53da7700 1 -- 172.21.15.32:0/1792221266 wait complete. 2024-06-03T01:22:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.313+0000 7f0e08de2700 1 -- 172.21.15.32:0/2982293920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 msgr2=0x7f0e04101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.313+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2982293920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e04101270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009910 tx=0x7f0dec009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 -- 172.21.15.32:0/2982293920 shutdown_connections 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2982293920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e04101270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2982293920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e04100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 -- 172.21.15.32:0/2982293920 >> 172.21.15.32:0/2982293920 conn(0x7f0e040fb190 msgr2=0x7f0e040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 -- 172.21.15.32:0/2982293920 shutdown_connections 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.314+0000 7f0e08de2700 1 -- 172.21.15.32:0/2982293920 wait complete. 2024-06-03T01:22:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.315+0000 7f0e08de2700 1 Processor -- start 2024-06-03T01:22:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.315+0000 7f0e08de2700 1 -- start start 2024-06-03T01:22:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0e08de2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e041953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0e08de2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e041958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0e08de2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04195f00 con 0x7f0e04100e10 2024-06-03T01:22:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0e08de2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04196040 con 0x7f0e040ffc10 2024-06-03T01:22:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e041958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e041958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38026/0 (socket says 172.21.15.32:38026) 2024-06-03T01:22:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0dfbfff700 1 -- 172.21.15.32:0/2222801667 learned_addr learned my addr 172.21.15.32:0/2222801667 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.316+0000 7f0e0259c700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e041953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e0259c700 1 -- 172.21.15.32:0/2222801667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 msgr2=0x7f0e041958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e0259c700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e041958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e0259c700 1 -- 172.21.15.32:0/2222801667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0095f0 con 0x7f0e040ffc10 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e0259c700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e041953a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0df400e940 tx=0x7f0df400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df4004200 con 0x7f0e040ffc10 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e04068580 con 0x7f0e040ffc10 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0df4004360 con 0x7f0e040ffc10 2024-06-03T01:22:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df4003d00 con 0x7f0e040ffc10 2024-06-03T01:22:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.317+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e04068ad0 con 0x7f0e040ffc10 2024-06-03T01:22:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.319+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e04108700 con 0x7f0e040ffc10 2024-06-03T01:22:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.319+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f0df4021430 con 0x7f0e040ffc10 2024-06-03T01:22:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.320+0000 7f0dfb7fe700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 0x7f0df00748e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.320+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0df4014070 con 0x7f0e040ffc10 2024-06-03T01:22:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.320+0000 7f0dfbfff700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 0x7f0df00748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.321+0000 7f0dfbfff700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 0x7f0df00748e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009fd0 tx=0x7f0dec005a90 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:10.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.324+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f0df405b2d0 con 0x7f0e040ffc10 2024-06-03T01:22:10.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:10 smithi032 ceph-mon[29360]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:10.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:10 smithi032 ceph-mon[29360]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:10.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:10 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1792221266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.681+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 --> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e04105760 con 0x7f0df0072420 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.683+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/2222801667 <== mgr.24365 v2:172.21.15.44:6828/2860560309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0e04105760 con 0x7f0df0072420 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:22:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/2 daemons upgraded", 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mgr daemons", 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:22:10.685 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:22:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 msgr2=0x7f0df00748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 0x7f0df00748e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009fd0 tx=0x7f0dec005a90 comp rx=0 tx=0).stop 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 msgr2=0x7f0e041953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e041953a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0df400e940 tx=0x7f0df400ec50 comp rx=0 tx=0).stop 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 shutdown_connections 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:6828/2860560309,v1:172.21.15.44:6829/2860560309] conn(0x7f0df0072420 0x7f0df00748e0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009fd0 tx=0x7f0dec005a90 comp rx=0 tx=0).stop 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e040ffc10 0x7f0e041953a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 --2- 172.21.15.32:0/2222801667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e04100e10 0x7f0e041958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 >> 172.21.15.32:0/2222801667 conn(0x7f0e040fb190 msgr2=0x7f0e04104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 shutdown_connections 2024-06-03T01:22:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:10.688+0000 7f0e08de2700 1 -- 172.21.15.32:0/2222801667 wait complete. 2024-06-03T01:22:10.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:10 smithi044 ceph-mon[36956]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:10.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:10 smithi044 ceph-mon[36956]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:10.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:10 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1792221266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:11.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:11 smithi032 ceph-mon[29360]: pgmap v40: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-06-03T01:22:11.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:11 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:11.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:11 smithi044 ceph-mon[36956]: pgmap v40: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-06-03T01:22:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:11 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:12.629 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:12 smithi032 ceph-mon[29360]: from='client.24441 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:12.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:12 smithi044 ceph-mon[36956]: from='client.24441 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:13.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:13 smithi044 ceph-mon[36956]: pgmap v41: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-06-03T01:22:13.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:13 smithi032 ceph-mon[29360]: pgmap v41: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-06-03T01:22:15.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: pgmap v42: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 470 op/s 2024-06-03T01:22:15.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: osdmap e48: 6 total, 6 up, 6 in 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd='[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]': finished 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: mgrmap e26: smithi032.kfcfaa(active, starting, since 0.00766983s) 2024-06-03T01:22:15.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:22:15.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:22:15.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: pgmap v42: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 470 op/s 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 172.21.15.44:0/55769862' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: osdmap e48: 6 total, 6 up, 6 in 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.24365 ' entity='mgr.smithi044.wszjlw' cmd='[{"prefix": "mgr fail", "who": "smithi044.wszjlw"}]': finished 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: mgrmap e26: smithi032.kfcfaa(active, starting, since 0.00766983s) 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:22:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:22:15.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:22:17.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:17 smithi032 ceph-mon[29360]: mgrmap e27: smithi032.kfcfaa(active, since 1.01252s) 2024-06-03T01:22:17.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:17 smithi032 ceph-mon[29360]: pgmap v3: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:17.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:17 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:17.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:17 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:17.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:17 smithi044 ceph-mon[36956]: mgrmap e27: smithi032.kfcfaa(active, since 1.01252s) 2024-06-03T01:22:17.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:17 smithi044 ceph-mon[36956]: pgmap v3: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:17.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:17 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:17.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:17 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: [03/Jun/2024:01:22:16] ENGINE Bus STARTING 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: [03/Jun/2024:01:22:16] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: [03/Jun/2024:01:22:16] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: [03/Jun/2024:01:22:16] ENGINE Bus STARTED 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: pgmap v4: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:18.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:18 smithi032 ceph-mon[29360]: mgrmap e28: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: [03/Jun/2024:01:22:16] ENGINE Bus STARTING 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: [03/Jun/2024:01:22:16] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: [03/Jun/2024:01:22:16] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: [03/Jun/2024:01:22:16] ENGINE Bus STARTED 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: pgmap v4: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:18.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:18 smithi044 ceph-mon[36956]: mgrmap e28: smithi032.kfcfaa(active, since 2s) 2024-06-03T01:22:20.356 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:20 smithi032 ceph-mon[29360]: pgmap v5: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:20.356 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:20 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:20.357 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:20 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:20.357 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:20 smithi032 ceph-mon[29360]: mgrmap e29: smithi032.kfcfaa(active, since 4s) 2024-06-03T01:22:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:20 smithi044 ceph-mon[36956]: pgmap v5: 65 pgs: 65 active+clean; 321 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:22:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:20 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:20 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:20 smithi044 ceph-mon[36956]: mgrmap e29: smithi032.kfcfaa(active, since 4s) 2024-06-03T01:22:21.653 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:21 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:21.653 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:21 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:21.653 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:21 smithi044 ceph-mon[36956]: pgmap v6: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 75 op/s 2024-06-03T01:22:21.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:21 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:21.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:21 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:21.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:21 smithi032 ceph-mon[29360]: pgmap v6: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 75 op/s 2024-06-03T01:22:23.290 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-06-03T01:22:23.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-06-03T01:22:23.627 DEBUG:teuthology.parallel:result is None 2024-06-03T01:22:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: pgmap v7: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66 KiB/s rd, 1.6 MiB/s wr, 386 op/s 2024-06-03T01:22:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:22:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:22:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:22:24.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:22:24.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:24 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:22:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: pgmap v7: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66 KiB/s rd, 1.6 MiB/s wr, 386 op/s 2024-06-03T01:22:24.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:22:24.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:22:24.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:22:24.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:22:24.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:24 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/248339551' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:22:25.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:25 smithi044 ceph-mon[36956]: mgrmap e30: smithi032.kfcfaa(active, since 9s), standbys: smithi044.wszjlw 2024-06-03T01:22:25.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:25 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:25.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:25 smithi032 ceph-mon[29360]: mgrmap e30: smithi032.kfcfaa(active, since 9s), standbys: smithi044.wszjlw 2024-06-03T01:22:25.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:25 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:26.372 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:26 smithi044 ceph-mon[36956]: pgmap v8: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-06-03T01:22:26.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:26 smithi032 ceph-mon[29360]: pgmap v8: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-06-03T01:22:28.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:28 smithi044 ceph-mon[36956]: pgmap v9: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s rd, 1.8 MiB/s wr, 433 op/s 2024-06-03T01:22:28.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:28 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:28.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:28 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:28 smithi032 ceph-mon[29360]: pgmap v9: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s rd, 1.8 MiB/s wr, 433 op/s 2024-06-03T01:22:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:28 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:28 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:29.953 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:29 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:29 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:29 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:29 smithi044 ceph-mon[36956]: pgmap v10: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 396 op/s 2024-06-03T01:22:30.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:29 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:30.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:29 smithi032 ceph-mon[29360]: pgmap v10: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 396 op/s 2024-06-03T01:22:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:30 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:30 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:31 smithi044 ceph-mon[36956]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 435 op/s 2024-06-03T01:22:32.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:31 smithi032 ceph-mon[29360]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 435 op/s 2024-06-03T01:22:34.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:34 smithi044 ceph-mon[36956]: pgmap v12: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 493 op/s 2024-06-03T01:22:34.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:34 smithi032 ceph-mon[29360]: pgmap v12: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 493 op/s 2024-06-03T01:22:34.864 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-06-03T01:22:34.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-03T01:22:35.247 DEBUG:teuthology.parallel:result is None 2024-06-03T01:22:35.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-03T01:22:35.287 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-03T01:22:35.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-06-03T01:22:35.322 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-06-03T01:22:35.322 DEBUG:teuthology.parallel:result is None 2024-06-03T01:22:36.030 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:35 smithi032 ceph-mon[29360]: pgmap v13: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 299 op/s 2024-06-03T01:22:36.082 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:35 smithi044 ceph-mon[36956]: pgmap v13: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 299 op/s 2024-06-03T01:22:38.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:38 smithi044 ceph-mon[36956]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 401 op/s 2024-06-03T01:22:38.473 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:38 smithi032 ceph-mon[29360]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 401 op/s 2024-06-03T01:22:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:39.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 969 KiB/s wr, 230 op/s 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:22:39.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:39 smithi032 ceph-mon[29360]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:22:39.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 969 KiB/s wr, 230 op/s 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:22:39.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:22:39.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:39 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 -- 172.21.15.32:0/2317886068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 msgr2=0x7f8f90103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 --2- 172.21.15.32:0/2317886068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90103ba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009910 tx=0x7f8f80009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 -- 172.21.15.32:0/2317886068 shutdown_connections 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 --2- 172.21.15.32:0/2317886068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 0x7f8f901064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 --2- 172.21.15.32:0/2317886068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90103ba0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 -- 172.21.15.32:0/2317886068 >> 172.21.15.32:0/2317886068 conn(0x7f8f900fb160 msgr2=0x7f8f900fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 -- 172.21.15.32:0/2317886068 shutdown_connections 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 -- 172.21.15.32:0/2317886068 wait complete. 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.813+0000 7f8f95485700 1 Processor -- start 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f95485700 1 -- start start 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f95485700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f95485700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 0x7f8f900ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f95485700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ffca0 con 0x7f8f901017b0 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f95485700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ffde0 con 0x7f8f901040e0 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44650/0 (socket says 172.21.15.32:44650) 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 -- 172.21.15.32:0/3675006064 learned_addr learned my addr 172.21.15.32:0/3675006064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8e7fc700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 0x7f8f900ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 -- 172.21.15.32:0/3675006064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 msgr2=0x7f8f900ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 0x7f8f900ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 -- 172.21.15.32:0/3675006064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800095f0 con 0x7f8f901017b0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.814+0000 7f8f8effd700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bd70 tx=0x7f8f8000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.815+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f8001d070 con 0x7f8f901017b0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.815+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f80005940 con 0x7f8f901017b0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.815+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f90100060 con 0x7f8f901017b0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.815+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f80021430 con 0x7f8f901017b0 2024-06-03T01:22:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.815+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f90100550 con 0x7f8f901017b0 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.816+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90190630 con 0x7f8f901017b0 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.816+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f8f8000f860 con 0x7f8f901017b0 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.817+0000 7f8f77fff700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 0x7f8f78070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.817+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8f8008f200 con 0x7f8f901017b0 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.817+0000 7f8f8e7fc700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 0x7f8f78070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.817+0000 7f8f8e7fc700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 0x7f8f78070480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400ac50 tx=0x7f8f8400a380 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:40.822+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f8f8005c320 con 0x7f8f901017b0 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:22:40.939 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:40 smithi032 ceph-mon[29360]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:22:40.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-06-03T01:22:40.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:40 smithi044 ceph-mon[36956]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-06-03T01:22:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.060+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f9002cf20 con 0x7f8f7806dfc0 2024-06-03T01:22:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.061+0000 7f8f77fff700 1 -- 172.21.15.32:0/3675006064 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f8f9002cf20 con 0x7f8f7806dfc0 2024-06-03T01:22:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.063+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 msgr2=0x7f8f78070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.063+0000 7f8f95485700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 0x7f8f78070480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400ac50 tx=0x7f8f8400a380 comp rx=0 tx=0).stop 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.063+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 msgr2=0x7f8f90101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.063+0000 7f8f95485700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bd70 tx=0x7f8f8000bda0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 shutdown_connections 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8f7806dfc0 0x7f8f78070480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f901017b0 0x7f8f90101110 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 --2- 172.21.15.32:0/3675006064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f901040e0 0x7f8f900ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 >> 172.21.15.32:0/3675006064 conn(0x7f8f900fb160 msgr2=0x7f8f900fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 shutdown_connections 2024-06-03T01:22:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.064+0000 7f8f95485700 1 -- 172.21.15.32:0/3675006064 wait complete. 2024-06-03T01:22:41.076 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:22:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- 172.21.15.32:0/3976376499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 msgr2=0x7f95f41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/3976376499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f41010d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009910 tx=0x7f95e4009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- 172.21.15.32:0/3976376499 shutdown_connections 2024-06-03T01:22:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/3976376499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/3976376499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f41010d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- 172.21.15.32:0/3976376499 >> 172.21.15.32:0/3976376499 conn(0x7f95f40fc230 msgr2=0x7f95f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- 172.21.15.32:0/3976376499 shutdown_connections 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- 172.21.15.32:0/3976376499 wait complete. 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 Processor -- start 2024-06-03T01:22:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.174+0000 7f95fa5d9700 1 -- start start 2024-06-03T01:22:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95fa5d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95fa5d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95fa5d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f41992b0 con 0x7f95f4100cb0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95fa5d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f41993f0 con 0x7f95f4101eb0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44666/0 (socket says 172.21.15.32:44666) 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 -- 172.21.15.32:0/1301306510 learned_addr learned my addr 172.21.15.32:0/1301306510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f37fe700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 -- 172.21.15.32:0/1301306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 msgr2=0x7f95f4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 -- 172.21.15.32:0/1301306510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40095f0 con 0x7f95f4100cb0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f3fff700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95e400bda0 tx=0x7f95e400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e401d070 con 0x7f95f4100cb0 2024-06-03T01:22:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95e4004c50 con 0x7f95f4100cb0 2024-06-03T01:22:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.175+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e400f690 con 0x7f95f4100cb0 2024-06-03T01:22:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.176+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f419de40 con 0x7f95f4100cb0 2024-06-03T01:22:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.176+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f419e250 con 0x7f95f4100cb0 2024-06-03T01:22:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.177+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f4105f90 con 0x7f95f4100cb0 2024-06-03T01:22:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.177+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f95e40229b0 con 0x7f95f4100cb0 2024-06-03T01:22:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.178+0000 7f95f17fa700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 0x7f95dc070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.178+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f95e408f560 con 0x7f95f4100cb0 2024-06-03T01:22:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.178+0000 7f95f37fe700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 0x7f95dc070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.178+0000 7f95f37fe700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 0x7f95dc070480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95e8006fd0 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.182+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f95e405c730 con 0x7f95f4100cb0 2024-06-03T01:22:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.412+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95f40611d0 con 0x7f95dc06dfc0 2024-06-03T01:22:41.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.413+0000 7f95f17fa700 1 -- 172.21.15.32:0/1301306510 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f95f40611d0 con 0x7f95dc06dfc0 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.415+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 msgr2=0x7f95dc070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 0x7f95dc070480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95e8006fd0 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 msgr2=0x7f95f4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95e400bda0 tx=0x7f95e400be80 comp rx=0 tx=0).stop 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 shutdown_connections 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f95dc06dfc0 0x7f95dc070480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f4100cb0 0x7f95f4198750 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 --2- 172.21.15.32:0/1301306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f4101eb0 0x7f95f4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 >> 172.21.15.32:0/1301306510 conn(0x7f95f40fc230 msgr2=0x7f95f41030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 shutdown_connections 2024-06-03T01:22:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.416+0000 7f95fa5d9700 1 -- 172.21.15.32:0/1301306510 wait complete. 2024-06-03T01:22:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 -- 172.21.15.32:0/3967398924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908100dc0 msgr2=0x7f2908101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 --2- 172.21.15.32:0/3967398924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908100dc0 0x7f2908101220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009910 tx=0x7f28f8009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 -- 172.21.15.32:0/3967398924 shutdown_connections 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 --2- 172.21.15.32:0/3967398924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908100dc0 0x7f2908101220 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 --2- 172.21.15.32:0/3967398924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29080ffbc0 0x7f29080fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 -- 172.21.15.32:0/3967398924 >> 172.21.15.32:0/3967398924 conn(0x7f29080fb160 msgr2=0x7f29080fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 -- 172.21.15.32:0/3967398924 shutdown_connections 2024-06-03T01:22:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.524+0000 7f290fb9c700 1 -- 172.21.15.32:0/3967398924 wait complete. 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 Processor -- start 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 -- start start 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908100dc0 0x7f29081021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081026e0 con 0x7f29080ffbc0 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290fb9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908102820 con 0x7f2908100dc0 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290d938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290d938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44678/0 (socket says 172.21.15.32:44678) 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.525+0000 7f290d938700 1 -- 172.21.15.32:0/2004868409 learned_addr learned my addr 172.21.15.32:0/2004868409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290d938700 1 -- 172.21.15.32:0/2004868409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908100dc0 msgr2=0x7f29081021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290d938700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908100dc0 0x7f29081021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290d938700 1 -- 172.21.15.32:0/2004868409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80095f0 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290d938700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f290400d900 tx=0x7f290400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29040041f0 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2904004350 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2908102b00 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2904003da0 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.526+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290819e830 con 0x7f29080ffbc0 2024-06-03T01:22:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.527+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2908068af0 con 0x7f29080ffbc0 2024-06-03T01:22:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.527+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f2904009730 con 0x7f29080ffbc0 2024-06-03T01:22:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.528+0000 7f28feffd700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 0x7f28f4070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.528+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f290408dfc0 con 0x7f29080ffbc0 2024-06-03T01:22:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.528+0000 7f290d137700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 0x7f28f4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.528+0000 7f290d137700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 0x7f28f4070480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28f800b5c0 tx=0x7f28f8005960 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.533+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f290405b0e0 con 0x7f29080ffbc0 2024-06-03T01:22:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.751+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2908069560 con 0x7f28f406dfc0 2024-06-03T01:22:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.762+0000 7f28feffd700 1 -- 172.21.15.32:0/2004868409 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2908069560 con 0x7f28f406dfc0 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (8m) 14s ago 9m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (10m) 14s ago 10m 7818k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (9m) 22s ago 9m 7646k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (10m) 14s ago 10m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (8m) 22s ago 8m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (8m) 14s ago 9m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (4m) 14s ago 4m 141M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (4m) 14s ago 4m 142M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (4m) 22s ago 4m 13.8M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (4m) 22s ago 4m 15.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:22:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (61s) 14s ago 11m 480M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (2m) 22s ago 8m 82.1M - 19.0.0-2946-g3d89accf b045ac545f82 57506042450c 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (11m) 14s ago 11m 52.8M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (8m) 22s ago 8m 42.6M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (10m) 14s ago 10m 14.5M - 1.5.0 0da6a335fe13 c14b53b298bd 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 22s ago 8m 16.3M - 1.5.0 0da6a335fe13 a6ef56dd09de 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (8m) 14s ago 8m 227M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (7m) 14s ago 7m 230M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (7m) 14s ago 7m 191M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (6m) 22s ago 6m 259M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (6m) 22s ago 6m 237M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (5m) 22s ago 5m 221M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:22:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (69s) 14s ago 9m 60.4M - 2.43.0 a07b618ecd1d 80107eab4dbc 2024-06-03T01:22:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.764+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 msgr2=0x7f28f4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.764+0000 7f290fb9c700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 0x7f28f4070480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28f800b5c0 tx=0x7f28f8005960 comp rx=0 tx=0).stop 2024-06-03T01:22:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.764+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 msgr2=0x7f2908103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.764+0000 7f290fb9c700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f290400d900 tx=0x7f290400dcc0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 shutdown_connections 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f28f406dfc0 0x7f28f4070480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29080ffbc0 0x7f2908103b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 --2- 172.21.15.32:0/2004868409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908100dc0 0x7f29081021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 >> 172.21.15.32:0/2004868409 conn(0x7f29080fb160 msgr2=0x7f2908068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 shutdown_connections 2024-06-03T01:22:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.765+0000 7f290fb9c700 1 -- 172.21.15.32:0/2004868409 wait complete. 2024-06-03T01:22:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 -- 172.21.15.32:0/2024804522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 msgr2=0x7ff3b01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/2024804522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b01023a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff398009910 tx=0x7ff398009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 -- 172.21.15.32:0/2024804522 shutdown_connections 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/2024804522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b01023a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/2024804522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 0x7ff3b00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 -- 172.21.15.32:0/2024804522 >> 172.21.15.32:0/2024804522 conn(0x7ff3b00fa8c0 msgr2=0x7ff3b00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 -- 172.21.15.32:0/2024804522 shutdown_connections 2024-06-03T01:22:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.873+0000 7ff3b49c2700 1 -- 172.21.15.32:0/2024804522 wait complete. 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 Processor -- start 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 -- start start 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 0x7ff3b01018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0100430 con 0x7ff3b0101f40 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.874+0000 7ff3b49c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0100570 con 0x7ff3b00fef60 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44708/0 (socket says 172.21.15.32:44708) 2024-06-03T01:22:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 -- 172.21.15.32:0/1538800025 learned_addr learned my addr 172.21.15.32:0/1538800025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3ae59c700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 0x7ff3b01018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 -- 172.21.15.32:0/1538800025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 msgr2=0x7ff3b01018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 0x7ff3b01018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 -- 172.21.15.32:0/1538800025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980095f0 con 0x7ff3b0101f40 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a7fff700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff39801d070 con 0x7ff3b0101f40 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff39800f460 con 0x7ff3b0101f40 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff398021640 con 0x7ff3b0101f40 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b01007f0 con 0x7ff3b0101f40 2024-06-03T01:22:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.875+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b0100c90 con 0x7ff3b0101f40 2024-06-03T01:22:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.876+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0109800 con 0x7ff3b0101f40 2024-06-03T01:22:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.877+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 92158+0+0 (secure 0 0 0) 0x7ff39800fab0 con 0x7ff3b0101f40 2024-06-03T01:22:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.877+0000 7ff3a77fe700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 0x7ff39c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.877+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff39808f110 con 0x7ff3b0101f40 2024-06-03T01:22:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.877+0000 7ff3ae59c700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 0x7ff39c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.878+0000 7ff3ae59c700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 0x7ff39c070480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0068c10 tx=0x7ff3a000b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:41.883+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff39805c230 con 0x7ff3b0101f40 2024-06-03T01:22:41.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:41 smithi044 ceph-mon[36956]: pgmap v16: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 974 KiB/s wr, 230 op/s 2024-06-03T01:22:41.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:41 smithi044 ceph-mon[36956]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:41.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:41 smithi044 ceph-mon[36956]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:41.986 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:41 smithi032 ceph-mon[29360]: pgmap v16: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 974 KiB/s wr, 230 op/s 2024-06-03T01:22:41.986 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:41 smithi032 ceph-mon[29360]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:41.986 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:41 smithi032 ceph-mon[29360]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.185+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3b01099f0 con 0x7ff3b0101f40 2024-06-03T01:22:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.185+0000 7ff3a77fe700 1 -- 172.21.15.32:0/1538800025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7ff39805bdc0 con 0x7ff3b0101f40 2024-06-03T01:22:42.185 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:22:42.185 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:22:42.185 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:22:42.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:22:42.187 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:22:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 msgr2=0x7ff39c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 0x7ff39c070480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0068c10 tx=0x7ff3a000b410 comp rx=0 tx=0).stop 2024-06-03T01:22:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 msgr2=0x7ff3b00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39800bd70 comp rx=0 tx=0).stop 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 shutdown_connections 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff39c06dfc0 0x7ff39c070480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3b00fef60 0x7ff3b01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 --2- 172.21.15.32:0/1538800025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b0101f40 0x7ff3b00ffef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 >> 172.21.15.32:0/1538800025 conn(0x7ff3b00fa8c0 msgr2=0x7ff3b0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 shutdown_connections 2024-06-03T01:22:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.187+0000 7ff3b49c2700 1 -- 172.21.15.32:0/1538800025 wait complete. 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 -- 172.21.15.32:0/1527776791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4100cb0 msgr2=0x7fd2a41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 --2- 172.21.15.32:0/1527776791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4100cb0 0x7fd2a41010d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd294009910 tx=0x7fd294009c20 comp rx=0 tx=0).stop 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 -- 172.21.15.32:0/1527776791 shutdown_connections 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 --2- 172.21.15.32:0/1527776791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 --2- 172.21.15.32:0/1527776791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4100cb0 0x7fd2a41010d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.297+0000 7fd2aa069700 1 -- 172.21.15.32:0/1527776791 >> 172.21.15.32:0/1527776791 conn(0x7fd2a40fc230 msgr2=0x7fd2a40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.298+0000 7fd2aa069700 1 -- 172.21.15.32:0/1527776791 shutdown_connections 2024-06-03T01:22:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.298+0000 7fd2aa069700 1 -- 172.21.15.32:0/1527776791 wait complete. 2024-06-03T01:22:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.298+0000 7fd2aa069700 1 Processor -- start 2024-06-03T01:22:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.298+0000 7fd2aa069700 1 -- start start 2024-06-03T01:22:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4100cb0 0x7fd2a4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4199270 con 0x7fd2a4101eb0 2024-06-03T01:22:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41993b0 con 0x7fd2a4100cb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44714/0 (socket says 172.21.15.32:44714) 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 -- 172.21.15.32:0/2349326496 learned_addr learned my addr 172.21.15.32:0/2349326496 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a37fe700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4100cb0 0x7fd2a4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 -- 172.21.15.32:0/2349326496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4100cb0 msgr2=0x7fd2a4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4100cb0 0x7fd2a4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 -- 172.21.15.32:0/2349326496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940095f0 con 0x7fd2a4101eb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a2ffd700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c8f0 tx=0x7fd29800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd298004a00 con 0x7fd2a4101eb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a419de60 con 0x7fd2a4101eb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd2980054d0 con 0x7fd2a4101eb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd298007ce0 con 0x7fd2a4101eb0 2024-06-03T01:22:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.299+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a419e380 con 0x7fd2a4101eb0 2024-06-03T01:22:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.300+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a4105f90 con 0x7fd2a4101eb0 2024-06-03T01:22:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.301+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 92158+0+0 (secure 0 0 0) 0x7fd298005020 con 0x7fd2a4101eb0 2024-06-03T01:22:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.301+0000 7fd2a0ff9700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 0x7fd28c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.302+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd298013070 con 0x7fd2a4101eb0 2024-06-03T01:22:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.302+0000 7fd2a37fe700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 0x7fd28c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.302+0000 7fd2a37fe700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 0x7fd28c070480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd294006010 tx=0x7fd29400b540 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.306+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fd29805ada0 con 0x7fd2a4101eb0 2024-06-03T01:22:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.540+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2a40611d0 con 0x7fd28c06dfc0 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.542+0000 7fd2a0ff9700 1 -- 172.21.15.32:0/2349326496 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fd2a40611d0 con 0x7fd28c06dfc0 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/2 daemons upgraded", 2024-06-03T01:22:42.542 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-06-03T01:22:42.543 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:22:42.543 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 msgr2=0x7fd28c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 0x7fd28c070480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd294006010 tx=0x7fd29400b540 comp rx=0 tx=0).stop 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 msgr2=0x7fd2a4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c8f0 tx=0x7fd29800cc00 comp rx=0 tx=0).stop 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 shutdown_connections 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd28c06dfc0 0x7fd28c070480 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a4100cb0 0x7fd2a4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 --2- 172.21.15.32:0/2349326496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2a4101eb0 0x7fd2a4198c50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 >> 172.21.15.32:0/2349326496 conn(0x7fd2a40fc230 msgr2=0x7fd2a41030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 shutdown_connections 2024-06-03T01:22:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:22:42.544+0000 7fd2aa069700 1 -- 172.21.15.32:0/2349326496 wait complete. 2024-06-03T01:22:42.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:42 smithi032 ceph-mon[29360]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:42.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:42 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1538800025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:42.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:42 smithi044 ceph-mon[36956]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:42.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:42 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1538800025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:43.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:43 smithi032 ceph-mon[29360]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:43.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:43 smithi032 ceph-mon[29360]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 684 KiB/s wr, 192 op/s 2024-06-03T01:22:43.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:43 smithi044 ceph-mon[36956]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:22:43.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:43 smithi044 ceph-mon[36956]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 684 KiB/s wr, 192 op/s 2024-06-03T01:22:46.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 411 KiB/s wr, 103 op/s 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:22:46.055 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 411 KiB/s wr, 103 op/s 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:22:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:46 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:22:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:46 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:22:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:46 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:46 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:22:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:46 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:22:47.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:46 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:47 smithi032 ceph-mon[29360]: Upgrade: Updating mgr.smithi044.wszjlw 2024-06-03T01:22:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:47 smithi032 ceph-mon[29360]: Deploying daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:22:48.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:47 smithi032 ceph-mon[29360]: pgmap v19: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 412 KiB/s wr, 103 op/s 2024-06-03T01:22:48.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:47 smithi044 ceph-mon[36956]: Upgrade: Updating mgr.smithi044.wszjlw 2024-06-03T01:22:48.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:47 smithi044 ceph-mon[36956]: Deploying daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:22:48.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:47 smithi044 ceph-mon[36956]: pgmap v19: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 412 KiB/s wr, 103 op/s 2024-06-03T01:22:50.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:50 smithi044 ceph-mon[36956]: pgmap v20: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 6.3 KiB/s wr, 1 op/s 2024-06-03T01:22:50.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:50 smithi032 ceph-mon[29360]: pgmap v20: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 6.3 KiB/s wr, 1 op/s 2024-06-03T01:22:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:51 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:51 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:51 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:51.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:51 smithi032 ceph-mon[29360]: pgmap v21: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 6.3 KiB/s wr, 1 op/s 2024-06-03T01:22:51.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:51 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:51.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:51 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:51.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:51 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:51.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:51 smithi044 ceph-mon[36956]: pgmap v21: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 6.3 KiB/s wr, 1 op/s 2024-06-03T01:22:54.397 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:54 smithi032 ceph-mon[29360]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1.4 KiB/s wr, 1 op/s 2024-06-03T01:22:54.397 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:54 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:54.397 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:54 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:54 smithi044 ceph-mon[36956]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1.4 KiB/s wr, 1 op/s 2024-06-03T01:22:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:54 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:54 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.446 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:56 smithi044 ceph-mon[36956]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-06-03T01:22:56.446 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:56 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.446 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:56 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.446 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:56 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.447 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:56 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:56 smithi032 ceph-mon[29360]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-06-03T01:22:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:56 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:56 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:56 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:56 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: pgmap v24: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:22:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:58 smithi032 ceph-mon[29360]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:22:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: pgmap v24: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-06-03T01:22:58.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:22:58.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:22:58.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:22:58.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:22:58.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:22:58.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:58 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/1110288553' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:22:59.122 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: mgrmap e31: smithi032.kfcfaa(active, since 43s), standbys: smithi044.wszjlw 2024-06-03T01:22:59.122 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.kfcfaa"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.kfcfaa"}]': finished 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:59.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.wszjlw"}]': finished 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.125 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.125 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.125 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.125 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:22:59 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: mgrmap e31: smithi032.kfcfaa(active, since 43s), standbys: smithi044.wszjlw 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.445 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.kfcfaa"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.kfcfaa"}]': finished 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.wszjlw"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.wszjlw"}]': finished 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.446 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.447 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:22:59.447 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:22:59.447 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:22:59 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:00.418 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all mgr 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all rgw 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all rbd-mirror 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all cephfs-mirror 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all iscsi 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all nfs 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all nvmeof 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: pgmap v25: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:00.419 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:00 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:00.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all mgr 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all rgw 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all rbd-mirror 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all cephfs-mirror 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all iscsi 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all nfs 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all nvmeof 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: pgmap v25: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:00.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:00 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:01.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:01 smithi032 ceph-mon[29360]: Upgrade: Updating node-exporter.smithi032 (1/2) 2024-06-03T01:23:01.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:01 smithi032 ceph-mon[29360]: Deploying daemon node-exporter.smithi032 on smithi032 2024-06-03T01:23:01.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:01 smithi044 ceph-mon[36956]: Upgrade: Updating node-exporter.smithi032 (1/2) 2024-06-03T01:23:01.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:01 smithi044 ceph-mon[36956]: Deploying daemon node-exporter.smithi032 on smithi032 2024-06-03T01:23:02.417 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:02 smithi032 ceph-mon[29360]: pgmap v26: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:02 smithi044 ceph-mon[36956]: pgmap v26: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:04.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:04 smithi032 ceph-mon[29360]: pgmap v27: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:04.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:04 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:04.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:04 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:04 smithi044 ceph-mon[36956]: pgmap v27: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:23:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:04 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:04 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:06.061 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:05 smithi044 ceph-mon[36956]: Upgrade: Updating node-exporter.smithi044 (2/2) 2024-06-03T01:23:06.061 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:05 smithi044 ceph-mon[36956]: Deploying daemon node-exporter.smithi044 on smithi044 2024-06-03T01:23:06.061 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:05 smithi044 ceph-mon[36956]: pgmap v28: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:06.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:05 smithi032 ceph-mon[29360]: Upgrade: Updating node-exporter.smithi044 (2/2) 2024-06-03T01:23:06.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:05 smithi032 ceph-mon[29360]: Deploying daemon node-exporter.smithi044 on smithi044 2024-06-03T01:23:06.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:05 smithi032 ceph-mon[29360]: pgmap v28: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:08 smithi032 ceph-mon[29360]: pgmap v29: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:08 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:08 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:08 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:08.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:08 smithi044 ceph-mon[36956]: pgmap v29: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:08.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:08 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:08.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:08 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:08.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:08 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:10 smithi032 ceph-mon[29360]: pgmap v30: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:10 smithi044 ceph-mon[36956]: pgmap v30: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:11 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:11 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:11 smithi032 ceph-mon[29360]: pgmap v31: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:11 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:11 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:11 smithi044 ceph-mon[36956]: pgmap v31: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 -- 172.21.15.32:0/2831322749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 msgr2=0x7f49dc103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 --2- 172.21.15.32:0/2831322749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc103b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009910 tx=0x7f49cc009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 -- 172.21.15.32:0/2831322749 shutdown_connections 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 --2- 172.21.15.32:0/2831322749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 0x7f49dc106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 --2- 172.21.15.32:0/2831322749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc103b20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 -- 172.21.15.32:0/2831322749 >> 172.21.15.32:0/2831322749 conn(0x7f49dc0fb120 msgr2=0x7f49dc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 -- 172.21.15.32:0/2831322749 shutdown_connections 2024-06-03T01:23:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.677+0000 7f49e0e18700 1 -- 172.21.15.32:0/2831322749 wait complete. 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 Processor -- start 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 -- start start 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 0x7f49dc1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc196fd0 con 0x7f49dc101730 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.678+0000 7f49e0e18700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc197110 con 0x7f49dc104060 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40534/0 (socket says 172.21.15.32:40534) 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 -- 172.21.15.32:0/3713978976 learned_addr learned my addr 172.21.15.32:0/3713978976 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49d9d9b700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 0x7f49dc1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 -- 172.21.15.32:0/3713978976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 msgr2=0x7f49dc1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 0x7f49dc1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 -- 172.21.15.32:0/3713978976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0095f0 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49da59c700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc01d070 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc19bb60 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f49cc00f460 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.679+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc19c050 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.680+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc021640 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.680+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc190630 con 0x7f49dc101730 2024-06-03T01:23:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.681+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f49cc00fab0 con 0x7f49dc101730 2024-06-03T01:23:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.682+0000 7f49cb7fe700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 0x7f49c4070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.682+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f49cc08f060 con 0x7f49dc101730 2024-06-03T01:23:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.682+0000 7f49d9d9b700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 0x7f49c4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.682+0000 7f49d9d9b700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 0x7f49c4070480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49d0004630 tx=0x7f49d000b3c0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.687+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f49cc05c180 con 0x7f49dc101730 2024-06-03T01:23:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.922+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49dc0611d0 con 0x7f49c406dfc0 2024-06-03T01:23:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.923+0000 7f49cb7fe700 1 -- 172.21.15.32:0/3713978976 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f49dc0611d0 con 0x7f49c406dfc0 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.925+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 msgr2=0x7f49c4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.925+0000 7f49e0e18700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 0x7f49c4070480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49d0004630 tx=0x7f49d000b3c0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.925+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 msgr2=0x7f49dc196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.925+0000 7f49e0e18700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc00bd70 comp rx=0 tx=0).stop 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 shutdown_connections 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f49c406dfc0 0x7f49c4070480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49dc101730 0x7f49dc196470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 --2- 172.21.15.32:0/3713978976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49dc104060 0x7f49dc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 >> 172.21.15.32:0/3713978976 conn(0x7f49dc0fb120 msgr2=0x7f49dc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 shutdown_connections 2024-06-03T01:23:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:12.926+0000 7f49e0e18700 1 -- 172.21.15.32:0/3713978976 wait complete. 2024-06-03T01:23:12.936 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:23:12.937 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.937 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.937 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:12.937 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 -- 172.21.15.32:0/1978399887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 msgr2=0x7f5a940fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1978399887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a940fffd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009910 tx=0x7f5a84009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 -- 172.21.15.32:0/1978399887 shutdown_connections 2024-06-03T01:23:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1978399887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 0x7f5a94101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1978399887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a940fffd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 -- 172.21.15.32:0/1978399887 >> 172.21.15.32:0/1978399887 conn(0x7f5a940fb150 msgr2=0x7f5a940fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 -- 172.21.15.32:0/1978399887 shutdown_connections 2024-06-03T01:23:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.039+0000 7f5a9892c700 1 -- 172.21.15.32:0/1978399887 wait complete. 2024-06-03T01:23:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 Processor -- start 2024-06-03T01:23:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 -- start start 2024-06-03T01:23:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 0x7f5a9419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9419b810 con 0x7f5a940ffbb0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9892c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9419b950 con 0x7f5a94100db0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40554/0 (socket says 172.21.15.32:40554) 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 -- 172.21.15.32:0/1054842060 learned_addr learned my addr 172.21.15.32:0/1054842060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a91d9b700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 0x7f5a9419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 -- 172.21.15.32:0/1054842060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 msgr2=0x7f5a9419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 0x7f5a9419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.040+0000 7f5a9259c700 1 -- 172.21.15.32:0/1054842060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840095f0 con 0x7f5a940ffbb0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a9259c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400bda0 tx=0x7f5a8400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8401d070 con 0x7f5a940ffbb0 2024-06-03T01:23:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a8400f460 con 0x7f5a940ffbb0 2024-06-03T01:23:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9419bbd0 con 0x7f5a940ffbb0 2024-06-03T01:23:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84021620 con 0x7f5a940ffbb0 2024-06-03T01:23:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.041+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9419c0c0 con 0x7f5a940ffbb0 2024-06-03T01:23:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.042+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94108710 con 0x7f5a940ffbb0 2024-06-03T01:23:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.042+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f5a8400f5d0 con 0x7f5a940ffbb0 2024-06-03T01:23:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.043+0000 7f5a837fe700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 0x7f5a7c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.043+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5a8408f1a0 con 0x7f5a940ffbb0 2024-06-03T01:23:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.043+0000 7f5a91d9b700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 0x7f5a7c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.044+0000 7f5a91d9b700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 0x7f5a7c070480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94069ef0 tx=0x7f5a8800b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.048+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5a8405c220 con 0x7f5a940ffbb0 2024-06-03T01:23:13.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:13.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:13.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:13.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.312+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a94105700 con 0x7f5a7c06dfc0 2024-06-03T01:23:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.314+0000 7f5a837fe700 1 -- 172.21.15.32:0/1054842060 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f5a94105700 con 0x7f5a7c06dfc0 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 msgr2=0x7f5a7c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 0x7f5a7c070480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94069ef0 tx=0x7f5a8800b410 comp rx=0 tx=0).stop 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 msgr2=0x7f5a94069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400bda0 tx=0x7f5a8400bdd0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 shutdown_connections 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5a7c06dfc0 0x7f5a7c070480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a940ffbb0 0x7f5a94069970 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 --2- 172.21.15.32:0/1054842060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a94100db0 0x7f5a9419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 >> 172.21.15.32:0/1054842060 conn(0x7f5a940fb150 msgr2=0x7f5a94103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 shutdown_connections 2024-06-03T01:23:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.316+0000 7f5a9892c700 1 -- 172.21.15.32:0/1054842060 wait complete. 2024-06-03T01:23:13.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 -- 172.21.15.32:0/1343627455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 msgr2=0x7f5eb8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/1343627455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb8101270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009870 tx=0x7f5ea4009b80 comp rx=0 tx=0).stop 2024-06-03T01:23:13.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 -- 172.21.15.32:0/1343627455 shutdown_connections 2024-06-03T01:23:13.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/1343627455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb8101270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/1343627455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 -- 172.21.15.32:0/1343627455 >> 172.21.15.32:0/1343627455 conn(0x7f5eb80fb190 msgr2=0x7f5eb80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.455+0000 7f5ebeff6700 1 -- 172.21.15.32:0/1343627455 shutdown_connections 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 -- 172.21.15.32:0/1343627455 wait complete. 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 Processor -- start 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 -- start start 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8102730 con 0x7f5eb8100e10 2024-06-03T01:23:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5ebeff6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8102870 con 0x7f5eb80ffc10 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.456+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40564/0 (socket says 172.21.15.32:40564) 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 -- 172.21.15.32:0/3978755467 learned_addr learned my addr 172.21.15.32:0/3978755467 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5ebcd92700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 -- 172.21.15.32:0/3978755467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 msgr2=0x7f5eb8103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 -- 172.21.15.32:0/3978755467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4009520 con 0x7f5eb8100e10 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5ebcd92700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb7fff700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009230 tx=0x7f5ea400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea401c070 con 0x7f5eb8100e10 2024-06-03T01:23:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb8102af0 con 0x7f5eb8100e10 2024-06-03T01:23:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ea400fe40 con 0x7f5eb8100e10 2024-06-03T01:23:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.457+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea40173f0 con 0x7f5eb8100e10 2024-06-03T01:23:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.458+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb8102fe0 con 0x7f5eb8100e10 2024-06-03T01:23:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.459+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb8108720 con 0x7f5eb8100e10 2024-06-03T01:23:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.460+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f5ea4020070 con 0x7f5eb8100e10 2024-06-03T01:23:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.460+0000 7f5eb5ffb700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 0x7f5ea8070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.460+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5ea408f0a0 con 0x7f5eb8100e10 2024-06-03T01:23:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.460+0000 7f5ebcd92700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 0x7f5ea8070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.461+0000 7f5ebcd92700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 0x7f5ea8070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8100c70 tx=0x7f5eac009310 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.465+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5ea405c240 con 0x7f5eb8100e10 2024-06-03T01:23:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.705+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eb819ea10 con 0x7f5ea806dfc0 2024-06-03T01:23:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.716+0000 7f5eb5ffb700 1 -- 172.21.15.32:0/3978755467 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5eb819ea10 con 0x7f5ea806dfc0 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (8m) 0s ago 10m 24.5M - 0.25.0 c8568f914cd2 397594517aa3 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (10m) 0s ago 10m 7898k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (9m) 2s ago 9m 7713k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (10m) 0s ago 10m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (9m) 2s ago 9m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (8m) 0s ago 10m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (4m) 0s ago 4m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:23:13.717 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (4m) 0s ago 4m 124M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (4m) 2s ago 4m 14.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (4m) 2s ago 4m 16.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (93s) 0s ago 11m 496M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (23s) 2s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (11m) 0s ago 11m 55.6M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (9m) 2s ago 9m 44.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (10s) 0s ago 10m 3426k - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (6s) 2s ago 9m 5465k - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (8m) 0s ago 8m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (8m) 0s ago 8m 239M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (7m) 0s ago 7m 203M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (7m) 2s ago 7m 266M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (6m) 2s ago 6m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (6m) 2s ago 6m 228M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (29s) 0s ago 9m 52.0M - 2.43.0 a07b618ecd1d 4cd419f03eaf 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 msgr2=0x7f5ea8070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 0x7f5ea8070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8100c70 tx=0x7f5eac009310 comp rx=0 tx=0).stop 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 msgr2=0x7f5eb81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009230 tx=0x7f5ea400f740 comp rx=0 tx=0).stop 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 shutdown_connections 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f5ea806dfc0 0x7f5ea8070480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb80ffc10 0x7f5eb8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 --2- 172.21.15.32:0/3978755467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5eb8100e10 0x7f5eb81021f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 >> 172.21.15.32:0/3978755467 conn(0x7f5eb80fb190 msgr2=0x7f5eb8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 shutdown_connections 2024-06-03T01:23:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.719+0000 7f5ebeff6700 1 -- 172.21.15.32:0/3978755467 wait complete. 2024-06-03T01:23:13.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.828+0000 7f0f2afc6700 1 -- 172.21.15.32:0/3417413128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 msgr2=0x7f0f24103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.828+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/3417413128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24103b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009910 tx=0x7f0f18009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.828+0000 7f0f2afc6700 1 -- 172.21.15.32:0/3417413128 shutdown_connections 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.828+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/3417413128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 0x7f0f241064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.828+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/3417413128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24103b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- 172.21.15.32:0/3417413128 >> 172.21.15.32:0/3417413128 conn(0x7f0f240fb150 msgr2=0x7f0f240fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- 172.21.15.32:0/3417413128 shutdown_connections 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- 172.21.15.32:0/3417413128 wait complete. 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 Processor -- start 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- start start 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 0x7f0f240ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240ffc90 con 0x7f0f241017a0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.829+0000 7f0f2afc6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240ffdd0 con 0x7f0f241040d0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40584/0 (socket says 172.21.15.32:40584) 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 -- 172.21.15.32:0/2811338019 learned_addr learned my addr 172.21.15.32:0/2811338019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f23fff700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 0x7f0f240ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 -- 172.21.15.32:0/2811338019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 msgr2=0x7f0f240ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 0x7f0f240ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 -- 172.21.15.32:0/2811338019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180095f0 con 0x7f0f241017a0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f28d62700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006010 tx=0x7f0f1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1801d070 con 0x7f0f241017a0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f24100050 con 0x7f0f241017a0 2024-06-03T01:23:13.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f1800f460 con 0x7f0f241017a0 2024-06-03T01:23:13.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.830+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f18021640 con 0x7f0f241017a0 2024-06-03T01:23:13.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.831+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f24100540 con 0x7f0f241017a0 2024-06-03T01:23:13.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.832+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f241906a0 con 0x7f0f241017a0 2024-06-03T01:23:13.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.832+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f0f1800fab0 con 0x7f0f241017a0 2024-06-03T01:23:13.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.833+0000 7f0f21ffb700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 0x7f0f14070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:13.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.833+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0f1808f270 con 0x7f0f241017a0 2024-06-03T01:23:13.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.833+0000 7f0f23fff700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 0x7f0f14070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:13.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.833+0000 7f0f23fff700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 0x7f0f14070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009a10 tx=0x7f0f10009190 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:13.837+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f0f1805c390 con 0x7f0f241017a0 2024-06-03T01:23:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:13 smithi032 ceph-mon[29360]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:13 smithi032 ceph-mon[29360]: pgmap v32: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:13 smithi032 ceph-mon[29360]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.148+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f2402cc70 con 0x7f0f241017a0 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.149+0000 7f0f21ffb700 1 -- 172.21.15.32:0/2811338019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f0f1807c020 con 0x7f0f241017a0 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:23:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:23:14.150 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:23:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 msgr2=0x7f0f14070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 0x7f0f14070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009a10 tx=0x7f0f10009190 comp rx=0 tx=0).stop 2024-06-03T01:23:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 msgr2=0x7f0f24101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006010 tx=0x7f0f1800bd70 comp rx=0 tx=0).stop 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 shutdown_connections 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0f1406dfc0 0x7f0f14070480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f241017a0 0x7f0f24101100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 --2- 172.21.15.32:0/2811338019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f241040d0 0x7f0f240ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 >> 172.21.15.32:0/2811338019 conn(0x7f0f240fb150 msgr2=0x7f0f240fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 shutdown_connections 2024-06-03T01:23:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.151+0000 7f0f2afc6700 1 -- 172.21.15.32:0/2811338019 wait complete. 2024-06-03T01:23:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:13 smithi044 ceph-mon[36956]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:13 smithi044 ceph-mon[36956]: pgmap v32: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:13 smithi044 ceph-mon[36956]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 -- 172.21.15.32:0/1280463997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41040d0 msgr2=0x7ff9a41064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 --2- 172.21.15.32:0/1280463997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41040d0 0x7ff9a41064c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff994009910 tx=0x7ff994009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 -- 172.21.15.32:0/1280463997 shutdown_connections 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 --2- 172.21.15.32:0/1280463997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41040d0 0x7ff9a41064c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 --2- 172.21.15.32:0/1280463997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41017a0 0x7ff9a4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 -- 172.21.15.32:0/1280463997 >> 172.21.15.32:0/1280463997 conn(0x7ff9a40fb190 msgr2=0x7ff9a40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 -- 172.21.15.32:0/1280463997 shutdown_connections 2024-06-03T01:23:14.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.264+0000 7ff9ac665700 1 -- 172.21.15.32:0/1280463997 wait complete. 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 Processor -- start 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 -- start start 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41040d0 0x7ff9a4196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4196fb0 con 0x7ff9a41017a0 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9ac665700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41970f0 con 0x7ff9a41040d0 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9aa401700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9aa401700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40600/0 (socket says 172.21.15.32:40600) 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9aa401700 1 -- 172.21.15.32:0/979836320 learned_addr learned my addr 172.21.15.32:0/979836320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.265+0000 7ff9a9c00700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41040d0 0x7ff9a4196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9aa401700 1 -- 172.21.15.32:0/979836320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41040d0 msgr2=0x7ff9a4196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9aa401700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41040d0 0x7ff9a4196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9aa401700 1 -- 172.21.15.32:0/979836320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9940095f0 con 0x7ff9a41017a0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9aa401700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d8d0 tx=0x7ff9a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a0009940 con 0x7ff9a41017a0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a419bba0 con 0x7ff9a41017a0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0010460 con 0x7ff9a41017a0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a000f5f0 con 0x7ff9a41017a0 2024-06-03T01:23:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.266+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a419c0f0 con 0x7ff9a41017a0 2024-06-03T01:23:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.267+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4190610 con 0x7ff9a41017a0 2024-06-03T01:23:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.268+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7ff9a00105d0 con 0x7ff9a41017a0 2024-06-03T01:23:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.269+0000 7ff99b7fe700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 0x7ff990070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.269+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff9a008de20 con 0x7ff9a41017a0 2024-06-03T01:23:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.269+0000 7ff9a9c00700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 0x7ff990070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.269+0000 7ff9a9c00700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 0x7ff990070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff99400b5c0 tx=0x7ff994005960 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.274+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff9a005af40 con 0x7ff9a41017a0 2024-06-03T01:23:14.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.508+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a40611d0 con 0x7ff99006dfc0 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.509+0000 7ff99b7fe700 1 -- 172.21.15.32:0/979836320 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ff9a40611d0 con 0x7ff99006dfc0 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:23:14.509 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:23:14.510 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:23:14.510 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/2 daemons upgraded", 2024-06-03T01:23:14.510 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading node-exporter daemons", 2024-06-03T01:23:14.510 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:23:14.510 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:23:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 msgr2=0x7ff990070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 0x7ff990070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff99400b5c0 tx=0x7ff994005960 comp rx=0 tx=0).stop 2024-06-03T01:23:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 msgr2=0x7ff9a4196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d8d0 tx=0x7ff9a000dc90 comp rx=0 tx=0).stop 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 shutdown_connections 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ff99006dfc0 0x7ff990070480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a41017a0 0x7ff9a4196450 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 --2- 172.21.15.32:0/979836320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a41040d0 0x7ff9a4196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.511+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 >> 172.21.15.32:0/979836320 conn(0x7ff9a40fb190 msgr2=0x7ff9a40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.512+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 shutdown_connections 2024-06-03T01:23:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:14.512+0000 7ff9ac665700 1 -- 172.21.15.32:0/979836320 wait complete. 2024-06-03T01:23:15.300 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:15 smithi044 ceph-mon[36956]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:15.300 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:15.301 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:15.301 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:15 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2811338019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:15 smithi032 ceph-mon[29360]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:15.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:15 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2811338019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:16.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:16 smithi032 ceph-mon[29360]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:16.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:16 smithi032 ceph-mon[29360]: pgmap v33: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:16.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:16 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:16.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:16 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:16.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:16 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:16 smithi044 ceph-mon[36956]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:16 smithi044 ceph-mon[36956]: pgmap v33: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:16.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:16 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:16.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:16 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:16.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:16 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: pgmap v34: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: pgmap v34: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:18.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:20.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:20 smithi044 ceph-mon[36956]: pgmap v35: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:20.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:20 smithi044 ceph-mon[36956]: Upgrade: Updating prometheus.smithi032 2024-06-03T01:23:20.521 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:20 smithi032 ceph-mon[29360]: pgmap v35: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:20.521 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:20 smithi032 ceph-mon[29360]: Upgrade: Updating prometheus.smithi032 2024-06-03T01:23:21.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:21 smithi044 ceph-mon[36956]: Deploying daemon prometheus.smithi032 on smithi032 2024-06-03T01:23:21.485 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:21 smithi032 ceph-mon[29360]: Deploying daemon prometheus.smithi032 on smithi032 2024-06-03T01:23:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:22 smithi044 ceph-mon[36956]: pgmap v36: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:22.497 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:22 smithi032 ceph-mon[29360]: pgmap v36: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:24.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:24 smithi032 ceph-mon[29360]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:24 smithi044 ceph-mon[36956]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:26 smithi032 ceph-mon[29360]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:26 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:26 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:26 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:26 smithi044 ceph-mon[36956]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:26 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:26 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:26 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:28.574 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:28 smithi032 ceph-mon[29360]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:28.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:28 smithi044 ceph-mon[36956]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:30.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:30 smithi044 ceph-mon[36956]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:30.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:30 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:30 smithi032 ceph-mon[29360]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:30 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:31 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:31 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:31 smithi032 ceph-mon[29360]: pgmap v41: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:31.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:31 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:31.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:31 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:31.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:31 smithi044 ceph-mon[36956]: pgmap v41: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:32.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:32 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:32.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:32 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:32 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:32 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:33 smithi032 ceph-mon[29360]: pgmap v42: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:33.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:33 smithi044 ceph-mon[36956]: pgmap v42: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:36.120 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: pgmap v43: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.121 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:36 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: pgmap v43: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:36.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:36 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:38.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:38 smithi032 ceph-mon[29360]: Upgrade: Updating alertmanager.smithi032 2024-06-03T01:23:38.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:38 smithi032 ceph-mon[29360]: Deploying daemon alertmanager.smithi032 on smithi032 2024-06-03T01:23:38.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:38 smithi032 ceph-mon[29360]: pgmap v44: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:38.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:38 smithi044 ceph-mon[36956]: Upgrade: Updating alertmanager.smithi032 2024-06-03T01:23:38.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:38 smithi044 ceph-mon[36956]: Deploying daemon alertmanager.smithi032 on smithi032 2024-06-03T01:23:38.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:38 smithi044 ceph-mon[36956]: pgmap v44: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:40.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:40 smithi032 ceph-mon[29360]: pgmap v45: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:40.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:40 smithi044 ceph-mon[36956]: pgmap v45: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:42.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:42 smithi032 ceph-mon[29360]: pgmap v46: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:42 smithi044 ceph-mon[36956]: pgmap v46: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:43.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: pgmap v47: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:43 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: pgmap v47: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:43 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 -- 172.21.15.32:0/1066908211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 msgr2=0x7f8d04106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 --2- 172.21.15.32:0/1066908211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04106540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009910 tx=0x7f8cf8009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 -- 172.21.15.32:0/1066908211 shutdown_connections 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 --2- 172.21.15.32:0/1066908211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04106540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 --2- 172.21.15.32:0/1066908211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 0x7f8d04103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 -- 172.21.15.32:0/1066908211 >> 172.21.15.32:0/1066908211 conn(0x7f8d040fb190 msgr2=0x7f8d040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 -- 172.21.15.32:0/1066908211 shutdown_connections 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.629+0000 7f8d0a299700 1 -- 172.21.15.32:0/1066908211 wait complete. 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 Processor -- start 2024-06-03T01:23:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 -- start start 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 0x7f8d04196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04196fb0 con 0x7f8d04104150 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d0a299700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d041970f0 con 0x7f8d04101820 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49952/0 (socket says 172.21.15.32:49952) 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d037fe700 1 -- 172.21.15.32:0/3655687848 learned_addr learned my addr 172.21.15.32:0/3655687848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.630+0000 7f8d03fff700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 0x7f8d04196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d037fe700 1 -- 172.21.15.32:0/3655687848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 msgr2=0x7f8d04196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d037fe700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 0x7f8d04196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d037fe700 1 -- 172.21.15.32:0/3655687848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf80095f0 con 0x7f8d04104150 2024-06-03T01:23:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d037fe700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8000c00 tx=0x7f8cf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf801d070 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0419bb40 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8cf8005dc0 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf80226a0 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.631+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0419c030 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.633+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d04190610 con 0x7f8d04104150 2024-06-03T01:23:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.633+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f8cf80038f0 con 0x7f8d04104150 2024-06-03T01:23:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.634+0000 7f8d017fa700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 0x7f8cec070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.634+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8cf808f3b0 con 0x7f8d04104150 2024-06-03T01:23:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.634+0000 7f8d03fff700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 0x7f8cec070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.634+0000 7f8d03fff700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 0x7f8cec070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009c60 tx=0x7f8cf4009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.639+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f8cf805c430 con 0x7f8d04104150 2024-06-03T01:23:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.878+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d040611d0 con 0x7f8cec06dfc0 2024-06-03T01:23:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.879+0000 7f8d017fa700 1 -- 172.21.15.32:0/3655687848 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f8d040611d0 con 0x7f8cec06dfc0 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 msgr2=0x7f8cec070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 0x7f8cec070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009c60 tx=0x7f8cf4009260 comp rx=0 tx=0).stop 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 msgr2=0x7f8d04196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8000c00 tx=0x7f8cf800f690 comp rx=0 tx=0).stop 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 shutdown_connections 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8cec06dfc0 0x7f8cec070480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d04101820 0x7f8d04196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 --2- 172.21.15.32:0/3655687848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d04104150 0x7f8d04196990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 >> 172.21.15.32:0/3655687848 conn(0x7f8d040fb190 msgr2=0x7f8d040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 shutdown_connections 2024-06-03T01:23:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.881+0000 7f8d0a299700 1 -- 172.21.15.32:0/3655687848 wait complete. 2024-06-03T01:23:44.892 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.992+0000 7f6cc761d700 1 -- 172.21.15.32:0/2626367836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc00ffbb0 msgr2=0x7f6cc00fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.992+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2626367836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc00fffd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009910 tx=0x7f6cb4009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 -- 172.21.15.32:0/2626367836 shutdown_connections 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2626367836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc0100db0 0x7f6cc0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2626367836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc00fffd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 -- 172.21.15.32:0/2626367836 >> 172.21.15.32:0/2626367836 conn(0x7f6cc00fb150 msgr2=0x7f6cc00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 -- 172.21.15.32:0/2626367836 shutdown_connections 2024-06-03T01:23:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 -- 172.21.15.32:0/2626367836 wait complete. 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 Processor -- start 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.993+0000 7f6cc761d700 1 -- start start 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc761d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc761d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc761d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0199230 con 0x7f6cc0100db0 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc761d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0199370 con 0x7f6cc00ffbb0 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 -- 172.21.15.32:0/2984616277 learned_addr learned my addr 172.21.15.32:0/2984616277 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc53b9700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 -- 172.21.15.32:0/2984616277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc00ffbb0 msgr2=0x7f6cc01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 -- 172.21.15.32:0/2984616277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40095f0 con 0x7f6cc0100db0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc4bb8700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c8a0 tx=0x7f6cbc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc00cea0 con 0x7f6cc0100db0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc004830 con 0x7f6cc0100db0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc019de20 con 0x7f6cc0100db0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.994+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc0055b0 con 0x7f6cc0100db0 2024-06-03T01:23:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.995+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc019e340 con 0x7f6cc0100db0 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.995+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc0108710 con 0x7f6cc0100db0 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.996+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f6cbc007ad0 con 0x7f6cc0100db0 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.996+0000 7f6cb27fc700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 0x7f6cac070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.997+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6cbc08dba0 con 0x7f6cc0100db0 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.997+0000 7f6cc53b9700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 0x7f6cac070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:44.997+0000 7f6cc53b9700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 0x7f6cac070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb400b580 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.001+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f6cbc05acc0 con 0x7f6cc0100db0 2024-06-03T01:23:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.236+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc0069520 con 0x7f6cac06dfc0 2024-06-03T01:23:45.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.237+0000 7f6cb27fc700 1 -- 172.21.15.32:0/2984616277 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f6cc0069520 con 0x7f6cac06dfc0 2024-06-03T01:23:45.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 msgr2=0x7f6cac070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 0x7f6cac070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb400b580 comp rx=0 tx=0).stop 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 msgr2=0x7f6cc0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c8a0 tx=0x7f6cbc00cc60 comp rx=0 tx=0).stop 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 shutdown_connections 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6cac06dfc0 0x7f6cac070480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cc00ffbb0 0x7f6cc01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 --2- 172.21.15.32:0/2984616277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cc0100db0 0x7f6cc0198c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.239+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 >> 172.21.15.32:0/2984616277 conn(0x7f6cc00fb150 msgr2=0x7f6cc0068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.240+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 shutdown_connections 2024-06-03T01:23:45.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.240+0000 7f6cc761d700 1 -- 172.21.15.32:0/2984616277 wait complete. 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- 172.21.15.32:0/3918161953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c100dc0 msgr2=0x7fd61c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 --2- 172.21.15.32:0/3918161953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c100dc0 0x7fd61c101220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd610009910 tx=0x7fd610009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- 172.21.15.32:0/3918161953 shutdown_connections 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 --2- 172.21.15.32:0/3918161953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c100dc0 0x7fd61c101220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 --2- 172.21.15.32:0/3918161953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- 172.21.15.32:0/3918161953 >> 172.21.15.32:0/3918161953 conn(0x7fd61c0fb160 msgr2=0x7fd61c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- 172.21.15.32:0/3918161953 shutdown_connections 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- 172.21.15.32:0/3918161953 wait complete. 2024-06-03T01:23:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 Processor -- start 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.348+0000 7fd622847700 1 -- start start 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c100dc0 0x7fd61c195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c196220 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c196360 con 0x7fd61c100dc0 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49994/0 (socket says 172.21.15.32:49994) 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 -- 172.21.15.32:0/854000555 learned_addr learned my addr 172.21.15.32:0/854000555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61b7fe700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c100dc0 0x7fd61c195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 -- 172.21.15.32:0/854000555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c100dc0 msgr2=0x7fd61c195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c100dc0 0x7fd61c195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 -- 172.21.15.32:0/854000555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100095f0 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd61bfff700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007b50 tx=0x7fd60c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c011070 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd60c00c2e0 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c00ac40 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c068e70 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.349+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c0693c0 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.350+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c068af0 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.351+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7fd60c00e710 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.351+0000 7fd6197fa700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 0x7fd604070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.352+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd60c08f460 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.352+0000 7fd61b7fe700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 0x7fd604070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.352+0000 7fd61b7fe700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 0x7fd604070480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd6100059d0 tx=0x7fd610005920 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.356+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fd60c05c560 con 0x7fd61c0ffbc0 2024-06-03T01:23:45.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.581+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd61c18f4c0 con 0x7fd60406dfc0 2024-06-03T01:23:45.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.593+0000 7fd6197fa700 1 -- 172.21.15.32:0/854000555 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd61c18f4c0 con 0x7fd60406dfc0 2024-06-03T01:23:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:23:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 starting - - - - 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (11m) 15s ago 11m 7918k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (10m) 34s ago 10m 7713k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (11m) 15s ago 11m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (10m) 34s ago 10m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (9m) 15s ago 10m 86.9M - 9.4.7 954c08fa6188 9d4e1e10cdaa 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (5m) 15s ago 5m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (5m) 15s ago 5m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (5m) 34s ago 5m 14.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:23:45.594 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (5m) 34s ago 5m 16.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (2m) 15s ago 12m 499M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (55s) 34s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (12m) 15s ago 12m 56.1M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (9m) 34s ago 9m 44.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (42s) 15s ago 11m 13.5M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (38s) 34s ago 9m 5465k - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (9m) 15s ago 9m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (8m) 15s ago 8m 239M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (8m) 15s ago 8m 203M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (7m) 34s ago 7m 266M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (7m) 34s ago 7m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (6m) 34s ago 6m 228M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:23:45.595 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (20s) 15s ago 10m 46.0M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:23:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 msgr2=0x7fd604070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 0x7fd604070480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd6100059d0 tx=0x7fd610005920 comp rx=0 tx=0).stop 2024-06-03T01:23:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 msgr2=0x7fd61c1956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007b50 tx=0x7fd60c007f10 comp rx=0 tx=0).stop 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 shutdown_connections 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd60406dfc0 0x7fd604070480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c0ffbc0 0x7fd61c1956c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 --2- 172.21.15.32:0/854000555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd61c100dc0 0x7fd61c195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 >> 172.21.15.32:0/854000555 conn(0x7fd61c0fb160 msgr2=0x7fd61c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 shutdown_connections 2024-06-03T01:23:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.596+0000 7fd622847700 1 -- 172.21.15.32:0/854000555 wait complete. 2024-06-03T01:23:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 -- 172.21.15.32:0/1657505140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3538102000 msgr2=0x7f3538102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 --2- 172.21.15.32:0/1657505140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3538102000 0x7f3538102460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3520009910 tx=0x7f3520009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 -- 172.21.15.32:0/1657505140 shutdown_connections 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 --2- 172.21.15.32:0/1657505140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3538102000 0x7f3538102460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 --2- 172.21.15.32:0/1657505140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35380fef80 0x7f35380ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 -- 172.21.15.32:0/1657505140 >> 172.21.15.32:0/1657505140 conn(0x7f35380fa8c0 msgr2=0x7f35380fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 -- 172.21.15.32:0/1657505140 shutdown_connections 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.703+0000 7f353e8fb700 1 -- 172.21.15.32:0/1657505140 wait complete. 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 Processor -- start 2024-06-03T01:23:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 -- start start 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 0x7f35381969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538197000 con 0x7f35380fef80 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f353e8fb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538197140 con 0x7f3538102000 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50012/0 (socket says 172.21.15.32:50012) 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 -- 172.21.15.32:0/791855198 learned_addr learned my addr 172.21.15.32:0/791855198 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f352ffff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 0x7f35381969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 -- 172.21.15.32:0/791855198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 msgr2=0x7f35381969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 0x7f35381969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f3537fff700 1 -- 172.21.15.32:0/791855198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200095f0 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.704+0000 7f352ffff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 0x7f35381969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f3537fff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f352800d900 tx=0x7f352800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35280041f0 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3528004350 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353819bbf0 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3528003da0 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.705+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353819c140 con 0x7f35380fef80 2024-06-03T01:23:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.706+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3538109800 con 0x7f35380fef80 2024-06-03T01:23:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.706+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f3528009730 con 0x7f35380fef80 2024-06-03T01:23:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.707+0000 7f3535ffb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 0x7f3524070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.707+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f352808ded0 con 0x7f35380fef80 2024-06-03T01:23:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.707+0000 7f352ffff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 0x7f3524070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.707+0000 7f352ffff700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 0x7f3524070480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f352000b5c0 tx=0x7f3520005e20 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:45.712+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f352805b070 con 0x7f35380fef80 2024-06-03T01:23:45.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:45 smithi032 ceph-mon[29360]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:45.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:45 smithi032 ceph-mon[29360]: pgmap v48: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:45.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:45.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:45 smithi032 ceph-mon[29360]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.025+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f353819c7b0 con 0x7f35380fef80 2024-06-03T01:23:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.025+0000 7f3535ffb700 1 -- 172.21.15.32:0/791855198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f352805ac00 con 0x7f35380fef80 2024-06-03T01:23:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:23:46.027 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 msgr2=0x7f3524070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 0x7f3524070480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f352000b5c0 tx=0x7f3520005e20 comp rx=0 tx=0).stop 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 msgr2=0x7f35381964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f352800d900 tx=0x7f352800dcc0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 shutdown_connections 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f352406dfc0 0x7f3524070480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35380fef80 0x7f35381964a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 --2- 172.21.15.32:0/791855198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3538102000 0x7f35381969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.027+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 >> 172.21.15.32:0/791855198 conn(0x7f35380fa8c0 msgr2=0x7f3538105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.028+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 shutdown_connections 2024-06-03T01:23:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.028+0000 7f353e8fb700 1 -- 172.21.15.32:0/791855198 wait complete. 2024-06-03T01:23:46.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 -- 172.21.15.32:0/3769703332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 msgr2=0x7faf481023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3769703332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf481023a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009910 tx=0x7faf3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 -- 172.21.15.32:0/3769703332 shutdown_connections 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3769703332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf481023a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3769703332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 0x7faf480ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 -- 172.21.15.32:0/3769703332 >> 172.21.15.32:0/3769703332 conn(0x7faf480fa8c0 msgr2=0x7faf480fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 -- 172.21.15.32:0/3769703332 shutdown_connections 2024-06-03T01:23:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.136+0000 7faf4d00e700 1 -- 172.21.15.32:0/3769703332 wait complete. 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.137+0000 7faf4d00e700 1 Processor -- start 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.137+0000 7faf4d00e700 1 -- start start 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4d00e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 0x7faf481964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4d00e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4d00e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48197030 con 0x7faf48101f40 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4d00e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48197170 con 0x7faf480fef60 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50038/0 (socket says 172.21.15.32:50038) 2024-06-03T01:23:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 -- 172.21.15.32:0/3148883322 learned_addr learned my addr 172.21.15.32:0/3148883322 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf46d9d700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 0x7faf481964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 -- 172.21.15.32:0/3148883322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 msgr2=0x7faf481964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 0x7faf481964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 -- 172.21.15.32:0/3148883322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0095f0 con 0x7faf48101f40 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4659c700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bd70 tx=0x7faf3c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c01d070 con 0x7faf48101f40 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faf3c005dc0 con 0x7faf48101f40 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4819bbc0 con 0x7faf48101f40 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.138+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c021410 con 0x7faf48101f40 2024-06-03T01:23:46.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.139+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4819c0b0 con 0x7faf48101f40 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.140+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf48109800 con 0x7faf48101f40 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.140+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7faf3c003a60 con 0x7faf48101f40 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.140+0000 7faf2ffff700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 0x7faf30070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.141+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faf3c08f140 con 0x7faf48101f40 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.141+0000 7faf46d9d700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 0x7faf30070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.141+0000 7faf46d9d700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 0x7faf30070480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf48068c10 tx=0x7faf38007400 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.146+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7faf3c05c260 con 0x7faf48101f40 2024-06-03T01:23:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:45 smithi044 ceph-mon[36956]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:46.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:45 smithi044 ceph-mon[36956]: pgmap v48: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:23:46.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:23:46.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:45 smithi044 ceph-mon[36956]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.382+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf48106890 con 0x7faf3006dfc0 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.383+0000 7faf2ffff700 1 -- 172.21.15.32:0/3148883322 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7faf48106890 con 0x7faf3006dfc0 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:23:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:23:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/2 daemons upgraded", 2024-06-03T01:23:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading alertmanager daemons", 2024-06-03T01:23:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:23:46.384 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:23:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 msgr2=0x7faf30070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 0x7faf30070480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf48068c10 tx=0x7faf38007400 comp rx=0 tx=0).stop 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 msgr2=0x7faf48196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bd70 tx=0x7faf3c00bda0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 shutdown_connections 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7faf3006dfc0 0x7faf30070480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf480fef60 0x7faf481964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 --2- 172.21.15.32:0/3148883322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf48101f40 0x7faf48196a10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.385+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 >> 172.21.15.32:0/3148883322 conn(0x7faf480fa8c0 msgr2=0x7faf48105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.386+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 shutdown_connections 2024-06-03T01:23:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:23:46.386+0000 7faf4d00e700 1 -- 172.21.15.32:0/3148883322 wait complete. 2024-06-03T01:23:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:46 smithi032 ceph-mon[29360]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:46 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/791855198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:46 smithi032 ceph-mon[29360]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:46 smithi044 ceph-mon[36956]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:46 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/791855198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:46 smithi044 ceph-mon[36956]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:23:48.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:47 smithi032 ceph-mon[29360]: mgrmap e32: smithi032.kfcfaa(active, since 91s), standbys: smithi044.wszjlw 2024-06-03T01:23:48.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:47 smithi032 ceph-mon[29360]: pgmap v49: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:47 smithi044 ceph-mon[36956]: mgrmap e32: smithi032.kfcfaa(active, since 91s), standbys: smithi044.wszjlw 2024-06-03T01:23:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:47 smithi044 ceph-mon[36956]: pgmap v49: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:49.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:48 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:49.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:48 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:48 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:48 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:49 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:49 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:49 smithi032 ceph-mon[29360]: pgmap v50: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:50.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:49 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:50.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:49 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:50.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:49 smithi044 ceph-mon[36956]: pgmap v50: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:52.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:52 smithi032 ceph-mon[29360]: pgmap v51: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:52 smithi044 ceph-mon[36956]: pgmap v51: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:53.707 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:53 smithi032 ceph-mon[29360]: pgmap v52: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:23:53.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:53 smithi044 ceph-mon[36956]: pgmap v52: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:54.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:54 smithi044 ceph-mon[36956]: Upgrade: Updating grafana.smithi032 2024-06-03T01:23:54.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:54 smithi044 ceph-mon[36956]: Deploying daemon grafana.smithi032 on smithi032 2024-06-03T01:23:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:54 smithi032 ceph-mon[29360]: Upgrade: Updating grafana.smithi032 2024-06-03T01:23:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:54 smithi032 ceph-mon[29360]: Deploying daemon grafana.smithi032 on smithi032 2024-06-03T01:23:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:55 smithi032 ceph-mon[29360]: pgmap v53: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:55 smithi044 ceph-mon[36956]: pgmap v53: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:23:58 smithi032 ceph-mon[29360]: pgmap v54: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:23:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:23:58 smithi044 ceph-mon[36956]: pgmap v54: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-03T01:24:00.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:00 smithi044 ceph-mon[36956]: pgmap v55: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:00.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:00 smithi032 ceph-mon[29360]: pgmap v55: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:01.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:01 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:01.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:01 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:02.247 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:02 smithi032 ceph-mon[29360]: pgmap v56: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:02 smithi044 ceph-mon[36956]: pgmap v56: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:04.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:04 smithi032 ceph-mon[29360]: pgmap v57: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:04 smithi044 ceph-mon[36956]: pgmap v57: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:05.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:05 smithi032 ceph-mon[29360]: pgmap v58: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:06.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:05 smithi044 ceph-mon[36956]: pgmap v58: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:07.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:06 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:07.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:06 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:07.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:06 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:07.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:06 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:07.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:06 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:07.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:06 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:08.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:07 smithi032 ceph-mon[29360]: pgmap v59: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:07 smithi044 ceph-mon[36956]: pgmap v59: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:10 smithi032 ceph-mon[29360]: pgmap v60: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:10 smithi044 ceph-mon[36956]: pgmap v60: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:12.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:12 smithi032 ceph-mon[29360]: pgmap v61: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:12.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:12.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:12 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:12.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:12 smithi044 ceph-mon[36956]: pgmap v61: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:12.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:12.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:12 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:13.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:13 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:13.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:13 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:13.749 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:13 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:13.749 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:13 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:14 smithi032 ceph-mon[29360]: pgmap v62: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:14.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:14 smithi044 ceph-mon[36956]: pgmap v62: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:16.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:15 smithi032 ceph-mon[29360]: pgmap v63: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:16.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:16.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:15 smithi044 ceph-mon[36956]: pgmap v63: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 -- 172.21.15.32:0/3087186004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0101810 msgr2=0x7f0cd0103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3087186004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0101810 0x7f0cd0103c00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009910 tx=0x7f0cc0009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 -- 172.21.15.32:0/3087186004 shutdown_connections 2024-06-03T01:24:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3087186004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0104140 0x7f0cd0106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3087186004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0101810 0x7f0cd0103c00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 -- 172.21.15.32:0/3087186004 >> 172.21.15.32:0/3087186004 conn(0x7f0cd00fb160 msgr2=0x7f0cd00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.510+0000 7f0cd5561700 1 -- 172.21.15.32:0/3087186004 shutdown_connections 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.511+0000 7f0cd5561700 1 -- 172.21.15.32:0/3087186004 wait complete. 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.511+0000 7f0cd5561700 1 Processor -- start 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.511+0000 7f0cd5561700 1 -- start start 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cd5561700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0101810 0x7f0cd01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cd5561700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cd5561700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0197300 con 0x7f0cd0104140 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cd5561700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0197440 con 0x7f0cd0101810 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60030/0 (socket says 172.21.15.32:60030) 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 -- 172.21.15.32:0/3543224820 learned_addr learned my addr 172.21.15.32:0/3543224820 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cceffd700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0101810 0x7f0cd01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 -- 172.21.15.32:0/3543224820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0101810 msgr2=0x7f0cd01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0101810 0x7f0cd01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 -- 172.21.15.32:0/3543224820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00095f0 con 0x7f0cd0104140 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.512+0000 7f0cce7fc700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800d8d0 tx=0x7f0cb800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.513+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb80098a0 con 0x7f0cd0104140 2024-06-03T01:24:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.513+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cb8010460 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.513+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd019bef0 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.513+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb800f5f0 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.513+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd019c410 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.514+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd0190630 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.514+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f0cb8009a00 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.515+0000 7f0cc7fff700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 0x7f0cbc070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.515+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0cb808da70 con 0x7f0cd0104140 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.515+0000 7f0cceffd700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 0x7f0cbc070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.516+0000 7f0cceffd700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 0x7f0cbc070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0006010 tx=0x7f0cc000b540 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:16.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.520+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f0cb805adb0 con 0x7f0cd0104140 2024-06-03T01:24:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.757+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cd002d050 con 0x7f0cbc06de80 2024-06-03T01:24:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.758+0000 7f0cc7fff700 1 -- 172.21.15.32:0/3543224820 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f0cd002d050 con 0x7f0cbc06de80 2024-06-03T01:24:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 msgr2=0x7f0cbc070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 0x7f0cbc070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0006010 tx=0x7f0cc000b540 comp rx=0 tx=0).stop 2024-06-03T01:24:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 msgr2=0x7f0cd0196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800d8d0 tx=0x7f0cb800dbe0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 shutdown_connections 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f0cbc06de80 0x7f0cbc070340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd0101810 0x7f0cd01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 --2- 172.21.15.32:0/3543224820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd0104140 0x7f0cd0196ce0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 >> 172.21.15.32:0/3543224820 conn(0x7f0cd00fb160 msgr2=0x7f0cd00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 shutdown_connections 2024-06-03T01:24:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.760+0000 7f0cd5561700 1 -- 172.21.15.32:0/3543224820 wait complete. 2024-06-03T01:24:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:24:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 -- 172.21.15.32:0/2252078978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 msgr2=0x7f94a4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 --2- 172.21.15.32:0/2252078978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4103b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009910 tx=0x7f94a0009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 -- 172.21.15.32:0/2252078978 shutdown_connections 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 --2- 172.21.15.32:0/2252078978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 0x7f94a41064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 --2- 172.21.15.32:0/2252078978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4103b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 -- 172.21.15.32:0/2252078978 >> 172.21.15.32:0/2252078978 conn(0x7f94a40fb190 msgr2=0x7f94a40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 -- 172.21.15.32:0/2252078978 shutdown_connections 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.869+0000 7f94ac357700 1 -- 172.21.15.32:0/2252078978 wait complete. 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 Processor -- start 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 -- start start 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 0x7f94a41969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4196fc0 con 0x7f94a41017a0 2024-06-03T01:24:16.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94ac357700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4197100 con 0x7f94a41040d0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94aa0f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94aa0f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50346/0 (socket says 172.21.15.32:50346) 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94aa0f3700 1 -- 172.21.15.32:0/577974842 learned_addr learned my addr 172.21.15.32:0/577974842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.870+0000 7f94a98f2700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 0x7f94a41969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94aa0f3700 1 -- 172.21.15.32:0/577974842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 msgr2=0x7f94a41969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94aa0f3700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 0x7f94a41969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94aa0f3700 1 -- 172.21.15.32:0/577974842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00095f0 con 0x7f94a41017a0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94aa0f3700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a001d070 con 0x7f94a41017a0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94a0005dc0 con 0x7f94a41017a0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a419bb50 con 0x7f94a41017a0 2024-06-03T01:24:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a00226a0 con 0x7f94a41017a0 2024-06-03T01:24:16.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.871+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a419c040 con 0x7f94a41017a0 2024-06-03T01:24:16.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.872+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a4190610 con 0x7f94a41017a0 2024-06-03T01:24:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.872+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f94a00038f0 con 0x7f94a41017a0 2024-06-03T01:24:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.873+0000 7f949b7fe700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 0x7f9490070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.873+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f94a008ef90 con 0x7f94a41017a0 2024-06-03T01:24:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.873+0000 7f94a98f2700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 0x7f9490070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:16.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.874+0000 7f94a98f2700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 0x7f9490070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9494009c60 tx=0x7f9494009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:16.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:16.878+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f94a005c350 con 0x7f94a41017a0 2024-06-03T01:24:17.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.107+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a40611d0 con 0x7f949006de80 2024-06-03T01:24:17.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.109+0000 7f949b7fe700 1 -- 172.21.15.32:0/577974842 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f94a40611d0 con 0x7f949006de80 2024-06-03T01:24:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 msgr2=0x7f9490070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 0x7f9490070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9494009c60 tx=0x7f9494009260 comp rx=0 tx=0).stop 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 msgr2=0x7f94a4196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a000f690 comp rx=0 tx=0).stop 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 shutdown_connections 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f949006de80 0x7f9490070340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a41017a0 0x7f94a4196460 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 --2- 172.21.15.32:0/577974842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a41040d0 0x7f94a41969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 >> 172.21.15.32:0/577974842 conn(0x7f94a40fb190 msgr2=0x7f94a40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 shutdown_connections 2024-06-03T01:24:17.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.111+0000 7f94ac357700 1 -- 172.21.15.32:0/577974842 wait complete. 2024-06-03T01:24:17.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/467910313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 msgr2=0x7f6c88101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/467910313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88101270 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009910 tx=0x7f6c70009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/467910313 shutdown_connections 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/467910313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88101270 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/467910313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 0x7f6c881004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/467910313 >> 172.21.15.32:0/467910313 conn(0x7f6c880fa8b0 msgr2=0x7f6c880fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/467910313 shutdown_connections 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.219+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/467910313 wait complete. 2024-06-03T01:24:17.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 Processor -- start 2024-06-03T01:24:17.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 -- start start 2024-06-03T01:24:17.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 0x7f6c881964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88197050 con 0x7f6c88100df0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c8d3e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88197190 con 0x7f6c881000d0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c867fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50364/0 (socket says 172.21.15.32:50364) 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.220+0000 7f6c867fc700 1 -- 172.21.15.32:0/503089564 learned_addr learned my addr 172.21.15.32:0/503089564 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c86ffd700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 0x7f6c881964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c867fc700 1 -- 172.21.15.32:0/503089564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 msgr2=0x7f6c881964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c867fc700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 0x7f6c881964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c867fc700 1 -- 172.21.15.32:0/503089564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700095f0 con 0x7f6c88100df0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c867fc700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000bda0 tx=0x7f6c7000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7001d070 con 0x7f6c88100df0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c7000f460 con 0x7f6c88100df0 2024-06-03T01:24:17.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8819bbe0 con 0x7f6c88100df0 2024-06-03T01:24:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c70021640 con 0x7f6c88100df0 2024-06-03T01:24:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.221+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8819c0d0 con 0x7f6c88100df0 2024-06-03T01:24:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.222+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c88108740 con 0x7f6c88100df0 2024-06-03T01:24:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.223+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f6c7000f5d0 con 0x7f6c88100df0 2024-06-03T01:24:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.223+0000 7f6c7ffff700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 0x7f6c74070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.223+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6c7008ee40 con 0x7f6c88100df0 2024-06-03T01:24:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.223+0000 7f6c86ffd700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 0x7f6c74070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.224+0000 7f6c86ffd700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 0x7f6c74070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c88068a00 tx=0x7f6c7800bef0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.228+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f6c7005c180 con 0x7f6c88100df0 2024-06-03T01:24:17.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.446+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c880ff340 con 0x7f6c7406de80 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.457+0000 7f6c7ffff700 1 -- 172.21.15.32:0/503089564 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6c880ff340 con 0x7f6c7406de80 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (35s) 6s ago 11m 17.2M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (11m) 6s ago 11m 7972k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (10m) 66s ago 10m 7713k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (11m) 6s ago 11m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (10m) 66s ago 10m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (11s) 6s ago 11m 52.1M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (6m) 6s ago 6m 125M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (5m) 6s ago 5m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (5m) 66s ago 5m 14.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:24:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (5m) 66s ago 5m 16.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (2m) 6s ago 12m 507M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (87s) 66s ago 10m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (12m) 6s ago 13m 57.7M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (10m) 66s ago 10m 44.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (74s) 6s ago 11m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (70s) 66s ago 10m 5465k - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (9m) 6s ago 9m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (9m) 6s ago 9m 239M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (8m) 6s ago 8m 202M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (8m) 66s ago 8m 266M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (7m) 66s ago 7m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (7m) 66s ago 7m 228M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:24:17.458 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (52s) 6s ago 10m 60.2M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:24:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 msgr2=0x7f6c74070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 0x7f6c74070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c88068a00 tx=0x7f6c7800bef0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 msgr2=0x7f6c88196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000bda0 tx=0x7f6c7000be80 comp rx=0 tx=0).stop 2024-06-03T01:24:17.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 shutdown_connections 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f6c7406de80 0x7f6c74070340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c881000d0 0x7f6c881964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 --2- 172.21.15.32:0/503089564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c88100df0 0x7f6c88196a30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 >> 172.21.15.32:0/503089564 conn(0x7f6c880fa8b0 msgr2=0x7f6c880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 shutdown_connections 2024-06-03T01:24:17.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.459+0000 7f6c8d3e8700 1 -- 172.21.15.32:0/503089564 wait complete. 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 -- 172.21.15.32:0/4153645417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38100dc0 msgr2=0x7f3f38101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/4153645417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38100dc0 0x7f3f38101220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009910 tx=0x7f3f2c009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 -- 172.21.15.32:0/4153645417 shutdown_connections 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/4153645417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38100dc0 0x7f3f38101220 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/4153645417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f380ffbc0 0x7f3f380fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 -- 172.21.15.32:0/4153645417 >> 172.21.15.32:0/4153645417 conn(0x7f3f380fb160 msgr2=0x7f3f380fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 -- 172.21.15.32:0/4153645417 shutdown_connections 2024-06-03T01:24:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 -- 172.21.15.32:0/4153645417 wait complete. 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.566+0000 7f3f3dd27700 1 Processor -- start 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f3dd27700 1 -- start start 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f3dd27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f3dd27700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f38100dc0 0x7f3f381021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f3dd27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381026e0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f3dd27700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38102820 con 0x7f3f38100dc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50372/0 (socket says 172.21.15.32:50372) 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 -- 172.21.15.32:0/707364317 learned_addr learned my addr 172.21.15.32:0/707364317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f36ffd700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f38100dc0 0x7f3f381021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 -- 172.21.15.32:0/707364317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f38100dc0 msgr2=0x7f3f381021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f38100dc0 0x7f3f381021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.567+0000 7f3f377fe700 1 -- 172.21.15.32:0/707364317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0095f0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f377fe700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c8f0 tx=0x7f3f2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f28007a70 con 0x7f3f380ffbc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f38102b00 con 0x7f3f380ffbc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f28007bd0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f28018630 con 0x7f3f380ffbc0 2024-06-03T01:24:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.568+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3819e830 con 0x7f3f380ffbc0 2024-06-03T01:24:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.569+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38068af0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.569+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f3f2800f450 con 0x7f3f380ffbc0 2024-06-03T01:24:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.570+0000 7f3f34ff9700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 0x7f3f20070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.570+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3f2808dc20 con 0x7f3f380ffbc0 2024-06-03T01:24:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.570+0000 7f3f36ffd700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 0x7f3f20070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.570+0000 7f3f36ffd700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 0x7f3f20070340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0059d0 tx=0x7f3f2c005920 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.574+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f3f2805b010 con 0x7f3f380ffbc0 2024-06-03T01:24:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.872+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f3819e9c0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.873+0000 7f3f34ff9700 1 -- 172.21.15.32:0/707364317 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f3f2805aba0 con 0x7f3f380ffbc0 2024-06-03T01:24:17.873 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:24:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:24:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:24:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:24:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:24:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:24:17.875 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:24:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 msgr2=0x7f3f20070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 0x7f3f20070340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0059d0 tx=0x7f3f2c005920 comp rx=0 tx=0).stop 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 msgr2=0x7f3f38103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c8f0 tx=0x7f3f2800ccb0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 shutdown_connections 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f3f2006de80 0x7f3f20070340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f380ffbc0 0x7f3f38103b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 --2- 172.21.15.32:0/707364317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f38100dc0 0x7f3f381021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 >> 172.21.15.32:0/707364317 conn(0x7f3f380fb160 msgr2=0x7f3f38068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 shutdown_connections 2024-06-03T01:24:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.875+0000 7f3f3dd27700 1 -- 172.21.15.32:0/707364317 wait complete. 2024-06-03T01:24:17.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 -- 172.21.15.32:0/970871271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4104080 msgr2=0x7fabf4106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/970871271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4104080 0x7fabf4106470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009910 tx=0x7fabe8009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:17.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 -- 172.21.15.32:0/970871271 shutdown_connections 2024-06-03T01:24:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/970871271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4104080 0x7fabf4106470 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/970871271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4101750 0x7fabf4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 -- 172.21.15.32:0/970871271 >> 172.21.15.32:0/970871271 conn(0x7fabf40fb160 msgr2=0x7fabf40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 -- 172.21.15.32:0/970871271 shutdown_connections 2024-06-03T01:24:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.984+0000 7fabfc4ad700 1 -- 172.21.15.32:0/970871271 wait complete. 2024-06-03T01:24:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 Processor -- start 2024-06-03T01:24:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 -- start start 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4104080 0x7fabf4196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4197300 con 0x7fabf4101750 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfc4ad700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4197440 con 0x7fabf4104080 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfa249700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfa249700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50378/0 (socket says 172.21.15.32:50378) 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabfa249700 1 -- 172.21.15.32:0/1019561153 learned_addr learned my addr 172.21.15.32:0/1019561153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.985+0000 7fabf9a48700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4104080 0x7fabf4196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfa249700 1 -- 172.21.15.32:0/1019561153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4104080 msgr2=0x7fabf4196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfa249700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4104080 0x7fabf4196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfa249700 1 -- 172.21.15.32:0/1019561153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80095f0 con 0x7fabf4101750 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfa249700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d8d0 tx=0x7fabf000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf00098a0 con 0x7fabf4101750 2024-06-03T01:24:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fabf0010460 con 0x7fabf4101750 2024-06-03T01:24:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf419bef0 con 0x7fabf4101750 2024-06-03T01:24:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf000f5f0 con 0x7fabf4101750 2024-06-03T01:24:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.986+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf419c3b0 con 0x7fabf4101750 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.987+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4190630 con 0x7fabf4101750 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.987+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fabf0009a00 con 0x7fabf4101750 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.988+0000 7fabe77fe700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 0x7fabe0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.988+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fabf008db00 con 0x7fabf4101750 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.988+0000 7fabf9a48700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 0x7fabe0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.989+0000 7fabf9a48700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 0x7fabe0070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabe8006010 tx=0x7fabe800b560 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:17.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:17.993+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fabf005ade0 con 0x7fabf4101750 2024-06-03T01:24:18.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.212+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabf005ade0 con 0x7fabf4101750 2024-06-03T01:24:18.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.229+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf40611d0 con 0x7fabe006de80 2024-06-03T01:24:18.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.230+0000 7fabe77fe700 1 -- 172.21.15.32:0/1019561153 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fabf40611d0 con 0x7fabe006de80 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/2 daemons upgraded", 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading grafana daemons", 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:24:18.231 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 msgr2=0x7fabe0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 0x7fabe0070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabe8006010 tx=0x7fabe800b560 comp rx=0 tx=0).stop 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 msgr2=0x7fabf41967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d8d0 tx=0x7fabf000dbe0 comp rx=0 tx=0).stop 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 shutdown_connections 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fabe006de80 0x7fabe0070340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabf4101750 0x7fabf41967a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 --2- 172.21.15.32:0/1019561153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf4104080 0x7fabf4196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 >> 172.21.15.32:0/1019561153 conn(0x7fabf40fb160 msgr2=0x7fabf40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 shutdown_connections 2024-06-03T01:24:18.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:18.233+0000 7fabfc4ad700 1 -- 172.21.15.32:0/1019561153 wait complete. 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: pgmap v64: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/707364317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:18.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: pgmap v64: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/707364317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:18.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: Upgrade: Finalizing container_image settings 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-03T01:24:19.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-03T01:24:19.338 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: Upgrade: Complete! 2024-06-03T01:24:19.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:19.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: Upgrade: Finalizing container_image settings 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-03T01:24:19.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-03T01:24:19.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: Upgrade: Complete! 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-03T01:24:19.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:19.476 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:20 smithi044 ceph-mon[36956]: pgmap v65: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:20.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:20 smithi032 ceph-mon[29360]: pgmap v65: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:21.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:21 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:21.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:21 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:24:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:22 smithi044 ceph-mon[36956]: pgmap v66: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:22 smithi032 ceph-mon[29360]: pgmap v66: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:24 smithi044 ceph-mon[36956]: pgmap v67: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:24.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:24 smithi032 ceph-mon[29360]: pgmap v67: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:26.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:25 smithi032 ceph-mon[29360]: pgmap v68: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:26.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:25 smithi044 ceph-mon[36956]: pgmap v68: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:28 smithi032 ceph-mon[29360]: pgmap v69: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:28 smithi044 ceph-mon[36956]: pgmap v69: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:30.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:30 smithi044 ceph-mon[36956]: pgmap v70: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:30 smithi032 ceph-mon[29360]: pgmap v70: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:31.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:31 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:31 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:32.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:32 smithi044 ceph-mon[36956]: pgmap v71: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:32 smithi032 ceph-mon[29360]: pgmap v71: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:34.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:34 smithi032 ceph-mon[29360]: pgmap v72: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:34.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:34 smithi044 ceph-mon[36956]: pgmap v72: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:35 smithi032 ceph-mon[29360]: pgmap v73: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:35 smithi044 ceph-mon[36956]: pgmap v73: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:38 smithi032 ceph-mon[29360]: pgmap v74: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:38.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:38 smithi044 ceph-mon[36956]: pgmap v74: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:40.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:40 smithi044 ceph-mon[36956]: pgmap v75: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:40 smithi032 ceph-mon[29360]: pgmap v75: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:42 smithi044 ceph-mon[36956]: pgmap v76: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:42.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:42 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:24:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:42 smithi032 ceph-mon[29360]: pgmap v76: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:42 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:24:43.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:43 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:24:43.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:43 smithi032 ceph-mon[29360]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:24:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:44 smithi032 ceph-mon[29360]: pgmap v77: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:44 smithi044 ceph-mon[36956]: pgmap v77: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:45 smithi032 ceph-mon[29360]: pgmap v78: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:45 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:45 smithi044 ceph-mon[36956]: pgmap v78: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:45 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:24:48.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:48 smithi032 ceph-mon[29360]: pgmap v79: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 -- 172.21.15.32:0/3508948452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254101810 msgr2=0x7f5254103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 --2- 172.21.15.32:0/3508948452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254101810 0x7f5254103c00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5244009910 tx=0x7f5244009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 -- 172.21.15.32:0/3508948452 shutdown_connections 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 --2- 172.21.15.32:0/3508948452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254104140 0x7f5254106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 --2- 172.21.15.32:0/3508948452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254101810 0x7f5254103c00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 -- 172.21.15.32:0/3508948452 >> 172.21.15.32:0/3508948452 conn(0x7f52540fb160 msgr2=0x7f52540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 -- 172.21.15.32:0/3508948452 shutdown_connections 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 -- 172.21.15.32:0/3508948452 wait complete. 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.358+0000 7f525987c700 1 Processor -- start 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f525987c700 1 -- start start 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f525987c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254101810 0x7f52541967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f525987c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f525987c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254197300 con 0x7f5254104140 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f525987c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254197440 con 0x7f5254101810 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55484/0 (socket says 172.21.15.32:55484) 2024-06-03T01:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 -- 172.21.15.32:0/2715228961 learned_addr learned my addr 172.21.15.32:0/2715228961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f5252ffd700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254101810 0x7f52541967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 -- 172.21.15.32:0/2715228961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254101810 msgr2=0x7f52541967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254101810 0x7f52541967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.359+0000 7f52527fc700 1 -- 172.21.15.32:0/2715228961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440095f0 con 0x7f5254104140 2024-06-03T01:24:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f52527fc700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f524800c8a0 tx=0x7f524800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524800cea0 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5248004830 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525419bef0 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52480055b0 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.360+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525419c410 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.361+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5254190630 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.362+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f52480049a0 con 0x7f5254104140 2024-06-03T01:24:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.362+0000 7f525887a700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 0x7f523c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.362+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f524808d780 con 0x7f5254104140 2024-06-03T01:24:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.362+0000 7f5252ffd700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 0x7f523c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.363+0000 7f5252ffd700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 0x7f523c070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52440059d0 tx=0x7f5244005920 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.367+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f524805ab40 con 0x7f5254104140 2024-06-03T01:24:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:48 smithi044 ceph-mon[36956]: pgmap v79: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.593+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f525402d050 con 0x7f523c06de80 2024-06-03T01:24:48.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.594+0000 7f525887a700 1 -- 172.21.15.32:0/2715228961 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f525402d050 con 0x7f523c06de80 2024-06-03T01:24:48.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 msgr2=0x7f523c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 0x7f523c070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52440059d0 tx=0x7f5244005920 comp rx=0 tx=0).stop 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 msgr2=0x7f5254196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f524800c8a0 tx=0x7f524800cc60 comp rx=0 tx=0).stop 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 shutdown_connections 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f523c06de80 0x7f523c070340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5254101810 0x7f52541967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 --2- 172.21.15.32:0/2715228961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5254104140 0x7f5254196ce0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 >> 172.21.15.32:0/2715228961 conn(0x7f52540fb160 msgr2=0x7f52540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.596+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 shutdown_connections 2024-06-03T01:24:48.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:48.597+0000 7f525987c700 1 -- 172.21.15.32:0/2715228961 wait complete. 2024-06-03T01:24:49.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-06-03T01:24:49.419 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:24:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:50 smithi044 ceph-mon[36956]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:50 smithi044 ceph-mon[36956]: pgmap v80: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:50.488 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:50 smithi032 ceph-mon[29360]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:50.488 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:50 smithi032 ceph-mon[29360]: pgmap v80: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 -- 172.21.15.32:0/2051092916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 msgr2=0x7fb508101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 --2- 172.21.15.32:0/2051092916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508101280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009910 tx=0x7fb4f8009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 -- 172.21.15.32:0/2051092916 shutdown_connections 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 --2- 172.21.15.32:0/2051092916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5081058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 --2- 172.21.15.32:0/2051092916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508101280 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 -- 172.21.15.32:0/2051092916 >> 172.21.15.32:0/2051092916 conn(0x7fb5080fc750 msgr2=0x7fb5080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.979+0000 7fb50ed25700 1 -- 172.21.15.32:0/2051092916 shutdown_connections 2024-06-03T01:24:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 -- 172.21.15.32:0/2051092916 wait complete. 2024-06-03T01:24:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 Processor -- start 2024-06-03T01:24:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 -- start start 2024-06-03T01:24:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5080735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.980+0000 7fb50ed25700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508073b10 con 0x7fb508100ea0 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50ed25700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508073c50 con 0x7fb508101850 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55506/0 (socket says 172.21.15.32:55506) 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 -- 172.21.15.32:0/193438895 learned_addr learned my addr 172.21.15.32:0/193438895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb507fff700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5080735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 -- 172.21.15.32:0/193438895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 msgr2=0x7fb5080735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5080735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 -- 172.21.15.32:0/193438895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80095f0 con 0x7fb508100ea0 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb507fff700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5080735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:24:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50cac1700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8006010 tx=0x7fb4f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f801d070 con 0x7fb508100ea0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4f8004fb0 con 0x7fb508100ea0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb508073ed0 con 0x7fb508100ea0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.981+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f8022ea0 con 0x7fb508100ea0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.982+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081a2dd0 con 0x7fb508100ea0 2024-06-03T01:24:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.983+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb508109240 con 0x7fb508100ea0 2024-06-03T01:24:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.983+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fb4f8022470 con 0x7fb508100ea0 2024-06-03T01:24:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.984+0000 7fb505ffb700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 0x7fb4f0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.984+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb4f808f060 con 0x7fb508100ea0 2024-06-03T01:24:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.984+0000 7fb507fff700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 0x7fb4f0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.984+0000 7fb507fff700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 0x7fb4f0070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc005e30 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:50.988+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fb4f80589a0 con 0x7fb508100ea0 2024-06-03T01:24:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.302+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb508108c20 con 0x7fb508100ea0 2024-06-03T01:24:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.303+0000 7fb505ffb700 1 -- 172.21.15.32:0/193438895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7fb4f8022720 con 0x7fb508100ea0 2024-06-03T01:24:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 msgr2=0x7fb4f0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 0x7fb4f0070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc005e30 comp rx=0 tx=0).stop 2024-06-03T01:24:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 msgr2=0x7fb508073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8006010 tx=0x7fb4f800be20 comp rx=0 tx=0).stop 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 shutdown_connections 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fb4f006de80 0x7fb4f0070340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb508100ea0 0x7fb508073090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 --2- 172.21.15.32:0/193438895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb508101850 0x7fb5080735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 >> 172.21.15.32:0/193438895 conn(0x7fb5080fc750 msgr2=0x7fb5080fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 shutdown_connections 2024-06-03T01:24:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:51.305+0000 7fb50ed25700 1 -- 172.21.15.32:0/193438895 wait complete. 2024-06-03T01:24:51.316 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:24:51.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-06-03T01:24:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:24:52.314 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:52 smithi032 ceph-mon[29360]: pgmap v81: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:52.314 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:52 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/193438895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:52 smithi044 ceph-mon[36956]: pgmap v81: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:52 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/193438895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.960+0000 7f57f6b61700 1 -- 172.21.15.32:0/3558430491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0100ea0 msgr2=0x7f57f0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.960+0000 7f57f6b61700 1 --2- 172.21.15.32:0/3558430491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0100ea0 0x7f57f0101280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009910 tx=0x7f57e0009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- 172.21.15.32:0/3558430491 shutdown_connections 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 --2- 172.21.15.32:0/3558430491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0101850 0x7f57f01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 --2- 172.21.15.32:0/3558430491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0100ea0 0x7f57f0101280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- 172.21.15.32:0/3558430491 >> 172.21.15.32:0/3558430491 conn(0x7f57f00fc750 msgr2=0x7f57f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- 172.21.15.32:0/3558430491 shutdown_connections 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- 172.21.15.32:0/3558430491 wait complete. 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 Processor -- start 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- start start 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0100ea0 0x7f57f019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019f590 con 0x7f57f0101850 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.961+0000 7f57f6b61700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0198a40 con 0x7f57f0100ea0 2024-06-03T01:24:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55516/0 (socket says 172.21.15.32:55516) 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 -- 172.21.15.32:0/2372179296 learned_addr learned my addr 172.21.15.32:0/2372179296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57f48fd700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0100ea0 0x7f57f019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 -- 172.21.15.32:0/2372179296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0100ea0 msgr2=0x7f57f019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0100ea0 0x7f57f019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 -- 172.21.15.32:0/2372179296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00095f0 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57effff700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b480 tx=0x7f57d800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d8011800 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f0198d20 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57d8011e20 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.962+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d8010ea0 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.963+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f0199270 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.963+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f0105020 con 0x7f57f0101850 2024-06-03T01:24:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.964+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f57d8010380 con 0x7f57f0101850 2024-06-03T01:24:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.964+0000 7f57edffb700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 0x7f57dc070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.965+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f57d808d770 con 0x7f57f0101850 2024-06-03T01:24:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.965+0000 7f57f48fd700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 0x7f57dc070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.965+0000 7f57f48fd700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 0x7f57dc070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e001a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:53.969+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f57d805aa10 con 0x7f57f0101850 2024-06-03T01:24:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.294+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f57f0199f50 con 0x7f57f0101850 2024-06-03T01:24:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.294+0000 7f57edffb700 1 -- 172.21.15.32:0/2372179296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f57d805a5a0 con 0x7f57f0101850 2024-06-03T01:24:54.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:54 smithi032 ceph-mon[29360]: pgmap v82: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 msgr2=0x7f57dc070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 0x7f57dc070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e001a040 comp rx=0 tx=0).stop 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 msgr2=0x7f57f019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b480 tx=0x7f57d800b790 comp rx=0 tx=0).stop 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 shutdown_connections 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f57dc06de80 0x7f57dc070340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f0100ea0 0x7f57f019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 --2- 172.21.15.32:0/2372179296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f0101850 0x7f57f019ef00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.296+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 >> 172.21.15.32:0/2372179296 conn(0x7f57f00fc750 msgr2=0x7f57f00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.297+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 shutdown_connections 2024-06-03T01:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:54.297+0000 7f57f6b61700 1 -- 172.21.15.32:0/2372179296 wait complete. 2024-06-03T01:24:54.307 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)" 2024-06-03T01:24:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:54 smithi044 ceph-mon[36956]: pgmap v82: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:54.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-06-03T01:24:55.068 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:24:55.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:55 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2372179296' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:55.475 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:55 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2372179296' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:56.397 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:56 smithi032 ceph-mon[29360]: pgmap v83: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:56.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:56 smithi044 ceph-mon[36956]: pgmap v83: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 -- 172.21.15.32:0/759130845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 msgr2=0x7f09f0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 --2- 172.21.15.32:0/759130845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0100180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009910 tx=0x7f09e0009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 -- 172.21.15.32:0/759130845 shutdown_connections 2024-06-03T01:24:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 --2- 172.21.15.32:0/759130845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 0x7f09f01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 --2- 172.21.15.32:0/759130845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0100180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 -- 172.21.15.32:0/759130845 >> 172.21.15.32:0/759130845 conn(0x7f09f00fb650 msgr2=0x7f09f00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 -- 172.21.15.32:0/759130845 shutdown_connections 2024-06-03T01:24:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 -- 172.21.15.32:0/759130845 wait complete. 2024-06-03T01:24:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.737+0000 7f09f57a2700 1 Processor -- start 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09f57a2700 1 -- start start 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09f57a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09f57a2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 0x7f09f0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09f57a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0102350 con 0x7f09f00ffda0 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09f57a2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f01024c0 con 0x7f09f0100750 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55528/0 (socket says 172.21.15.32:55528) 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 -- 172.21.15.32:0/1936050235 learned_addr learned my addr 172.21.15.32:0/1936050235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09ee7fc700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 0x7f09f0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 -- 172.21.15.32:0/1936050235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 msgr2=0x7f09f0101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 0x7f09f0101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 -- 172.21.15.32:0/1936050235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00095f0 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.738+0000 7f09eeffd700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006010 tx=0x7f09e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.739+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e001d070 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.739+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09e0005dc0 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.739+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f01a2aa0 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.739+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e0017400 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.739+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f01a2e50 con 0x7f09f00ffda0 2024-06-03T01:24:56.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.740+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f004f2e0 con 0x7f09f00ffda0 2024-06-03T01:24:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.741+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f09e0003a60 con 0x7f09f00ffda0 2024-06-03T01:24:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.741+0000 7f09d7fff700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 0x7f09d8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.741+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f09e008f010 con 0x7f09f00ffda0 2024-06-03T01:24:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.741+0000 7f09ee7fc700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 0x7f09d8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.742+0000 7f09ee7fc700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 0x7f09d8070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09f0103030 tx=0x7f09e400b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:56.746+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f09e0058820 con 0x7f09f00ffda0 2024-06-03T01:24:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.050+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09f01a3040 con 0x7f09f00ffda0 2024-06-03T01:24:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.051+0000 7f09d7fff700 1 -- 172.21.15.32:0/1936050235 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f09e005bee0 con 0x7f09f00ffda0 2024-06-03T01:24:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 msgr2=0x7f09d8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 0x7f09d8070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09f0103030 tx=0x7f09e400b410 comp rx=0 tx=0).stop 2024-06-03T01:24:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 msgr2=0x7f09f0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006010 tx=0x7f09e000bd70 comp rx=0 tx=0).stop 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 shutdown_connections 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f09d806de80 0x7f09d8070340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f00ffda0 0x7f09f0101840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 --2- 172.21.15.32:0/1936050235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f0100750 0x7f09f0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 >> 172.21.15.32:0/1936050235 conn(0x7f09f00fb650 msgr2=0x7f09f0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 shutdown_connections 2024-06-03T01:24:57.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:57.053+0000 7f09f57a2700 1 -- 172.21.15.32:0/1936050235 wait complete. 2024-06-03T01:24:57.063 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:24:57.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:57 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1936050235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:57.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:57 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1936050235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:24:57.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-06-03T01:24:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:24:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:24:58 smithi032 ceph-mon[29360]: pgmap v84: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:24:58 smithi044 ceph-mon[36956]: pgmap v84: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:24:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/392853207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 msgr2=0x7f8e4810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/392853207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4810ac70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009910 tx=0x7f8e38009c20 comp rx=0 tx=0).stop 2024-06-03T01:24:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/392853207 shutdown_connections 2024-06-03T01:24:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/392853207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4810ac70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/392853207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 0x7f8e48102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/392853207 >> 172.21.15.32:0/392853207 conn(0x7f8e480fb650 msgr2=0x7f8e480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/392853207 shutdown_connections 2024-06-03T01:24:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.398+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/392853207 wait complete. 2024-06-03T01:24:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 Processor -- start 2024-06-03T01:24:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 -- start start 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 0x7f8e4819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4819d3b0 con 0x7f8e48102780 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.399+0000 7f8e4f8b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48196860 con 0x7f8e48101e60 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37506/0 (socket says 172.21.15.32:37506) 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 -- 172.21.15.32:0/1158302086 learned_addr learned my addr 172.21.15.32:0/1158302086 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4d64f700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 0x7f8e4819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 -- 172.21.15.32:0/1158302086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 msgr2=0x7f8e4819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 0x7f8e4819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 -- 172.21.15.32:0/1158302086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380095f0 con 0x7f8e48102780 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4ce4e700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005f50 tx=0x7f8e3800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3801d070 con 0x7f8e48102780 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e3800f460 con 0x7f8e48102780 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e48196ae0 con 0x7f8e48102780 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.400+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e38017630 con 0x7f8e48102780 2024-06-03T01:24:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.401+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e48196fd0 con 0x7f8e48102780 2024-06-03T01:24:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.401+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e48108300 con 0x7f8e48102780 2024-06-03T01:24:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.402+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f8e3800f5d0 con 0x7f8e48102780 2024-06-03T01:24:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.403+0000 7f8e3e7fc700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 0x7f8e34070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.403+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8e3808f070 con 0x7f8e48102780 2024-06-03T01:24:59.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.403+0000 7f8e4d64f700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 0x7f8e34070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:59.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.403+0000 7f8e4d64f700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 0x7f8e34070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009a10 tx=0x7f8e44009190 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.407+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f8e3805c430 con 0x7f8e48102780 2024-06-03T01:24:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:24:59.635+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e48197300 con 0x7f8e3406de80 2024-06-03T01:25:00.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:00 smithi044 ceph-mon[36956]: pgmap v85: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:00.473 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:00 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:00.519 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:00 smithi032 ceph-mon[29360]: pgmap v85: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:00.519 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:00 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:01.459 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:01 smithi032 ceph-mon[29360]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:01.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:01 smithi044 ceph-mon[36956]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.505+0000 7f8e3e7fc700 1 -- 172.21.15.32:0/1158302086 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5390 (secure 0 0 0) 0x7f8e48197300 con 0x7f8e3406de80 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 msgr2=0x7f8e34070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 0x7f8e34070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009a10 tx=0x7f8e44009190 comp rx=0 tx=0).stop 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 msgr2=0x7f8e4819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005f50 tx=0x7f8e3800bd70 comp rx=0 tx=0).stop 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 shutdown_connections 2024-06-03T01:25:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f8e3406de80 0x7f8e34070340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e48101e60 0x7f8e4819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 --2- 172.21.15.32:0/1158302086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e48102780 0x7f8e4819cd20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 >> 172.21.15.32:0/1158302086 conn(0x7f8e480fb650 msgr2=0x7f8e48105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:01.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 shutdown_connections 2024-06-03T01:25:01.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:01.507+0000 7f8e4f8b3700 1 -- 172.21.15.32:0/1158302086 wait complete. 2024-06-03T01:25:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:25:02.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ps' 2024-06-03T01:25:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:02.413 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:02 smithi032 ceph-mon[29360]: pgmap v86: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:02 smithi044 ceph-mon[36956]: pgmap v86: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:03.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 -- 172.21.15.32:0/3818488373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24068580 msgr2=0x7f7a240689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 --2- 172.21.15.32:0/3818488373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24068580 0x7f7a240689e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009910 tx=0x7f7a18009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 -- 172.21.15.32:0/3818488373 shutdown_connections 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 --2- 172.21.15.32:0/3818488373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24068580 0x7f7a240689e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 --2- 172.21.15.32:0/3818488373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24106cd0 0x7f7a241070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 -- 172.21.15.32:0/3818488373 >> 172.21.15.32:0/3818488373 conn(0x7f7a24075670 msgr2=0x7f7a24075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 -- 172.21.15.32:0/3818488373 shutdown_connections 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.952+0000 7f7a2b934700 1 -- 172.21.15.32:0/3818488373 wait complete. 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 Processor -- start 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 -- start start 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24068580 0x7f7a2419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2419d200 con 0x7f7a24106cd0 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a2b934700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241966b0 con 0x7f7a24068580 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a28ecf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a28ecf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37528/0 (socket says 172.21.15.32:37528) 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a28ecf700 1 -- 172.21.15.32:0/2339502967 learned_addr learned my addr 172.21.15.32:0/2339502967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.953+0000 7f7a296d0700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24068580 0x7f7a2419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a28ecf700 1 -- 172.21.15.32:0/2339502967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24068580 msgr2=0x7f7a2419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a28ecf700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24068580 0x7f7a2419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a28ecf700 1 -- 172.21.15.32:0/2339502967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180095f0 con 0x7f7a24106cd0 2024-06-03T01:25:03.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a28ecf700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006010 tx=0x7f7a1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:03.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a1801d070 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a24196930 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a18005dc0 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a18017400 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.954+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a24196e20 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.955+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a241091d0 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.956+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f7a18003a60 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.956+0000 7f7a227fc700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 0x7f7a0c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.956+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7a1808f040 con 0x7f7a24106cd0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.956+0000 7f7a296d0700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 0x7f7a0c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:03.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.957+0000 7f7a296d0700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 0x7f7a0c070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009550 tx=0x7f7a140091b0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:03.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:03.961+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f7a1805c380 con 0x7f7a24106cd0 2024-06-03T01:25:04.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.186+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a24197c60 con 0x7f7a0c06de80 2024-06-03T01:25:04.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.196+0000 7f7a227fc700 1 -- 172.21.15.32:0/2339502967 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7a24197c60 con 0x7f7a0c06de80 2024-06-03T01:25:04.196 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (82s) 52s ago 12m 17.2M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (12m) 52s ago 12m 7972k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (11m) 113s ago 11m 7713k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (12m) 52s ago 12m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (11m) 113s ago 11m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (58s) 52s ago 12m 52.1M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (6m) 52s ago 6m 125M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (6m) 52s ago 6m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (6m) 113s ago 6m 14.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (6m) 113s ago 6m 16.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:25:04.197 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (3m) 52s ago 13m 507M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (2m) 113s ago 11m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (13m) 52s ago 13m 57.7M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (11m) 113s ago 11m 44.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 52s ago 12m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (117s) 113s ago 11m 5465k - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (10m) 52s ago 10m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (9m) 52s ago 9m 239M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (9m) 52s ago 9m 202M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (9m) 113s ago 9m 266M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (8m) 113s ago 8m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (8m) 113s ago 8m 228M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:25:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (99s) 52s ago 11m 60.2M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.198+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 msgr2=0x7f7a0c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.198+0000 7f7a2b934700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 0x7f7a0c070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009550 tx=0x7f7a140091b0 comp rx=0 tx=0).stop 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.198+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 msgr2=0x7f7a2419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.198+0000 7f7a2b934700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006010 tx=0x7f7a1800bd70 comp rx=0 tx=0).stop 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 shutdown_connections 2024-06-03T01:25:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f7a0c06de80 0x7f7a0c070340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a24068580 0x7f7a2419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 --2- 172.21.15.32:0/2339502967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a24106cd0 0x7f7a2419cb70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 >> 172.21.15.32:0/2339502967 conn(0x7f7a24075670 msgr2=0x7f7a240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 shutdown_connections 2024-06-03T01:25:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:04.199+0000 7f7a2b934700 1 -- 172.21.15.32:0/2339502967 wait complete. 2024-06-03T01:25:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:04 smithi044 ceph-mon[36956]: pgmap v87: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:04.496 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:04 smithi032 ceph-mon[29360]: pgmap v87: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:04.785 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-03T01:25:04.787 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:25:04.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-06-03T01:25:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:05.451 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:05 smithi032 ceph-mon[29360]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:05.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:05 smithi044 ceph-mon[36956]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:06.384 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:06 smithi032 ceph-mon[29360]: pgmap v88: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:06.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:06 smithi044 ceph-mon[36956]: pgmap v88: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- 172.21.15.32:0/1211635604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 msgr2=0x7f902c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 --2- 172.21.15.32:0/1211635604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c1058a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9018009910 tx=0x7f9018009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- 172.21.15.32:0/1211635604 shutdown_connections 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 --2- 172.21.15.32:0/1211635604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c1058a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 --2- 172.21.15.32:0/1211635604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- 172.21.15.32:0/1211635604 >> 172.21.15.32:0/1211635604 conn(0x7f902c0fc750 msgr2=0x7f902c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- 172.21.15.32:0/1211635604 shutdown_connections 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- 172.21.15.32:0/1211635604 wait complete. 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 Processor -- start 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.642+0000 7f9033243700 1 -- start start 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9033243700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9033243700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9033243700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c19f4f0 con 0x7f902c101850 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9033243700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c1989a0 con 0x7f902c100ea0 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37544/0 (socket says 172.21.15.32:37544) 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 -- 172.21.15.32:0/928106034 learned_addr learned my addr 172.21.15.32:0/928106034 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9030fdf700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 -- 172.21.15.32:0/928106034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 msgr2=0x7f902c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 -- 172.21.15.32:0/928106034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180095f0 con 0x7f902c101850 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9030fdf700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f902bfff700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90180092e0 tx=0x7f901800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901801d070 con 0x7f902c101850 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f901800f460 con 0x7f902c101850 2024-06-03T01:25:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c198c20 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.643+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90180175f0 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.644+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c199110 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.645+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c105020 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.645+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f901800f5d0 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.646+0000 7f9029ffb700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 0x7f901c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.646+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f901808f190 con 0x7f902c101850 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.646+0000 7f9030fdf700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 0x7f901c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.646+0000 7f9030fdf700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 0x7f901c070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90200099d0 tx=0x7f9020008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.650+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f901805c430 con 0x7f902c101850 2024-06-03T01:25:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.866+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f902c04f2e0 con 0x7f902c101850 2024-06-03T01:25:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.870+0000 7f9029ffb700 1 -- 172.21.15.32:0/928106034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f901805bfc0 con 0x7f902c101850 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 msgr2=0x7f901c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 0x7f901c070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90200099d0 tx=0x7f9020008040 comp rx=0 tx=0).stop 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 msgr2=0x7f902c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90180092e0 tx=0x7f901800be20 comp rx=0 tx=0).stop 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 shutdown_connections 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f901c06de80 0x7f901c070340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f902c100ea0 0x7f902c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 --2- 172.21.15.32:0/928106034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f902c101850 0x7f902c19ee60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.872+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 >> 172.21.15.32:0/928106034 conn(0x7f902c0fc750 msgr2=0x7f902c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.873+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 shutdown_connections 2024-06-03T01:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:06.873+0000 7f9033243700 1 -- 172.21.15.32:0/928106034 wait complete. 2024-06-03T01:25:07.488 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-03T01:25:07.489 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:25:07.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-03T01:25:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:08.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/928106034' entity='client.admin' 2024-06-03T01:25:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:08.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:07 smithi032 ceph-mon[29360]: pgmap v89: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:08.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/928106034' entity='client.admin' 2024-06-03T01:25:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:08.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:07 smithi044 ceph-mon[36956]: pgmap v89: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 -- 172.21.15.32:0/123213287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 msgr2=0x7fe19c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/123213287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c105fb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe198009910 tx=0x7fe198009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 -- 172.21.15.32:0/123213287 shutdown_connections 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/123213287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 0x7fe19c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/123213287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c105fb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 -- 172.21.15.32:0/123213287 >> 172.21.15.32:0/123213287 conn(0x7fe19c075670 msgr2=0x7fe19c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 -- 172.21.15.32:0/123213287 shutdown_connections 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.184+0000 7fe1a3e85700 1 -- 172.21.15.32:0/123213287 wait complete. 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 Processor -- start 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 -- start start 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 0x7fe19c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c19d1f0 con 0x7fe19c105bd0 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a3e85700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1966a0 con 0x7fe19c0ffc20 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a1420700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a1420700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45852/0 (socket says 172.21.15.32:45852) 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a1420700 1 -- 172.21.15.32:0/3720639139 learned_addr learned my addr 172.21.15.32:0/3720639139 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.185+0000 7fe1a1c21700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 0x7fe19c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a1420700 1 -- 172.21.15.32:0/3720639139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 msgr2=0x7fe19c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a1420700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 0x7fe19c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a1420700 1 -- 172.21.15.32:0/3720639139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980095f0 con 0x7fe19c105bd0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a1420700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d8d0 tx=0x7fe18c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe18c0098a0 con 0x7fe19c105bd0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe18c010460 con 0x7fe19c105bd0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c196980 con 0x7fe19c105bd0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c196ed0 con 0x7fe19c105bd0 2024-06-03T01:25:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.186+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe18c00f5f0 con 0x7fe19c105bd0 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.187+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c04f2e0 con 0x7fe19c105bd0 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.188+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fe18c009a00 con 0x7fe19c105bd0 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.188+0000 7fe192ffd700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 0x7fe188070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.188+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe18c08dab0 con 0x7fe19c105bd0 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.188+0000 7fe1a1c21700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 0x7fe188070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.189+0000 7fe1a1c21700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 0x7fe188070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe198006010 tx=0x7fe19801a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.193+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe18c05adf0 con 0x7fe19c105bd0 2024-06-03T01:25:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.403+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fe19c197140 con 0x7fe19c105bd0 2024-06-03T01:25:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.403+0000 7fe192ffd700 1 -- 172.21.15.32:0/3720639139 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe18c05a980 con 0x7fe19c105bd0 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 msgr2=0x7fe188070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 0x7fe188070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe198006010 tx=0x7fe19801a040 comp rx=0 tx=0).stop 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 msgr2=0x7fe19c19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d8d0 tx=0x7fe18c00dbe0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 shutdown_connections 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe18806de80 0x7fe188070340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe19c0ffc20 0x7fe19c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 --2- 172.21.15.32:0/3720639139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19c105bd0 0x7fe19c19cb60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 >> 172.21.15.32:0/3720639139 conn(0x7fe19c075670 msgr2=0x7fe19c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 shutdown_connections 2024-06-03T01:25:09.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:09.406+0000 7fe1a3e85700 1 -- 172.21.15.32:0/3720639139 wait complete. 2024-06-03T01:25:10.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-03T01:25:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:10 smithi032 ceph-mon[29360]: pgmap v90: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:10 smithi044 ceph-mon[36956]: pgmap v90: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/960178830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d8105bf0 msgr2=0x7fe8d8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/960178830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d8105bf0 0x7fe8d8105fd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009910 tx=0x7fe8c8009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/960178830 shutdown_connections 2024-06-03T01:25:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/960178830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d80ffb40 0x7fe8d80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/960178830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d8105bf0 0x7fe8d8105fd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.868+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/960178830 >> 172.21.15.32:0/960178830 conn(0x7fe8d80fb650 msgr2=0x7fe8d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.869+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/960178830 shutdown_connections 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.869+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/960178830 wait complete. 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.869+0000 7fe8dd9ee700 1 Processor -- start 2024-06-03T01:25:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.869+0000 7fe8dd9ee700 1 -- start start 2024-06-03T01:25:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dd9ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dd9ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d8105bf0 0x7fe8d819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dd9ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d819d170 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dd9ee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8196620 con 0x7fe8d8105bf0 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45872/0 (socket says 172.21.15.32:45872) 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 -- 172.21.15.32:0/4070214862 learned_addr learned my addr 172.21.15.32:0/4070214862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d67fc700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d8105bf0 0x7fe8d819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 -- 172.21.15.32:0/4070214862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d8105bf0 msgr2=0x7fe8d819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d8105bf0 0x7fe8d819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 -- 172.21.15.32:0/4070214862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80095f0 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8d6ffd700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006010 tx=0x7fe8c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c801d070 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d81968a0 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8c8005dc0 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.870+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c8017400 con 0x7fe8d80ffb40 2024-06-03T01:25:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.871+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8196d90 con 0x7fe8d80ffb40 2024-06-03T01:25:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.871+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d804f2e0 con 0x7fe8d80ffb40 2024-06-03T01:25:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.872+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fe8c8003a60 con 0x7fe8d80ffb40 2024-06-03T01:25:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.873+0000 7fe8dc9ec700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 0x7fe8c0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.873+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe8c808f040 con 0x7fe8d80ffb40 2024-06-03T01:25:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.873+0000 7fe8d67fc700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 0x7fe8c0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.873+0000 7fe8d67fc700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 0x7fe8c0070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80fec00 tx=0x7fe8cc00b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:11.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:11.877+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe8c805c380 con 0x7fe8d80ffb40 2024-06-03T01:25:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.084+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fe8d8197b30 con 0x7fe8d80ffb40 2024-06-03T01:25:12.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.084+0000 7fe8dc9ec700 1 -- 172.21.15.32:0/4070214862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7fe8c805bf10 con 0x7fe8d80ffb40 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 msgr2=0x7fe8c0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 0x7fe8c0070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80fec00 tx=0x7fe8cc00b410 comp rx=0 tx=0).stop 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 msgr2=0x7fe8d819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006010 tx=0x7fe8c800bd70 comp rx=0 tx=0).stop 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 shutdown_connections 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe8c006de80 0x7fe8c0070340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d80ffb40 0x7fe8d819c5a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 --2- 172.21.15.32:0/4070214862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8d8105bf0 0x7fe8d819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 >> 172.21.15.32:0/4070214862 conn(0x7fe8d80fb650 msgr2=0x7fe8d8068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 shutdown_connections 2024-06-03T01:25:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:12.087+0000 7fe8dd9ee700 1 -- 172.21.15.32:0/4070214862 wait complete. 2024-06-03T01:25:12.441 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:12 smithi032 ceph-mon[29360]: pgmap v91: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:12 smithi044 ceph-mon[36956]: pgmap v91: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:12.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-03T01:25:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:14.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:14 smithi032 ceph-mon[29360]: pgmap v92: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 -- 172.21.15.32:0/145192077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 msgr2=0x7f9dcc10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/145192077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc10abf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009910 tx=0x7f9dbc009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 -- 172.21.15.32:0/145192077 shutdown_connections 2024-06-03T01:25:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/145192077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc10abf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/145192077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 0x7f9dcc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 -- 172.21.15.32:0/145192077 >> 172.21.15.32:0/145192077 conn(0x7f9dcc0fb610 msgr2=0x7f9dcc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 -- 172.21.15.32:0/145192077 shutdown_connections 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.449+0000 7f9dd34d6700 1 -- 172.21.15.32:0/145192077 wait complete. 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 Processor -- start 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 -- start start 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 0x7f9dcc0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1004c0 con 0x7f9dcc102700 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd34d6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc100600 con 0x7f9dcc101de0 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd0a71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd0a71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45878/0 (socket says 172.21.15.32:45878) 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.450+0000 7f9dd0a71700 1 -- 172.21.15.32:0/2885626324 learned_addr learned my addr 172.21.15.32:0/2885626324 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:14.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd1272700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 0x7f9dcc0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd0a71700 1 -- 172.21.15.32:0/2885626324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 msgr2=0x7f9dcc0ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd0a71700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 0x7f9dcc0ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd0a71700 1 -- 172.21.15.32:0/2885626324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0095f0 con 0x7f9dcc102700 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd0a71700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0059d0 tx=0x7f9dbc00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dbc01d070 con 0x7f9dcc102700 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dbc0052a0 con 0x7f9dcc102700 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dbc022470 con 0x7f9dcc102700 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc100880 con 0x7f9dcc102700 2024-06-03T01:25:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.451+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc1a2d20 con 0x7f9dcc102700 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.452+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc1082f0 con 0x7f9dcc102700 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.453+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f9dbc00f6f0 con 0x7f9dcc102700 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.453+0000 7f9dc27fc700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 0x7f9db8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.454+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9dbc08eef0 con 0x7f9dcc102700 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.454+0000 7f9dd1272700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 0x7f9db8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.454+0000 7f9dd1272700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 0x7f9db8070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800f540 tx=0x7f9dc800b040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.458+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f9dbc058700 con 0x7f9dcc102700 2024-06-03T01:25:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:14 smithi044 ceph-mon[36956]: pgmap v92: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.674+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f9dcc04f2e0 con 0x7f9dcc102700 2024-06-03T01:25:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.674+0000 7f9dc27fc700 1 -- 172.21.15.32:0/2885626324 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9dbc05bdc0 con 0x7f9dcc102700 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.676+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 msgr2=0x7f9db8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.676+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 0x7f9db8070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800f540 tx=0x7f9dc800b040 comp rx=0 tx=0).stop 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.676+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 msgr2=0x7f9dcc0fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.676+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0059d0 tx=0x7f9dbc00b5a0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 shutdown_connections 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f9db806de80 0x7f9db8070340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dcc101de0 0x7f9dcc0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 --2- 172.21.15.32:0/2885626324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9dcc102700 0x7f9dcc0fff80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 >> 172.21.15.32:0/2885626324 conn(0x7f9dcc0fb610 msgr2=0x7f9dcc105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:14.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 shutdown_connections 2024-06-03T01:25:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:14.677+0000 7f9dd34d6700 1 -- 172.21.15.32:0/2885626324 wait complete. 2024-06-03T01:25:15.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-06-03T01:25:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:15.942 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:15 smithi032 ceph-mon[29360]: pgmap v93: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:15.942 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:15 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:16.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:15 smithi044 ceph-mon[36956]: pgmap v93: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:16.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:15 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 -- 172.21.15.32:0/4058047839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c100750 msgr2=0x7f265c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 --2- 172.21.15.32:0/4058047839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c100750 0x7f265c1047a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2650009910 tx=0x7f2650009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 -- 172.21.15.32:0/4058047839 shutdown_connections 2024-06-03T01:25:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 --2- 172.21.15.32:0/4058047839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c100750 0x7f265c1047a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 --2- 172.21.15.32:0/4058047839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c0ffda0 0x7f265c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 -- 172.21.15.32:0/4058047839 >> 172.21.15.32:0/4058047839 conn(0x7f265c0fb650 msgr2=0x7f265c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 -- 172.21.15.32:0/4058047839 shutdown_connections 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.255+0000 7f2662e80700 1 -- 172.21.15.32:0/4058047839 wait complete. 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 Processor -- start 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 -- start start 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c100750 0x7f265c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c198410 con 0x7f265c0ffda0 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2662e80700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c198550 con 0x7f265c100750 2024-06-03T01:25:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2660c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2660c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47836/0 (socket says 172.21.15.32:47836) 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.256+0000 7f2660c1c700 1 -- 172.21.15.32:0/3443785008 learned_addr learned my addr 172.21.15.32:0/3443785008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f265bfff700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c100750 0x7f265c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2660c1c700 1 -- 172.21.15.32:0/3443785008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c100750 msgr2=0x7f265c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2660c1c700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c100750 0x7f265c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2660c1c700 1 -- 172.21.15.32:0/3443785008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500095f0 con 0x7f265c0ffda0 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2660c1c700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b4b0 tx=0x7f264c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264c011800 con 0x7f265c0ffda0 2024-06-03T01:25:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c1a0860 con 0x7f265c0ffda0 2024-06-03T01:25:17.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f264c011e20 con 0x7f265c0ffda0 2024-06-03T01:25:17.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.257+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264c00f2e0 con 0x7f265c0ffda0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.258+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c1a0d80 con 0x7f265c0ffda0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.258+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c1080d0 con 0x7f265c0ffda0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.259+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f264c010380 con 0x7f265c0ffda0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.259+0000 7f2659ffb700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 0x7f2644070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.260+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f264c08d730 con 0x7f265c0ffda0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.260+0000 7f265bfff700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 0x7f2644070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.260+0000 7f265bfff700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 0x7f2644070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f265c069ef0 tx=0x7f265001a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:17.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.264+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f264c056f40 con 0x7f265c0ffda0 2024-06-03T01:25:17.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.491+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}) v1 -- 0x7f265c0611d0 con 0x7f264406de80 2024-06-03T01:25:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.497+0000 7f2659ffb700 1 -- 172.21.15.32:0/3443785008 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f265c0611d0 con 0x7f264406de80 2024-06-03T01:25:17.497 INFO:teuthology.orchestra.run.smithi032.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 msgr2=0x7f2644070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 0x7f2644070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f265c069ef0 tx=0x7f265001a040 comp rx=0 tx=0).stop 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 msgr2=0x7f265c1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b4b0 tx=0x7f264c00b7c0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 shutdown_connections 2024-06-03T01:25:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f264406de80 0x7f2644070340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f265c0ffda0 0x7f265c1977f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 --2- 172.21.15.32:0/3443785008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f265c100750 0x7f265c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 >> 172.21.15.32:0/3443785008 conn(0x7f265c0fb650 msgr2=0x7f265c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 shutdown_connections 2024-06-03T01:25:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:17.499+0000 7f2662e80700 1 -- 172.21.15.32:0/3443785008 wait complete. 2024-06-03T01:25:18.148 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-03T01:25:18.149 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:25:18.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-06-03T01:25:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:18 smithi032 ceph-mon[29360]: pgmap v94: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:18 smithi032 ceph-mon[29360]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:18 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:25:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:18 smithi044 ceph-mon[36956]: pgmap v94: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:18 smithi044 ceph-mon[36956]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:18 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:19.436 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:19 smithi032 ceph-mon[29360]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:25:19.436 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:19.436 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:19.436 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:19 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:19.436 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:19 smithi032 ceph-mon[29360]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:25:19.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:19 smithi044 ceph-mon[36956]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:25:19.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:19.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:19 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:19.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:19 smithi044 ceph-mon[36956]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:25:19.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 -- 172.21.15.32:0/2205864815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3700ffda0 msgr2=0x7fe370100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:19.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 --2- 172.21.15.32:0/2205864815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3700ffda0 0x7fe370100180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b0a0 tx=0x7fe36c00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:25:19.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 -- 172.21.15.32:0/2205864815 shutdown_connections 2024-06-03T01:25:19.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 --2- 172.21.15.32:0/2205864815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe370100750 0x7fe3701047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 --2- 172.21.15.32:0/2205864815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3700ffda0 0x7fe370100180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 -- 172.21.15.32:0/2205864815 >> 172.21.15.32:0/2205864815 conn(0x7fe3700fb650 msgr2=0x7fe3700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 -- 172.21.15.32:0/2205864815 shutdown_connections 2024-06-03T01:25:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 -- 172.21.15.32:0/2205864815 wait complete. 2024-06-03T01:25:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.922+0000 7fe377636700 1 Processor -- start 2024-06-03T01:25:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe377636700 1 -- start start 2024-06-03T01:25:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe377636700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3700ffda0 0x7fe37019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe377636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe377636700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37019d2b0 con 0x7fe370100750 2024-06-03T01:25:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe377636700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370196760 con 0x7fe3700ffda0 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47864/0 (socket says 172.21.15.32:47864) 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 -- 172.21.15.32:0/3748543778 learned_addr learned my addr 172.21.15.32:0/3748543778 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe3753d2700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3700ffda0 0x7fe37019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 -- 172.21.15.32:0/3748543778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3700ffda0 msgr2=0x7fe37019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3700ffda0 0x7fe37019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.923+0000 7fe374bd1700 1 -- 172.21.15.32:0/3748543778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c009d00 con 0x7fe370100750 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe374bd1700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe36000d900 tx=0x7fe36000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe360009900 con 0x7fe370100750 2024-06-03T01:25:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe370196a40 con 0x7fe370100750 2024-06-03T01:25:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe360010460 con 0x7fe370100750 2024-06-03T01:25:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe36000f5f0 con 0x7fe370100750 2024-06-03T01:25:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.924+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe370196f90 con 0x7fe370100750 2024-06-03T01:25:19.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.925+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37004f2e0 con 0x7fe370100750 2024-06-03T01:25:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.925+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fe3600105d0 con 0x7fe370100750 2024-06-03T01:25:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.926+0000 7fe3667fc700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 0x7fe35c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.926+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe36008d840 con 0x7fe370100750 2024-06-03T01:25:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.926+0000 7fe3753d2700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 0x7fe35c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.926+0000 7fe3753d2700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 0x7fe35c070340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000f80 tx=0x7fe36c0060c0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:19.930+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe36005aae0 con 0x7fe370100750 2024-06-03T01:25:20.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.164+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe370197220 con 0x7fe35c06de80 2024-06-03T01:25:20.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.165+0000 7fe3667fc700 1 -- 172.21.15.32:0/3748543778 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe370197220 con 0x7fe35c06de80 2024-06-03T01:25:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 msgr2=0x7fe35c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 0x7fe35c070340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000f80 tx=0x7fe36c0060c0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 msgr2=0x7fe37019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe36000d900 tx=0x7fe36000dc10 comp rx=0 tx=0).stop 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 shutdown_connections 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe35c06de80 0x7fe35c070340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3700ffda0 0x7fe37019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 --2- 172.21.15.32:0/3748543778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100750 0x7fe37019cc20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 >> 172.21.15.32:0/3748543778 conn(0x7fe3700fb650 msgr2=0x7fe370103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 shutdown_connections 2024-06-03T01:25:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.167+0000 7fe377636700 1 -- 172.21.15.32:0/3748543778 wait complete. 2024-06-03T01:25:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:25:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 -- 172.21.15.32:0/3532414072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb4105bd0 msgr2=0x7febb4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 --2- 172.21.15.32:0/3532414072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb4105bd0 0x7febb4105fb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feba4009910 tx=0x7feba4009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 -- 172.21.15.32:0/3532414072 shutdown_connections 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 --2- 172.21.15.32:0/3532414072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb40ffc20 0x7febb4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 --2- 172.21.15.32:0/3532414072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb4105bd0 0x7febb4105fb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 -- 172.21.15.32:0/3532414072 >> 172.21.15.32:0/3532414072 conn(0x7febb4075670 msgr2=0x7febb4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 -- 172.21.15.32:0/3532414072 shutdown_connections 2024-06-03T01:25:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.282+0000 7febb980a700 1 -- 172.21.15.32:0/3532414072 wait complete. 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 Processor -- start 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 -- start start 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb4105bd0 0x7febb419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb419d1f0 con 0x7febb40ffc20 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.283+0000 7febb980a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb41966a0 con 0x7febb4105bd0 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47882/0 (socket says 172.21.15.32:47882) 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 -- 172.21.15.32:0/3156321959 learned_addr learned my addr 172.21.15.32:0/3156321959 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febaa7fc700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb4105bd0 0x7febb419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 -- 172.21.15.32:0/3156321959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb4105bd0 msgr2=0x7febb419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb4105bd0 0x7febb419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 -- 172.21.15.32:0/3156321959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40095f0 con 0x7febb40ffc20 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb2ffd700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feba4006010 tx=0x7feba400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feba401d070 con 0x7febb40ffc20 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feba4005dc0 con 0x7febb40ffc20 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb4196920 con 0x7febb40ffc20 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feba4017400 con 0x7febb40ffc20 2024-06-03T01:25:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.284+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb4196e10 con 0x7febb40ffc20 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.285+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb404f2e0 con 0x7febb40ffc20 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.286+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7feba4003a60 con 0x7febb40ffc20 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.286+0000 7febb8808700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 0x7feba0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.287+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7feba408efb0 con 0x7febb40ffc20 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.287+0000 7febaa7fc700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 0x7feba0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.287+0000 7febaa7fc700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 0x7feba0070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febb40ffa40 tx=0x7feb9c00b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.291+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7feba405c370 con 0x7febb40ffc20 2024-06-03T01:25:20.468 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:20 smithi032 ceph-mon[29360]: pgmap v95: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:20 smithi044 ceph-mon[36956]: pgmap v95: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.525+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febb4197120 con 0x7feba006de80 2024-06-03T01:25:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.526+0000 7febb8808700 1 -- 172.21.15.32:0/3156321959 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7febb4197120 con 0x7feba006de80 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 msgr2=0x7feba0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 0x7feba0070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febb40ffa40 tx=0x7feb9c00b410 comp rx=0 tx=0).stop 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 msgr2=0x7febb419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feba4006010 tx=0x7feba400bd70 comp rx=0 tx=0).stop 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 shutdown_connections 2024-06-03T01:25:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7feba006de80 0x7feba0070340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febb40ffc20 0x7febb419c620 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 --2- 172.21.15.32:0/3156321959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febb4105bd0 0x7febb419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 >> 172.21.15.32:0/3156321959 conn(0x7febb4075670 msgr2=0x7febb4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 shutdown_connections 2024-06-03T01:25:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.528+0000 7febb980a700 1 -- 172.21.15.32:0/3156321959 wait complete. 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 -- 172.21.15.32:0/3611674460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80ffb40 msgr2=0x7f83c80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 --2- 172.21.15.32:0/3611674460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80ffb40 0x7f83c80fffc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009910 tx=0x7f83bc009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 -- 172.21.15.32:0/3611674460 shutdown_connections 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 --2- 172.21.15.32:0/3611674460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80ffb40 0x7f83c80fffc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 --2- 172.21.15.32:0/3611674460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c8105bf0 0x7f83c8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 -- 172.21.15.32:0/3611674460 >> 172.21.15.32:0/3611674460 conn(0x7f83c80fb610 msgr2=0x7f83c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.637+0000 7f83ccef1700 1 -- 172.21.15.32:0/3611674460 shutdown_connections 2024-06-03T01:25:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.638+0000 7f83ccef1700 1 -- 172.21.15.32:0/3611674460 wait complete. 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.638+0000 7f83ccef1700 1 Processor -- start 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.638+0000 7f83ccef1700 1 -- start start 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83ccef1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c80ffb40 0x7f83c819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83ccef1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83ccef1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c819d1d0 con 0x7f83c8105bf0 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83ccef1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8196680 con 0x7f83c80ffb40 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47908/0 (socket says 172.21.15.32:47908) 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 -- 172.21.15.32:0/2028531921 learned_addr learned my addr 172.21.15.32:0/2028531921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c659c700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c80ffb40 0x7f83c819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 -- 172.21.15.32:0/2028531921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c80ffb40 msgr2=0x7f83c819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c80ffb40 0x7f83c819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 -- 172.21.15.32:0/2028531921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc0095f0 con 0x7f83c8105bf0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83c5d9b700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006010 tx=0x7f83bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc01d070 con 0x7f83c8105bf0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8196900 con 0x7f83c8105bf0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.639+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83bc005dc0 con 0x7f83c8105bf0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.640+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc017400 con 0x7f83c8105bf0 2024-06-03T01:25:20.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.640+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c8196df0 con 0x7f83c8105bf0 2024-06-03T01:25:20.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.640+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c80679c0 con 0x7f83c8105bf0 2024-06-03T01:25:20.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.641+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f83bc003a60 con 0x7f83c8105bf0 2024-06-03T01:25:20.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.642+0000 7f83b77fe700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 0x7f83b0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:20.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.642+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f83bc08f030 con 0x7f83c8105bf0 2024-06-03T01:25:20.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.642+0000 7f83c659c700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 0x7f83b0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:20.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.642+0000 7f83c659c700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 0x7f83b0070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005950 tx=0x7f83b800b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:20.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.647+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f83bc05c370 con 0x7f83c8105bf0 2024-06-03T01:25:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.892+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f83c8197170 con 0x7f83b006de80 2024-06-03T01:25:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.902+0000 7f83b77fe700 1 -- 172.21.15.32:0/2028531921 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f83c8197170 con 0x7f83b006de80 2024-06-03T01:25:20.902 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (98s) 69s ago 12m 17.2M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (12m) 69s ago 12m 7972k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (11m) 2m ago 11m 7713k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (12m) 69s ago 12m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (11m) 2m ago 11m 6656k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (75s) 69s ago 12m 52.1M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (7m) 69s ago 7m 125M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (6m) 69s ago 6m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (7m) 2m ago 7m 14.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (6m) 2m ago 6m 16.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (3m) 69s ago 13m 507M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (2m) 2m ago 11m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (14m) 69s ago 14m 57.7M 2048M 18.2.1 5be31c24972a 403a340e7d5e 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (11m) 2m ago 11m 44.7M 2048M 18.2.1 5be31c24972a b1e9d52e00f7 2024-06-03T01:25:20.903 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 69s ago 12m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (2m) 2m ago 11m 5465k - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (10m) 69s ago 10m 238M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (10m) 69s ago 10m 239M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (9m) 69s ago 9m 202M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (9m) 2m ago 9m 266M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (8m) 2m ago 8m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (8m) 2m ago 8m 228M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:25:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (115s) 69s ago 12m 60.2M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 msgr2=0x7f83b0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 0x7f83b0070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005950 tx=0x7f83b800b410 comp rx=0 tx=0).stop 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 msgr2=0x7f83c819cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006010 tx=0x7f83bc00bd70 comp rx=0 tx=0).stop 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 shutdown_connections 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f83b006de80 0x7f83b0070340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c80ffb40 0x7f83c819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 --2- 172.21.15.32:0/2028531921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c8105bf0 0x7f83c819cb40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.904+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 >> 172.21.15.32:0/2028531921 conn(0x7f83c80fb610 msgr2=0x7f83c8068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:20.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.905+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 shutdown_connections 2024-06-03T01:25:20.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:20.905+0000 7f83ccef1700 1 -- 172.21.15.32:0/2028531921 wait complete. 2024-06-03T01:25:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 -- 172.21.15.32:0/953159524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 msgr2=0x7f54181021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 --2- 172.21.15.32:0/953159524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54181021d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5414009910 tx=0x7f5414009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 -- 172.21.15.32:0/953159524 shutdown_connections 2024-06-03T01:25:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 --2- 172.21.15.32:0/953159524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f541810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 --2- 172.21.15.32:0/953159524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54181021d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 -- 172.21.15.32:0/953159524 >> 172.21.15.32:0/953159524 conn(0x7f54180fb620 msgr2=0x7f54180fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 -- 172.21.15.32:0/953159524 shutdown_connections 2024-06-03T01:25:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.011+0000 7f54202db700 1 -- 172.21.15.32:0/953159524 wait complete. 2024-06-03T01:25:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 Processor -- start 2024-06-03T01:25:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 -- start start 2024-06-03T01:25:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f54180fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181004d0 con 0x7f5418101df0 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f54202db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418100610 con 0x7f5418102710 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47924/0 (socket says 172.21.15.32:47924) 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 -- 172.21.15.32:0/1506787437 learned_addr learned my addr 172.21.15.32:0/1506787437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541d876700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f54180fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 -- 172.21.15.32:0/1506787437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 msgr2=0x7f54180fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f54180fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541e077700 1 -- 172.21.15.32:0/1506787437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140095f0 con 0x7f5418101df0 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.012+0000 7f541d876700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f54180fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f541e077700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5414006010 tx=0x7f541400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541401d070 con 0x7f5418101df0 2024-06-03T01:25:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5418100890 con 0x7f5418101df0 2024-06-03T01:25:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f541400f460 con 0x7f5418101df0 2024-06-03T01:25:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5414017630 con 0x7f5418101df0 2024-06-03T01:25:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.013+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181a2db0 con 0x7f5418101df0 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.014+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5418108300 con 0x7f5418101df0 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.014+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f541400f5d0 con 0x7f5418101df0 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.015+0000 7f540b7fe700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 0x7f5404070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.015+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f541408f0a0 con 0x7f5418101df0 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.015+0000 7f541d876700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 0x7f5404070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.016+0000 7f541d876700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 0x7f5404070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f540c005fd0 tx=0x7f540c005f00 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.020+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5414058850 con 0x7f5418101df0 2024-06-03T01:25:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.321+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f541804f2e0 con 0x7f5418101df0 2024-06-03T01:25:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.321+0000 7f540b7fe700 1 -- 172.21.15.32:0/1506787437 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+673 (secure 0 0 0) 0x7f541405bf10 con 0x7f5418101df0 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:25:21.323 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 msgr2=0x7f5404070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 0x7f5404070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f540c005fd0 tx=0x7f540c005f00 comp rx=0 tx=0).stop 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 msgr2=0x7f54180ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5414006010 tx=0x7f541400be20 comp rx=0 tx=0).stop 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 shutdown_connections 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7f540406de80 0x7f5404070340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5418101df0 0x7f54180ffa50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 --2- 172.21.15.32:0/1506787437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5418102710 0x7f54180fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.323+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 >> 172.21.15.32:0/1506787437 conn(0x7f54180fb620 msgr2=0x7f5418105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.324+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 shutdown_connections 2024-06-03T01:25:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.324+0000 7f54202db700 1 -- 172.21.15.32:0/1506787437 wait complete. 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.435+0000 7fd4afb06700 1 -- 172.21.15.32:0/2563226782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 msgr2=0x7fd4a8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.435+0000 7fd4afb06700 1 --2- 172.21.15.32:0/2563226782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8100180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009910 tx=0x7fd4a4009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.435+0000 7fd4afb06700 1 -- 172.21.15.32:0/2563226782 shutdown_connections 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 --2- 172.21.15.32:0/2563226782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 0x7fd4a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 --2- 172.21.15.32:0/2563226782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8100180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 -- 172.21.15.32:0/2563226782 >> 172.21.15.32:0/2563226782 conn(0x7fd4a80fb650 msgr2=0x7fd4a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 -- 172.21.15.32:0/2563226782 shutdown_connections 2024-06-03T01:25:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 -- 172.21.15.32:0/2563226782 wait complete. 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 Processor -- start 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 -- start start 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.436+0000 7fd4afb06700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 0x7fd4a8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4afb06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8102350 con 0x7fd4a80ffda0 2024-06-03T01:25:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4afb06700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81024c0 con 0x7fd4a8100750 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47946/0 (socket says 172.21.15.32:47946) 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 -- 172.21.15.32:0/917417080 learned_addr learned my addr 172.21.15.32:0/917417080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad0a1700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 0x7fd4a8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 -- 172.21.15.32:0/917417080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 msgr2=0x7fd4a8101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 0x7fd4a8101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 -- 172.21.15.32:0/917417080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40095f0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4ad8a2700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4a401d070 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4a4004fb0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4a4022ea0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a81a2aa0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.437+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a81a2e50 con 0x7fd4a80ffda0 2024-06-03T01:25:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.438+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a804f2e0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.439+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fd4a4022470 con 0x7fd4a80ffda0 2024-06-03T01:25:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.439+0000 7fd49effd700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 0x7fd494070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.439+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd4a408f1f0 con 0x7fd4a80ffda0 2024-06-03T01:25:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.439+0000 7fd4ad0a1700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 0x7fd494070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.440+0000 7fd4ad0a1700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 0x7fd494070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd498005fd0 tx=0x7fd49800a560 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.444+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fd4a4058a10 con 0x7fd4a80ffda0 2024-06-03T01:25:21.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:21 smithi032 ceph-mon[29360]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:21.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:21 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/1506787437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.701+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd4a81a3040 con 0x7fd4a80ffda0 2024-06-03T01:25:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.702+0000 7fd49effd700 1 -- 172.21.15.32:0/917417080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1832 (secure 0 0 0) 0x7fd4a4022720 con 0x7fd4a80ffda0 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:25:21.703 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:25:21.704 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:21.706 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:25:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:21.706 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:21.706 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 msgr2=0x7fd494070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 0x7fd494070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd498005fd0 tx=0x7fd49800a560 comp rx=0 tx=0).stop 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 msgr2=0x7fd4a8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a400be20 comp rx=0 tx=0).stop 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 shutdown_connections 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fd49406de80 0x7fd494070340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a80ffda0 0x7fd4a8101840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 --2- 172.21.15.32:0/917417080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a8100750 0x7fd4a8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 >> 172.21.15.32:0/917417080 conn(0x7fd4a80fb650 msgr2=0x7fd4a8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 shutdown_connections 2024-06-03T01:25:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.705+0000 7fd4afb06700 1 -- 172.21.15.32:0/917417080 wait complete. 2024-06-03T01:25:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:25:21.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:21 smithi044 ceph-mon[36956]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:21.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:21 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/1506787437' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2171065834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4068580 msgr2=0x7ffaa40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2171065834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4068580 0x7ffaa40689e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009910 tx=0x7ffa8c009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2171065834 shutdown_connections 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2171065834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4068580 0x7ffaa40689e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2171065834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4106cd0 0x7ffaa41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2171065834 >> 172.21.15.32:0/2171065834 conn(0x7ffaa4075670 msgr2=0x7ffaa4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2171065834 shutdown_connections 2024-06-03T01:25:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.821+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2171065834 wait complete. 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.822+0000 7ffaa99f4700 1 Processor -- start 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.822+0000 7ffaa99f4700 1 -- start start 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa99f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4068580 0x7ffaa419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa99f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa99f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa419d200 con 0x7ffaa4106cd0 2024-06-03T01:25:21.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa99f4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa41966b0 con 0x7ffaa4068580 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47956/0 (socket says 172.21.15.32:47956) 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 -- 172.21.15.32:0/2775525551 learned_addr learned my addr 172.21.15.32:0/2775525551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa2ffd700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4068580 0x7ffaa419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 -- 172.21.15.32:0/2775525551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4068580 msgr2=0x7ffaa419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4068580 0x7ffaa419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 -- 172.21.15.32:0/2775525551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0095f0 con 0x7ffaa4106cd0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.823+0000 7ffaa27fc700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.824+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8c01d070 con 0x7ffaa4106cd0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.824+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa8c00f460 con 0x7ffaa4106cd0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.824+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa4196930 con 0x7ffaa4106cd0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.824+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8c017630 con 0x7ffaa4106cd0 2024-06-03T01:25:21.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.824+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa4196e20 con 0x7ffaa4106cd0 2024-06-03T01:25:21.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.825+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa41091d0 con 0x7ffaa4106cd0 2024-06-03T01:25:21.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.825+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7ffa8c00f5d0 con 0x7ffaa4106cd0 2024-06-03T01:25:21.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.826+0000 7ffaa89f2700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 0x7ffa90070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.826+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffa8c08f080 con 0x7ffaa4106cd0 2024-06-03T01:25:21.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.826+0000 7ffaa2ffd700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 0x7ffa90070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.827+0000 7ffaa2ffd700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 0x7ffa90070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa94005950 tx=0x7ffa940058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:21.831+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ffa8c05c3c0 con 0x7ffaa4106cd0 2024-06-03T01:25:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.075+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 --> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaa4197100 con 0x7ffa9006de80 2024-06-03T01:25:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.077+0000 7ffaa89f2700 1 -- 172.21.15.32:0/2775525551 <== mgr.14652 v2:172.21.15.32:6800/2112372769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ffaa4197100 con 0x7ffa9006de80 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/23 daemons upgraded", 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mon daemons", 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:25:22.078 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 msgr2=0x7ffa90070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 0x7ffa90070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa94005950 tx=0x7ffa940058e0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 msgr2=0x7ffaa419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c00be20 comp rx=0 tx=0).stop 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 shutdown_connections 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7ffa9006de80 0x7ffa90070340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaa4068580 0x7ffaa419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 --2- 172.21.15.32:0/2775525551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa4106cd0 0x7ffaa419cb70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.080+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 >> 172.21.15.32:0/2775525551 conn(0x7ffaa4075670 msgr2=0x7ffaa40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.081+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 shutdown_connections 2024-06-03T01:25:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.081+0000 7ffaa99f4700 1 -- 172.21.15.32:0/2775525551 wait complete. 2024-06-03T01:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 -- 172.21.15.32:0/3024608647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 msgr2=0x7fe91c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 --2- 172.21.15.32:0/3024608647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c0ff980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009910 tx=0x7fe90c009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 -- 172.21.15.32:0/3024608647 shutdown_connections 2024-06-03T01:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 --2- 172.21.15.32:0/3024608647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 --2- 172.21.15.32:0/3024608647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c0ff980 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.191+0000 7fe9222ba700 1 -- 172.21.15.32:0/3024608647 >> 172.21.15.32:0/3024608647 conn(0x7fe91c074e30 msgr2=0x7fe91c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.192+0000 7fe9222ba700 1 -- 172.21.15.32:0/3024608647 shutdown_connections 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.192+0000 7fe9222ba700 1 -- 172.21.15.32:0/3024608647 wait complete. 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.192+0000 7fe9222ba700 1 Processor -- start 2024-06-03T01:25:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.192+0000 7fe9222ba700 1 -- start start 2024-06-03T01:25:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9222ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9222ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9222ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c100fa0 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9222ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c1010e0 con 0x7fe91c0fff50 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47980/0 (socket says 172.21.15.32:47980) 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 -- 172.21.15.32:0/2545790205 learned_addr learned my addr 172.21.15.32:0/2545790205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 -- 172.21.15.32:0/2545790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 msgr2=0x7fe91c100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91b7fe700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 -- 172.21.15.32:0/2545790205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0095f0 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91b7fe700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe91bfff700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006010 tx=0x7fe90c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe90c01d070 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c101360 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.193+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe90c004fb0 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.194+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe90c022ea0 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.194+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c1a2e30 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.194+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c109270 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.195+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fe90c022470 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.196+0000 7fe9197fa700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 0x7fe904070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.196+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe90c08f130 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.196+0000 7fe91b7fe700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 0x7fe904070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.196+0000 7fe91b7fe700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 0x7fe904070340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe910005fd0 tx=0x7fe910005dc0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:22.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.200+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe90c0589f0 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.517+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe91c101a50 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.517+0000 7fe9197fa700 1 -- 172.21.15.32:0/2545790205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe90c022720 con 0x7fe91c0ff5a0 2024-06-03T01:25:22.518 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:22.518 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 msgr2=0x7fe904070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 0x7fe904070340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe910005fd0 tx=0x7fe910005dc0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 msgr2=0x7fe91c100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006010 tx=0x7fe90c00be20 comp rx=0 tx=0).stop 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 shutdown_connections 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:6800/2112372769,v1:172.21.15.32:6801/2112372769] conn(0x7fe90406de80 0x7fe904070340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c0ff5a0 0x7fe91c100520 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 --2- 172.21.15.32:0/2545790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe91c0fff50 0x7fe91c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 >> 172.21.15.32:0/2545790205 conn(0x7fe91c074e30 msgr2=0x7fe91c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 shutdown_connections 2024-06-03T01:25:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:22.520+0000 7fe9222ba700 1 -- 172.21.15.32:0/2545790205 wait complete. 2024-06-03T01:25:22.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:22.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: pgmap v96: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:22.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:22.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-03T01:25:22.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:22 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/917417080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: pgmap v96: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:22.826 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:22.827 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-03T01:25:22.827 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:22 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/917417080' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: Upgrade: Target is version 19.0.0-2946-g3d89accf (squid) 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: Upgrade: Setting container_image for all mgr 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='client.? 172.21.15.32:0/2545790205' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:25:23.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:23.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:25:23.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:25:23.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:23 smithi032 ceph-mon[29360]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: Upgrade: Target is version 19.0.0-2946-g3d89accf (squid) 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9'] 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: Upgrade: Setting container_image for all mgr 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='client.? 172.21.15.32:0/2545790205' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:25:23.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:23 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:24.667 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:24 smithi032 ceph-mon[29360]: pgmap v97: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:24.667 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:24 smithi032 ceph-mon[29360]: Upgrade: Updating mon.smithi032 2024-06-03T01:25:24.668 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:24 smithi032 ceph-mon[29360]: Deploying daemon mon.smithi032 on smithi032 2024-06-03T01:25:24.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:24 smithi044 ceph-mon[36956]: pgmap v97: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:24.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:24 smithi044 ceph-mon[36956]: Upgrade: Updating mon.smithi032 2024-06-03T01:25:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:24 smithi044 ceph-mon[36956]: Deploying daemon mon.smithi032 on smithi032 2024-06-03T01:25:25.250 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:24 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:25:25.574 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[29356]: 2024-06-03T01:25:25.249+0000 7f584a85e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:25.574 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[29356]: 2024-06-03T01:25:25.249+0000 7f584a85e700 -1 mon.smithi032@0(leader) e2 *** Got Signal Terminated *** 2024-06-03T01:25:25.575 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 podman[111748]: 2024-06-03 01:25:25.280682385 +0000 UTC m=+0.148029200 container died 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-06-03T01:25:25.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 podman[111748]: 2024-06-03 01:25:25.666254978 +0000 UTC m=+0.533601785 container cleanup 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, ceph=True, RELEASE=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T01:25:25.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 bash[111748]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032 2024-06-03T01:25:26.271 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:25 smithi032 podman[111773]: 2024-06-03 01:25:25.900885589 +0000 UTC m=+0.614558541 container remove 403a340e7d5e13bf3738f04d331768f3905bfb2008c4b18b46aae45635479754 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.build-date=20240222, RELEASE=HEAD, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-03T01:25:26.271 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Deactivated successfully. 2024-06-03T01:25:26.271 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 systemd[1]: Stopped Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:25:26.271 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Consumed 15.403s CPU time. 2024-06-03T01:25:26.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 systemd[1]: Starting Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:25:26.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 podman[111900]: 2024-06-03 01:25:26.460336583 +0000 UTC m=+0.085635161 container create d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:25:26.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 podman[111900]: 2024-06-03 01:25:26.392093233 +0000 UTC m=+0.017391808 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 podman[111900]: 2024-06-03 01:25:26.646480341 +0000 UTC m=+0.271778900 container init d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 podman[111900]: 2024-06-03 01:25:26.649940619 +0000 UTC m=+0.275239179 container start d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: pidfile_write: ignore empty --pid-file 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: load: jerasure load: lrc 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: RocksDB version: 7.9.2 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Git sha 0 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T01:25:26.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: DB SUMMARY 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: DB Session ID: 9GQNR7M57PM6D4FRBUN5 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: CURRENT file: CURRENT 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi032/store.db dir, Total Num: 1, files: 000032.sst 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi032/store.db: 000030.log size: 4827798 ; 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.error_if_exists: 0 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.create_if_missing: 0 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.paranoid_checks: 1 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T01:25:26.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.env: 0x5615147f6c60 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.info_log: 0x5615165d19e0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.statistics: (nil) 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.use_fsync: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_log_file_size: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_fallocate: 1 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T01:25:26.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.use_direct_reads: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.db_log_dir: 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.wal_dir: 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.write_buffer_manager: 0x5615165a21e0 2024-06-03T01:25:26.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.unordered_write: 0 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T01:25:26.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.row_cache: None 2024-06-03T01:25:26.841 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.wal_filter: None 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.two_write_queues: 0 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.wal_compression: 0 2024-06-03T01:25:26.842 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.atomic_flush: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.log_readahead_size: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_background_jobs: 2 2024-06-03T01:25:26.843 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_background_compactions: -1 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_subcompactions: 1 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_open_files: -1 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T01:25:26.844 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_background_flushes: -1 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Compression algorithms supported: 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kZSTD supported: 0 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kXpressCompression supported: 0 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kZlibCompression supported: 1 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kSnappyCompression supported: 1 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kLZ4Compression supported: 1 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: kBZip2Compression supported: 0 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000015 2024-06-03T01:25:26.845 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.merge_operator: 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_filter: None 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5615165d1ae0) 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-06-03T01:25:26.846 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_type: 0 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_index_type: 0 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_shortening: 1 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: checksum: 4 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: no_block_cache: 0 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache: 0x5615165d3090 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_options: 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: capacity : 536870912 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_shard_bits : 4 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: strict_capacity_limit : 0 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_compressed: (nil) 2024-06-03T01:25:26.847 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: persistent_cache: (nil) 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size: 4096 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size_deviation: 10 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_restart_interval: 16 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_block_restart_interval: 1 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_block_size: 4096 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: partition_filters: 0 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: use_delta_encoding: 1 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: filter_policy: bloomfilter 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: whole_key_filtering: 1 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: verify_compression: 0 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: format_version: 5 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_index_compression: 1 2024-06-03T01:25:26.848 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_align: 0 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_auto_readahead_size: 262144 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: prepopulate_block_cache: 0 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: initial_auto_readahead_size: 8192 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression: NoCompression 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.num_levels: 7 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T01:25:26.850 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T01:25:26.851 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T01:25:26.852 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.table_properties_collectors: 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.inplace_update_support: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.bloom_locality: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.max_successive_merges: 0 2024-06-03T01:25:26.853 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.ttl: 2592000 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enable_blob_files: false 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.min_blob_size: 0 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T01:25:26.854 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12034, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cdeefeec-51c4-4327-a8c6-47189881b844 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377926680457, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377926694782, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4270015, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12035, "largest_seqno": 15455, "table_properties": {"data_size": 4256729, "index_size": 8425, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3845, "raw_key_size": 33578, "raw_average_key_size": 21, "raw_value_size": 4226604, "raw_average_value_size": 2767, "num_data_blocks": 381, "num_entries": 1527, "num_filter_entries": 1527, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717377926, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cdeefeec-51c4-4327-a8c6-47189881b844", "db_session_id": "9GQNR7M57PM6D4FRBUN5", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377926694890, "job": 1, "event": "recovery_finished"} 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-03T01:25:26.855 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi032/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5615166c2000 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: DB pointer 0x5615166ac000 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** DB Stats ** 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.856 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L0 1/0 4.07 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.3 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L6 1/0 4.98 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Sum 2/0 9.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.3 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.3 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 631.3 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.857 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative compaction: 0.00 GB write, 240.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval compaction: 0.00 GB write, 240.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache BinnedLRUCache@0x5615165d3090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T01:25:26.858 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: starting mon.smithi032 rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi032 fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???) e2 preinit fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).mds e21 new map 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).mds e21 print_map 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: e21 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: legacy client fscid: 1 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Filesystem 'cephfs' (1) 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: fs_name cephfs 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: epoch 21 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:25:26.859 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: tableserver 0 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: root 0 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: session_timeout 60 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: session_autoclose 300 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_file_size 1099511627776 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_xattr_size 65536 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: required_client_features {} 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: last_failure 0 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: last_failure_osd_epoch 46 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_mds 2 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: in 0,1 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: up {0=14478,1=14498} 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: failed 2024-06-03T01:25:26.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: damaged 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: stopped 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_pools [3] 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_pool 2 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: inline_data enabled 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: balancer 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: bal_rank_mask -1 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: standby_count_wanted 1 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: qdb_cluster leader: 0 members: 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Standby daemons: 2024-06-03T01:25:26.861 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 ceph-mon[111919]: mon.smithi032@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 bash[111900]: d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 2024-06-03T01:25:26.862 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:26 smithi032 systemd[1]: Started Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:25:29.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: pgmap v99: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: pgmap v100: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: mon.smithi032 calling monitor election 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: monmap epoch 2 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: last_changed 2024-06-03T01:13:57.712306+0000 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: created 2024-06-03T01:11:10.279598+0000 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: min_mon_release 18 (reef) 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: election_strategy: 1 2024-06-03T01:25:29.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: osdmap e48: 6 total, 6 up, 6 in 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: mgrmap e32: smithi032.kfcfaa(active, since 3m), standbys: smithi044.wszjlw 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: from='mgr.14652 ' entity='' 2024-06-03T01:25:29.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:29 smithi032 ceph-mon[111919]: mgrmap e33: smithi032.kfcfaa(active, since 3m), standbys: smithi044.wszjlw 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: from='mgr.14652 172.21.15.32:0/3365581716' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: pgmap v99: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: pgmap v100: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: mon.smithi032 calling monitor election 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: monmap epoch 2 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: last_changed 2024-06-03T01:13:57.712306+0000 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: created 2024-06-03T01:11:10.279598+0000 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: min_mon_release 18 (reef) 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: election_strategy: 1 2024-06-03T01:25:30.109 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: osdmap e48: 6 total, 6 up, 6 in 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: mgrmap e32: smithi032.kfcfaa(active, since 3m), standbys: smithi044.wszjlw 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: from='mgr.14652 ' entity='' 2024-06-03T01:25:30.110 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:29 smithi044 ceph-mon[36956]: mgrmap e33: smithi032.kfcfaa(active, since 3m), standbys: smithi044.wszjlw 2024-06-03T01:25:37.051 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: Active manager daemon smithi032.kfcfaa restarted 2024-06-03T01:25:37.051 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:25:37.051 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:37.051 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: mgrmap e34: smithi032.kfcfaa(active, starting, since 0.00542102s), standbys: smithi044.wszjlw 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:25:37.052 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:25:37.053 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:25:37.053 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:25:37.053 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:25:37.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:25:37.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:25:37.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:25:37.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:36 smithi044 ceph-mon[36956]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:25:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: Active manager daemon smithi032.kfcfaa restarted 2024-06-03T01:25:37.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: Activating manager daemon smithi032.kfcfaa 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: mgrmap e34: smithi032.kfcfaa(active, starting, since 0.00542102s), standbys: smithi044.wszjlw 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi032.kfcfaa", "id": "smithi032.kfcfaa"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr metadata", "who": "smithi044.wszjlw", "id": "smithi044.wszjlw"}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:25:37.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: Manager daemon smithi032.kfcfaa is now available 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: Standby manager daemon smithi044.wszjlw restarted 2024-06-03T01:25:37.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:36 smithi032 ceph-mon[111919]: Standby manager daemon smithi044.wszjlw started 2024-06-03T01:25:38.237 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:38.237 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:25:38.237 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:25:38.237 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: [03/Jun/2024:01:25:37] ENGINE Bus STARTING 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: [03/Jun/2024:01:25:37] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: [03/Jun/2024:01:25:37] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: [03/Jun/2024:01:25:37] ENGINE Bus STARTED 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:37 smithi032 ceph-mon[111919]: mgrmap e35: smithi032.kfcfaa(active, since 1.01169s), standbys: smithi044.wszjlw 2024-06-03T01:25:38.238 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/crt"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.wszjlw/key"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.? 172.21.15.44:0/2589026081' entity='mgr.smithi044.wszjlw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.kfcfaa/trash_purge_schedule"}]: dispatch 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: [03/Jun/2024:01:25:37] ENGINE Bus STARTING 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: [03/Jun/2024:01:25:37] ENGINE Serving on http://172.21.15.32:8765 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: [03/Jun/2024:01:25:37] ENGINE Serving on https://172.21.15.32:7150 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: [03/Jun/2024:01:25:37] ENGINE Bus STARTED 2024-06-03T01:25:38.239 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:37 smithi044 ceph-mon[36956]: mgrmap e35: smithi032.kfcfaa(active, since 1.01169s), standbys: smithi044.wszjlw 2024-06-03T01:25:39.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:38 smithi032 ceph-mon[111919]: pgmap v3: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:38 smithi044 ceph-mon[36956]: pgmap v3: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:39 smithi044 ceph-mon[36956]: pgmap v4: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:39 smithi044 ceph-mon[36956]: mgrmap e36: smithi032.kfcfaa(active, since 2s), standbys: smithi044.wszjlw 2024-06-03T01:25:40.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:39 smithi032 ceph-mon[111919]: pgmap v4: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:40.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:39 smithi032 ceph-mon[111919]: mgrmap e36: smithi032.kfcfaa(active, since 2s), standbys: smithi044.wszjlw 2024-06-03T01:25:41.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:41 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:41.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:41 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:41.786 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:41 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:41.786 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:41 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: pgmap v5: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: mgrmap e37: smithi032.kfcfaa(active, since 4s), standbys: smithi044.wszjlw 2024-06-03T01:25:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: pgmap v5: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:42.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: mgrmap e37: smithi032.kfcfaa(active, since 4s), standbys: smithi044.wszjlw 2024-06-03T01:25:42.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:42.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:42 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:44 smithi044 ceph-mon[36956]: pgmap v6: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:44.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:44 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:44.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:44 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:44.735 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:44 smithi032 ceph-mon[111919]: pgmap v6: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:44.736 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:44.736 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:46.566 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:46 smithi032 ceph-mon[111919]: pgmap v7: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-06-03T01:25:46.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:46 smithi044 ceph-mon[36956]: pgmap v7: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-06-03T01:25:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Detected new or changed devices on smithi032 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: pgmap v8: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:48.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:48.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:25:48.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:47 smithi044 ceph-mon[36956]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Detected new or changed devices on smithi032 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi032:/etc/ceph/ceph.conf 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi044:/etc/ceph/ceph.conf 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: pgmap v8: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:48.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:48.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:25:48.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:47 smithi032 ceph-mon[111919]: Updating smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:25:48.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:48 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-03T01:25:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:25:49.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-03T01:25:50.953 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: pgmap v9: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-03T01:25:50.953 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: Upgrade: Updating mon.smithi044 2024-06-03T01:25:50.953 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:50.953 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:25:50.954 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:25:50.954 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:50.954 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:50 smithi044 ceph-mon[36956]: Deploying daemon mon.smithi044 on smithi044 2024-06-03T01:25:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: pgmap v9: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-03T01:25:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: Upgrade: Updating mon.smithi044 2024-06-03T01:25:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:25:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:25:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:25:51.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:50 smithi032 ceph-mon[111919]: Deploying daemon mon.smithi044 on smithi044 2024-06-03T01:25:51.903 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:51 smithi044 systemd[1]: Stopping Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:25:51.903 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:51 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044[36944]: 2024-06-03T01:25:51.890+0000 7fa19cd8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi044 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:51.903 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:51 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044[36944]: 2024-06-03T01:25:51.890+0000 7fa19cd8f700 -1 mon.smithi044@1(peon) e2 *** Got Signal Terminated *** 2024-06-03T01:25:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:51 smithi044 podman[84057]: 2024-06-03 01:25:51.963275799 +0000 UTC m=+0.163047161 container died b1e9d52e00f738c55ce33d9f09abb8deff6b77a9a3bde02da99b176ff72baca4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 -- 172.21.15.32:0/3113639608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100ea0 msgr2=0x7fbf00101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 --2- 172.21.15.32:0/3113639608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100ea0 0x7fbf00101280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009910 tx=0x7fbefc009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 -- 172.21.15.32:0/3113639608 shutdown_connections 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 --2- 172.21.15.32:0/3113639608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00101850 0x7fbf001058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 --2- 172.21.15.32:0/3113639608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00100ea0 0x7fbf00101280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 -- 172.21.15.32:0/3113639608 >> 172.21.15.32:0/3113639608 conn(0x7fbf000fc750 msgr2=0x7fbf000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 -- 172.21.15.32:0/3113639608 shutdown_connections 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.645+0000 7fbf078f3700 1 -- 172.21.15.32:0/3113639608 wait complete. 2024-06-03T01:25:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.646+0000 7fbf078f3700 1 Processor -- start 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.646+0000 7fbf078f3700 1 -- start start 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.646+0000 7fbf078f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 0x7fbf0019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.646+0000 7fbf078f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.646+0000 7fbf078f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0019d2b0 con 0x7fbf00101850 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf078f3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00196760 con 0x7fbf00100ea0 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf0568f700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 msgr2=0x7fbf0019c6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf0568f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 0x7fbf0019c6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45752/0 (socket says 172.21.15.32:45752) 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 -- 172.21.15.32:0/2008414736 learned_addr learned my addr 172.21.15.32:0/2008414736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 -- 172.21.15.32:0/2008414736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 msgr2=0x7fbf0019c6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 0x7fbf0019c6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 -- 172.21.15.32:0/2008414736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc0095f0 con 0x7fbf00101850 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf04e8e700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b4f0 tx=0x7fbef000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef0011800 con 0x7fbf00101850 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbef0011e20 con 0x7fbf00101850 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00196a40 con 0x7fbf00101850 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.647+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef000f2e0 con 0x7fbf00101850 2024-06-03T01:25:52.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.648+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf00196f90 con 0x7fbf00101850 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.648+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00109240 con 0x7fbf00101850 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.649+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fbef0010380 con 0x7fbf00101850 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.650+0000 7fbef67fc700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 0x7fbeec0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.650+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbef008e2a0 con 0x7fbf00101850 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.650+0000 7fbf0568f700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 0x7fbeec0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.650+0000 7fbf0568f700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 0x7fbeec0707a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b540 tx=0x7fbefc009fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:52.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.654+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbef005b180 con 0x7fbf00101850 2024-06-03T01:25:52.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 podman[84057]: 2024-06-03 01:25:52.300010476 +0000 UTC m=+0.499781830 container cleanup b1e9d52e00f738c55ce33d9f09abb8deff6b77a9a3bde02da99b176ff72baca4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, maintainer=Guillaume Abrioux , RELEASE=HEAD, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:25:52.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 bash[84057]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044 2024-06-03T01:25:52.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 podman[84069]: 2024-06-03 01:25:52.468043487 +0000 UTC m=+0.501175523 container remove b1e9d52e00f738c55ce33d9f09abb8deff6b77a9a3bde02da99b176ff72baca4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-03T01:25:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.881+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf00197210 con 0x7fbeec06e2e0 2024-06-03T01:25:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.883+0000 7fbef67fc700 1 -- 172.21.15.32:0/2008414736 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbf00197210 con 0x7fbeec06e2e0 2024-06-03T01:25:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 msgr2=0x7fbeec0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 0x7fbeec0707a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b540 tx=0x7fbefc009fd0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 msgr2=0x7fbf0019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b4f0 tx=0x7fbef000b8b0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 shutdown_connections 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbeec06e2e0 0x7fbeec0707a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf00100ea0 0x7fbf0019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 --2- 172.21.15.32:0/2008414736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf00101850 0x7fbf0019cc20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.885+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 >> 172.21.15.32:0/2008414736 conn(0x7fbf000fc750 msgr2=0x7fbf000fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.886+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 shutdown_connections 2024-06-03T01:25:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.886+0000 7fbf078f3700 1 -- 172.21.15.32:0/2008414736 wait complete. 2024-06-03T01:25:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 -- 172.21.15.32:0/247858067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37800ffb40 msgr2=0x7f37800fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 --2- 172.21.15.32:0/247858067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37800ffb40 0x7f37800fffc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f377c009910 tx=0x7f377c009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 -- 172.21.15.32:0/247858067 shutdown_connections 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 --2- 172.21.15.32:0/247858067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37800ffb40 0x7f37800fffc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 --2- 172.21.15.32:0/247858067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3780105bf0 0x7f3780105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 -- 172.21.15.32:0/247858067 >> 172.21.15.32:0/247858067 conn(0x7f37800fb610 msgr2=0x7f37800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 -- 172.21.15.32:0/247858067 shutdown_connections 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.997+0000 7f3787f8a700 1 -- 172.21.15.32:0/247858067 wait complete. 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 Processor -- start 2024-06-03T01:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 -- start start 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 0x7f37801017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3780102230 con 0x7f3780105bf0 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3787f8a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3780102370 con 0x7f37800ffb40 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3785d26700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 msgr2=0x7f37801017b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3785d26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 0x7f37801017b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3785525700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3785525700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45774/0 (socket says 172.21.15.32:45774) 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.998+0000 7f3785525700 1 -- 172.21.15.32:0/515596438 learned_addr learned my addr 172.21.15.32:0/515596438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3785525700 1 -- 172.21.15.32:0/515596438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 msgr2=0x7f37801017b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3785525700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 0x7f37801017b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3785525700 1 -- 172.21.15.32:0/515596438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c0095f0 con 0x7f3780105bf0 2024-06-03T01:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3785525700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f377c005b40 tx=0x7f377c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377c01d070 con 0x7f3780105bf0 2024-06-03T01:25:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37801025f0 con 0x7f3780105bf0 2024-06-03T01:25:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f377c0048e0 con 0x7f3780105bf0 2024-06-03T01:25:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377c003b20 con 0x7f3780105bf0 2024-06-03T01:25:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:52.999+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37801a2dc0 con 0x7f3780105bf0 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.000+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378004f2e0 con 0x7f3780105bf0 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.001+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f377c004a50 con 0x7f3780105bf0 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.001+0000 7f3772ffd700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 0x7f376c070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.001+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f377c08fa80 con 0x7f3780105bf0 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.001+0000 7f3785d26700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 0x7f376c070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.002+0000 7f3785d26700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 0x7f376c070790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3774001050 tx=0x7f3774006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.006+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f377c058b50 con 0x7f3780105bf0 2024-06-03T01:25:53.153 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi044.service: Deactivated successfully. 2024-06-03T01:25:53.154 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 systemd[1]: Stopped Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:25:53.154 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi044.service: Consumed 8.260s CPU time. 2024-06-03T01:25:53.154 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:52 smithi044 systemd[1]: Starting Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:25:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.234+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37800695e0 con 0x7f376c06e2d0 2024-06-03T01:25:53.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.235+0000 7f3772ffd700 1 -- 172.21.15.32:0/515596438 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f37800695e0 con 0x7f376c06e2d0 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.237+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 msgr2=0x7f376c070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.237+0000 7f3787f8a700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 0x7f376c070790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3774001050 tx=0x7f3774006df0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.237+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 msgr2=0x7f3780101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.237+0000 7f3787f8a700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f377c005b40 tx=0x7f377c00be20 comp rx=0 tx=0).stop 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 shutdown_connections 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f376c06e2d0 0x7f376c070790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37800ffb40 0x7f37801017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 --2- 172.21.15.32:0/515596438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3780105bf0 0x7f3780101cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 >> 172.21.15.32:0/515596438 conn(0x7f37800fb610 msgr2=0x7f3780068ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:53.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 shutdown_connections 2024-06-03T01:25:53.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.238+0000 7f3787f8a700 1 -- 172.21.15.32:0/515596438 wait complete. 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- 172.21.15.32:0/1030563422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 msgr2=0x7f991010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 --2- 172.21.15.32:0/1030563422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991010d000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9900009910 tx=0x7f9900009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- 172.21.15.32:0/1030563422 shutdown_connections 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 --2- 172.21.15.32:0/1030563422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991010d000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 --2- 172.21.15.32:0/1030563422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 0x7f99100ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- 172.21.15.32:0/1030563422 >> 172.21.15.32:0/1030563422 conn(0x7f99100fb650 msgr2=0x7f99100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- 172.21.15.32:0/1030563422 shutdown_connections 2024-06-03T01:25:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- 172.21.15.32:0/1030563422 wait complete. 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 Processor -- start 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.346+0000 7f9917633700 1 -- start start 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9917633700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 0x7f991019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9917633700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9917633700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991019f560 con 0x7f9910100340 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9917633700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910198a10 con 0x7f99100ff990 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f99153cf700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 msgr2=0x7f991019e990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f99153cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 0x7f991019e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45798/0 (socket says 172.21.15.32:45798) 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 -- 172.21.15.32:0/2846410075 learned_addr learned my addr 172.21.15.32:0/2846410075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 -- 172.21.15.32:0/2846410075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 msgr2=0x7f991019e990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 0x7f991019e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 -- 172.21.15.32:0/2846410075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000095f0 con 0x7f9910100340 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f9914bce700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f9900005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.347+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990001d070 con 0x7f9910100340 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.348+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9910198c90 con 0x7f9910100340 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.348+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f990000f460 con 0x7f9910100340 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.348+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9900017630 con 0x7f9910100340 2024-06-03T01:25:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.348+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99101991e0 con 0x7f9910100340 2024-06-03T01:25:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.348+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991010a770 con 0x7f9910100340 2024-06-03T01:25:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.349+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f990000f5d0 con 0x7f9910100340 2024-06-03T01:25:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.350+0000 7f99067fc700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 0x7f98fc070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.350+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f990008fa20 con 0x7f9910100340 2024-06-03T01:25:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.350+0000 7f99153cf700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 0x7f98fc070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.350+0000 7f99153cf700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 0x7f98fc070740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f990c001050 tx=0x7f990c006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.355+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f990005c780 con 0x7f9910100340 2024-06-03T01:25:53.425 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 podman[84194]: 2024-06-03 01:25:53.057016727 +0000 UTC m=+0.017737863 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:25:53.425 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 podman[84194]: 2024-06-03 01:25:53.163378106 +0000 UTC m=+0.124099222 container create 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:25:53.425 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 podman[84194]: 2024-06-03 01:25:53.365727423 +0000 UTC m=+0.326448539 container init 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T01:25:53.425 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 podman[84194]: 2024-06-03 01:25:53.369161788 +0000 UTC m=+0.329882908 container start 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: pidfile_write: ignore empty --pid-file 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: load: jerasure load: lrc 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: RocksDB version: 7.9.2 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Git sha 0 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: DB SUMMARY 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: DB Session ID: 657NU927NBPGA9AQJRGX 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: CURRENT file: CURRENT 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T01:25:53.426 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi044/store.db dir, Total Num: 1, files: 000027.sst 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi044/store.db: 000025.log size: 9872058 ; 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.error_if_exists: 0 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.create_if_missing: 0 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.paranoid_checks: 1 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.env: 0x55d780b01c60 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.info_log: 0x55d7832b99e0 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.statistics: (nil) 2024-06-03T01:25:53.427 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.use_fsync: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_log_file_size: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_fallocate: 1 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.use_direct_reads: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.db_log_dir: 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.wal_dir: 2024-06-03T01:25:53.428 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.write_buffer_manager: 0x55d78328a1e0 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T01:25:53.429 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T01:25:53.430 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T01:25:53.430 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.unordered_write: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.row_cache: None 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.wal_filter: None 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.two_write_queues: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T01:25:53.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.wal_compression: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.atomic_flush: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.log_readahead_size: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T01:25:53.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_background_jobs: 2 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_background_compactions: -1 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_subcompactions: 1 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_open_files: -1 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T01:25:53.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_background_flushes: -1 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Compression algorithms supported: 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kZSTD supported: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kXpressCompression supported: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kZlibCompression supported: 1 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kSnappyCompression supported: 1 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kLZ4Compression supported: 1 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: kBZip2Compression supported: 0 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T01:25:53.434 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000010 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.merge_operator: 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_filter: None 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d7832b9ae0) 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T01:25:53.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_type: 0 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_index_type: 0 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_shortening: 1 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: checksum: 4 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: no_block_cache: 0 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache: 0x55d7832bb090 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_options: 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: capacity : 536870912 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_shard_bits : 4 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: strict_capacity_limit : 0 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-06-03T01:25:53.436 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_compressed: (nil) 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: persistent_cache: (nil) 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size: 4096 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size_deviation: 10 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_restart_interval: 16 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_block_restart_interval: 1 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_block_size: 4096 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: partition_filters: 0 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: use_delta_encoding: 1 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: filter_policy: bloomfilter 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: whole_key_filtering: 1 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: verify_compression: 0 2024-06-03T01:25:53.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: format_version: 5 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_index_compression: 1 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_align: 0 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_auto_readahead_size: 262144 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: prepopulate_block_cache: 0 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: initial_auto_readahead_size: 8192 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression: NoCompression 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T01:25:53.438 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.num_levels: 7 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:25:53.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T01:25:53.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T01:25:53.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T01:25:53.442 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.table_properties_collectors: 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.inplace_update_support: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.bloom_locality: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.max_successive_merges: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T01:25:53.443 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.ttl: 2592000 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enable_blob_files: false 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.min_blob_size: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T01:25:53.444 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12048, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9ff58016-de4e-4565-a30a-178f41e925c3 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377953400565, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377953425522, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 5813583, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12053, "largest_seqno": 16311, "table_properties": {"data_size": 5798360, "index_size": 9850, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4357, "raw_key_size": 38258, "raw_average_key_size": 22, "raw_value_size": 5764071, "raw_average_value_size": 3339, "num_data_blocks": 443, "num_entries": 1726, "num_filter_entries": 1726, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717377953, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9ff58016-de4e-4565-a30a-178f41e925c3", "db_session_id": "657NU927NBPGA9AQJRGX", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717377953425628, "job": 1, "event": "recovery_finished"} 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-06-03T01:25:53.445 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-03T01:25:53.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.574+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f991002cce0 con 0x7f98fc06e280 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.585+0000 7f99067fc700 1 -- 172.21.15.32:0/2846410075 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f991002cce0 con 0x7f98fc06e280 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (2m) 11s ago 13m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (13m) 11s ago 13m 8090k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (12m) 13s ago 12m 7914k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:25:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (13m) 11s ago 13m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (12m) 13s ago 12m 6660k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (108s) 11s ago 12m 76.2M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (7m) 11s ago 7m 125M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (7m) 11s ago 7m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (7m) 13s ago 7m 15.4M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (7m) 13s ago 7m 17.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (4m) 11s ago 14m 475M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (3m) 13s ago 12m 448M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (27s) 11s ago 14m 29.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 starting - - - 2048M 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 11s ago 13m 15.2M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:25:53.586 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (2m) 13s ago 12m 16.3M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (11m) 11s ago 11m 239M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (10m) 11s ago 10m 241M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (10m) 11s ago 10m 204M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (9m) 13s ago 9m 268M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (9m) 13s ago 9m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (9m) 13s ago 9m 230M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:25:53.587 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (2m) 11s ago 12m 71.7M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 msgr2=0x7f98fc070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 0x7f98fc070740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f990c001050 tx=0x7f990c006df0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 msgr2=0x7f991019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f9900005690 comp rx=0 tx=0).stop 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 shutdown_connections 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98fc06e280 0x7f98fc070740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99100ff990 0x7f991019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 --2- 172.21.15.32:0/2846410075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9910100340 0x7f991019eed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 >> 172.21.15.32:0/2846410075 conn(0x7f99100fb650 msgr2=0x7f99100fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 shutdown_connections 2024-06-03T01:25:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.588+0000 7f9917633700 1 -- 172.21.15.32:0/2846410075 wait complete. 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 -- 172.21.15.32:0/381583401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 msgr2=0x7fb374100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 --2- 172.21.15.32:0/381583401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374100180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb370009910 tx=0x7fb370009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 -- 172.21.15.32:0/381583401 shutdown_connections 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 --2- 172.21.15.32:0/381583401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb3741047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 --2- 172.21.15.32:0/381583401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374100180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 -- 172.21.15.32:0/381583401 >> 172.21.15.32:0/381583401 conn(0x7fb3740fb650 msgr2=0x7fb3740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 -- 172.21.15.32:0/381583401 shutdown_connections 2024-06-03T01:25:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.696+0000 7fb37c08d700 1 -- 172.21.15.32:0/381583401 wait complete. 2024-06-03T01:25:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 Processor -- start 2024-06-03T01:25:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 -- start start 2024-06-03T01:25:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb374102350 con 0x7fb3740ffda0 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb37c08d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741024c0 con 0x7fb374100750 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45818/0 (socket says 172.21.15.32:45818) 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 -- 172.21.15.32:0/216527222 learned_addr learned my addr 172.21.15.32:0/216527222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379628700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379628700 1 -- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 msgr2=0x7fb374101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 -- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 msgr2=0x7fb374101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379628700 1 -- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 msgr2=0x7fb374101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379628700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379628700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 -- 172.21.15.32:0/216527222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700095f0 con 0x7fb3740ffda0 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.697+0000 7fb379e29700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb37000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.698+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37001d070 con 0x7fb3740ffda0 2024-06-03T01:25:53.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.698+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb37000f460 con 0x7fb3740ffda0 2024-06-03T01:25:53.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.698+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3741a2a40 con 0x7fb3740ffda0 2024-06-03T01:25:53.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.698+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb370017630 con 0x7fb3740ffda0 2024-06-03T01:25:53.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.698+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3741a2df0 con 0x7fb3740ffda0 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.699+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37404f2e0 con 0x7fb3740ffda0 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.699+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb37000f5d0 con 0x7fb3740ffda0 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.700+0000 7fb36affd700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 0x7fb360070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.700+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb37008f7b0 con 0x7fb3740ffda0 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.700+0000 7fb379628700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 0x7fb360070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.700+0000 7fb379628700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 0x7fb360070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb364005950 tx=0x7fb3640058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:53.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:53.704+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb3700589a0 con 0x7fb3740ffda0 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi044/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d7833aa000 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: DB pointer 0x55d783394000 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** DB Stats ** 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T01:25:53.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L0 1/0 5.54 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 736.9 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L6 1/0 4.98 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Sum 2/0 10.52 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 736.9 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 736.9 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T01:25:53.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 736.9 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative compaction: 0.01 GB write, 185.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval compaction: 0.01 GB write, 185.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T01:25:53.724 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache BinnedLRUCache@0x55d7832bb090#2 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: starting mon.smithi044 rank 1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi044 fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???) e2 preinit fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).mds e21 new map 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).mds e21 print_map 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: e21 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: legacy client fscid: 1 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Filesystem 'cephfs' (1) 2024-06-03T01:25:53.725 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: fs_name cephfs 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: epoch 21 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: tableserver 0 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: root 0 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: session_timeout 60 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: session_autoclose 300 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_file_size 1099511627776 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_xattr_size 65536 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: required_client_features {} 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: last_failure 0 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: last_failure_osd_epoch 46 2024-06-03T01:25:53.726 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_mds 2 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: in 0,1 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: up {0=14478,1=14498} 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: failed 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: damaged 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: stopped 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_pools [3] 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_pool 2 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: inline_data enabled 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: balancer 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: bal_rank_mask -1 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: standby_count_wanted 1 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: qdb_cluster leader: 0 members: 2024-06-03T01:25:53.727 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Standby daemons: 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 ceph-mon[84214]: mon.smithi044@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 bash[84194]: 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 2024-06-03T01:25:53.728 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:53 smithi044 systemd[1]: Started Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:25:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.006+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb3741a2fe0 con 0x7fb3740ffda0 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.006+0000 7fb36affd700 1 -- 172.21.15.32:0/216527222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+776 (secure 0 0 0) 0x7fb37005c060 con 0x7fb3740ffda0 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:54.007 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 11, 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:25:54.008 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.008+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 msgr2=0x7fb360070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.008+0000 7fb37c08d700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 0x7fb360070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb364005950 tx=0x7fb3640058e0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.008+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 msgr2=0x7fb374101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.008+0000 7fb37c08d700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb37000be20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 shutdown_connections 2024-06-03T01:25:54.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb36006e340 0x7fb360070800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3740ffda0 0x7fb374101840 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 --2- 172.21.15.32:0/216527222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb374100750 0x7fb374101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 >> 172.21.15.32:0/216527222 conn(0x7fb3740fb650 msgr2=0x7fb3740fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 shutdown_connections 2024-06-03T01:25:54.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.009+0000 7fb37c08d700 1 -- 172.21.15.32:0/216527222 wait complete. 2024-06-03T01:25:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 -- 172.21.15.32:0/2297811319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 msgr2=0x7f085c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 --2- 172.21.15.32:0/2297811319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c101280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f084c009910 tx=0x7f084c009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 -- 172.21.15.32:0/2297811319 shutdown_connections 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 --2- 172.21.15.32:0/2297811319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 --2- 172.21.15.32:0/2297811319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c101280 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 -- 172.21.15.32:0/2297811319 >> 172.21.15.32:0/2297811319 conn(0x7f085c0fc750 msgr2=0x7f085c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 -- 172.21.15.32:0/2297811319 shutdown_connections 2024-06-03T01:25:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.117+0000 7f08614a2700 1 -- 172.21.15.32:0/2297811319 wait complete. 2024-06-03T01:25:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 Processor -- start 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 -- start start 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c073b10 con 0x7f085c100ea0 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.118+0000 7f08614a2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c073c50 con 0x7f085c101850 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45830/0 (socket says 172.21.15.32:45830) 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 -- 172.21.15.32:0/1031513825 learned_addr learned my addr 172.21.15.32:0/1031513825 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085a7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085a7fc700 1 -- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 msgr2=0x7f085c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085a7fc700 1 -- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 msgr2=0x7f085c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085a7fc700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T01:25:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085a7fc700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:54.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 -- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 msgr2=0x7f085c0735d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:54.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 -- 172.21.15.32:0/1031513825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c0095f0 con 0x7f085c100ea0 2024-06-03T01:25:54.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.119+0000 7f085affd700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.132+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c01d070 con 0x7f085c100ea0 2024-06-03T01:25:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.132+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c073ed0 con 0x7f085c100ea0 2024-06-03T01:25:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.132+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c1a2e20 con 0x7f085c100ea0 2024-06-03T01:25:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.133+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c109240 con 0x7f085c100ea0 2024-06-03T01:25:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.135+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f084c004fb0 con 0x7f085c100ea0 2024-06-03T01:25:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.135+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c017400 con 0x7f085c100ea0 2024-06-03T01:25:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.137+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f084c004af0 con 0x7f085c100ea0 2024-06-03T01:25:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.137+0000 7f0843fff700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 0x7f08440707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.137+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f084c08fe10 con 0x7f085c100ea0 2024-06-03T01:25:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.137+0000 7f085a7fc700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 0x7f08440707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.138+0000 7f085a7fc700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 0x7f08440707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0850005fd0 tx=0x7f0850005db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.141+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f084c0591d0 con 0x7f085c100ea0 2024-06-03T01:25:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:54 smithi032 ceph-mon[111919]: pgmap v10: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T01:25:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:54 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.395+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f085c04f2e0 con 0x7f085c100ea0 2024-06-03T01:25:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.395+0000 7f0843fff700 1 -- 172.21.15.32:0/1031513825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f084c031770 con 0x7f085c100ea0 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:25:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:25:54.398 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:54.399 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.398+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 msgr2=0x7f08440707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.398+0000 7f08614a2700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 0x7f08440707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0850005fd0 tx=0x7f0850005db0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.398+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 msgr2=0x7f085c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.398+0000 7f08614a2700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c00be20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 shutdown_connections 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f084406e330 0x7f08440707f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c100ea0 0x7f085c073090 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 --2- 172.21.15.32:0/1031513825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f085c101850 0x7f085c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 >> 172.21.15.32:0/1031513825 conn(0x7f085c0fc750 msgr2=0x7f085c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 shutdown_connections 2024-06-03T01:25:54.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.399+0000 7f08614a2700 1 -- 172.21.15.32:0/1031513825 wait complete. 2024-06-03T01:25:54.401 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:25:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:54 smithi044 ceph-mon[84214]: pgmap v10: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T01:25:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:54 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:25:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 -- 172.21.15.32:0/369785335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844106cd0 msgr2=0x7ff8441070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 --2- 172.21.15.32:0/369785335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844106cd0 0x7ff8441070b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff834009910 tx=0x7ff834009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 -- 172.21.15.32:0/369785335 shutdown_connections 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 --2- 172.21.15.32:0/369785335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844068580 0x7ff8440689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 --2- 172.21.15.32:0/369785335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844106cd0 0x7ff8441070b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 -- 172.21.15.32:0/369785335 >> 172.21.15.32:0/369785335 conn(0x7ff844075670 msgr2=0x7ff844075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 -- 172.21.15.32:0/369785335 shutdown_connections 2024-06-03T01:25:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.508+0000 7ff84b04b700 1 -- 172.21.15.32:0/369785335 wait complete. 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 Processor -- start 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 -- start start 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84419f420 con 0x7ff844068580 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff84b04b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441988d0 con 0x7ff844106cd0 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff848de7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff848de7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45850/0 (socket says 172.21.15.32:45850) 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff848de7700 1 -- 172.21.15.32:0/76903881 learned_addr learned my addr 172.21.15.32:0/76903881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff843fff700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff843fff700 1 -- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 msgr2=0x7ff84419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff843fff700 1 -- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 msgr2=0x7ff84419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff843fff700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.509+0000 7ff843fff700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff848de7700 1 -- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 msgr2=0x7ff84419ed90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff848de7700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff848de7700 1 -- 172.21.15.32:0/76903881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8340095f0 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff848de7700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff834006010 tx=0x7ff83400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff83401d070 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff844198b50 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff83400f460 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff834017630 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.510+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff844199040 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.511+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84404f2e0 con 0x7ff844068580 2024-06-03T01:25:54.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.512+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff83400f5d0 con 0x7ff844068580 2024-06-03T01:25:54.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.512+0000 7ff841ffb700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 0x7ff82c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.512+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff83408f820 con 0x7ff844068580 2024-06-03T01:25:54.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.512+0000 7ff843fff700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 0x7ff82c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.513+0000 7ff843fff700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 0x7ff82c070800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff838005950 tx=0x7ff8380058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.517+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff83405c4c0 con 0x7ff844068580 2024-06-03T01:25:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.741+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff844199320 con 0x7ff82c06e340 2024-06-03T01:25:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.742+0000 7ff841ffb700 1 -- 172.21.15.32:0/76903881 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff844199320 con 0x7ff82c06e340 2024-06-03T01:25:54.742 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "3/23 daemons upgraded", 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mon daemons", 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:25:54.743 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 msgr2=0x7ff82c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 0x7ff82c070800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff838005950 tx=0x7ff8380058e0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 msgr2=0x7ff84419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff834006010 tx=0x7ff83400be20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 shutdown_connections 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff82c06e340 0x7ff82c070800 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff844068580 0x7ff84419e850 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.744+0000 7ff84b04b700 1 --2- 172.21.15.32:0/76903881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff844106cd0 0x7ff84419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.745+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 >> 172.21.15.32:0/76903881 conn(0x7ff844075670 msgr2=0x7ff8440fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.745+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 shutdown_connections 2024-06-03T01:25:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.745+0000 7ff84b04b700 1 -- 172.21.15.32:0/76903881 wait complete. 2024-06-03T01:25:54.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 -- 172.21.15.32:0/2472805125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0068580 msgr2=0x7f63d00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 --2- 172.21.15.32:0/2472805125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0068580 0x7f63d00689e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009910 tx=0x7f63c4009c20 comp rx=0 tx=0).stop 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 -- 172.21.15.32:0/2472805125 shutdown_connections 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 --2- 172.21.15.32:0/2472805125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0068580 0x7f63d00689e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 --2- 172.21.15.32:0/2472805125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0106cd0 0x7f63d01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.853+0000 7f63d518f700 1 -- 172.21.15.32:0/2472805125 >> 172.21.15.32:0/2472805125 conn(0x7f63d0075670 msgr2=0x7f63d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.854+0000 7f63d518f700 1 -- 172.21.15.32:0/2472805125 shutdown_connections 2024-06-03T01:25:54.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.854+0000 7f63d518f700 1 -- 172.21.15.32:0/2472805125 wait complete. 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.854+0000 7f63d518f700 1 Processor -- start 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.854+0000 7f63d518f700 1 -- start start 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63d518f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63d518f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63d518f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d019d200 con 0x7f63d0106cd0 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63d518f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d01966b0 con 0x7f63d0068580 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45876/0 (socket says 172.21.15.32:45876) 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 -- 172.21.15.32:0/4153960194 learned_addr learned my addr 172.21.15.32:0/4153960194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ced9d700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ced9d700 1 -- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 msgr2=0x7f63d019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-03T01:25:54.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ced9d700 1 -- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 msgr2=0x7f63d019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ced9d700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ced9d700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 -- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 msgr2=0x7f63d019c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 -- 172.21.15.32:0/4153960194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40095f0 con 0x7f63d0106cd0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.855+0000 7f63ce59c700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c4006010 tx=0x7f63c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.856+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c401d070 con 0x7f63d0106cd0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.856+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63c400f460 con 0x7f63d0106cd0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.856+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d0196930 con 0x7f63d0106cd0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.856+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d0196e20 con 0x7f63d0106cd0 2024-06-03T01:25:54.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.856+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c4017610 con 0x7f63d0106cd0 2024-06-03T01:25:54.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.857+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d01091d0 con 0x7f63d0106cd0 2024-06-03T01:25:54.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.857+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f63c4017880 con 0x7f63d0106cd0 2024-06-03T01:25:54.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.858+0000 7f63b7fff700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 0x7f63b80707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:54.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.858+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f63c408f7d0 con 0x7f63d0106cd0 2024-06-03T01:25:54.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.858+0000 7f63ced9d700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 0x7f63b80707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:54.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.859+0000 7f63ced9d700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 0x7f63b80707f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63c0005920 tx=0x7f63c000b2c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:54.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:54.863+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f63c405c620 con 0x7f63d0106cd0 2024-06-03T01:25:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.170+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f63d0197c60 con 0x7f63d0106cd0 2024-06-03T01:25:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.170+0000 7f63b7fff700 1 -- 172.21.15.32:0/4153960194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f63c408de60 con 0x7f63d0106cd0 2024-06-03T01:25:55.171 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.171 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.171 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.172+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 msgr2=0x7f63b80707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.172+0000 7f63d518f700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 0x7f63b80707f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63c0005920 tx=0x7f63c000b2c0 comp rx=0 tx=0).stop 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 msgr2=0x7f63d019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c4006010 tx=0x7f63c400be20 comp rx=0 tx=0).stop 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 shutdown_connections 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f63b806e330 0x7f63b80707f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63d0068580 0x7f63d019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 --2- 172.21.15.32:0/4153960194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d0106cd0 0x7f63d019cb70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 >> 172.21.15.32:0/4153960194 conn(0x7f63d0075670 msgr2=0x7f63d00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 shutdown_connections 2024-06-03T01:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:25:55.173+0000 7f63d518f700 1 -- 172.21.15.32:0/4153960194 wait complete. 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: mgrmap e37: smithi032.kfcfaa(active, since 17s), standbys: smithi044.wszjlw 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:55.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: mon.smithi032 calling monitor election 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: mon.smithi044 calling monitor election 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: monmap epoch 3 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: last_changed 2024-06-03T01:25:54.121302+0000 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: created 2024-06-03T01:11:10.279598+0000 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: min_mon_release 19 (squid) 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: election_strategy: 1 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:55.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: mgrmap e37: smithi032.kfcfaa(active, since 17s), standbys: smithi044.wszjlw 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1031513825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:25:55.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:55 smithi044 ceph-mon[84214]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:55.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: mgrmap e37: smithi032.kfcfaa(active, since 17s), standbys: smithi044.wszjlw 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: mon.smithi032 calling monitor election 2024-06-03T01:25:55.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: mon.smithi044 calling monitor election 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: mon.smithi032 is new leader, mons smithi032,smithi044 in quorum (ranks 0,1) 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: monmap epoch 3 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: fsid 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: last_changed 2024-06-03T01:25:54.121302+0000 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: created 2024-06-03T01:11:10.279598+0000 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: min_mon_release 19 (squid) 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: election_strategy: 1 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.smithi032 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: osdmap e49: 6 total, 6 up, 6 in 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: mgrmap e37: smithi032.kfcfaa(active, since 17s), standbys: smithi044.wszjlw 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.587 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1031513825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:25:55.588 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:55 smithi032 ceph-mon[111919]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:25:56.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:56 smithi044 ceph-mon[84214]: pgmap v12: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T01:25:56.432 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:56 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/4153960194' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:25:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:56 smithi032 ceph-mon[111919]: pgmap v12: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T01:25:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:56 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/4153960194' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:25:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:58 smithi032 ceph-mon[111919]: pgmap v13: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:58 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:58 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:58 smithi044 ceph-mon[84214]: pgmap v13: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:25:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:58 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:58 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:59.567 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:59.567 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:25:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:59.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:25:59.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:25:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:00.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:00 smithi044 ceph-mon[84214]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:00 smithi032 ceph-mon[111919]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:26:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:26:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:26:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:02.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:02.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:26:02.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:03.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:03 smithi044 ceph-mon[84214]: Detected new or changed devices on smithi044 2024-06-03T01:26:03.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:03 smithi044 ceph-mon[84214]: Reconfiguring mon.smithi032 (monmap changed)... 2024-06-03T01:26:03.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:03 smithi044 ceph-mon[84214]: Reconfiguring daemon mon.smithi032 on smithi032 2024-06-03T01:26:03.794 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:03 smithi032 ceph-mon[111919]: Detected new or changed devices on smithi044 2024-06-03T01:26:03.794 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:03 smithi032 ceph-mon[111919]: Reconfiguring mon.smithi032 (monmap changed)... 2024-06-03T01:26:03.794 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:03 smithi032 ceph-mon[111919]: Reconfiguring daemon mon.smithi032 on smithi032 2024-06-03T01:26:04.714 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: pgmap v16: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:04.714 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:04.714 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:04.715 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:26:04.715 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:26:04.715 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:04.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: pgmap v16: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:04.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:04.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.kfcfaa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:26:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:26:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:05.453 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:05 smithi032 ceph-mon[111919]: Reconfiguring mgr.smithi032.kfcfaa (monmap changed)... 2024-06-03T01:26:05.453 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:05 smithi032 ceph-mon[111919]: Reconfiguring daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:26:05.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:05 smithi044 ceph-mon[84214]: Reconfiguring mgr.smithi032.kfcfaa (monmap changed)... 2024-06-03T01:26:05.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:05 smithi044 ceph-mon[84214]: Reconfiguring daemon mgr.smithi032.kfcfaa on smithi032 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: Reconfiguring ceph-exporter.smithi032 (monmap changed)... 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:06.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:26:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: Unable to update caps for client.ceph-exporter.smithi032 2024-06-03T01:26:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi032"}]: dispatch 2024-06-03T01:26:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:06 smithi032 ceph-mon[111919]: Reconfiguring daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:26:06.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:06.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: Reconfiguring ceph-exporter.smithi032 (monmap changed)... 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: Unable to update caps for client.ceph-exporter.smithi032 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi032"}]: dispatch 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:06.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:06 smithi044 ceph-mon[84214]: Reconfiguring daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:26:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:07.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:07.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:07.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:07.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:07.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:08.809 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:08 smithi032 ceph-mon[111919]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:08.809 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:08 smithi032 ceph-mon[111919]: Reconfiguring crash.smithi032 (monmap changed)... 2024-06-03T01:26:08.810 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:08 smithi032 ceph-mon[111919]: Reconfiguring daemon crash.smithi032 on smithi032 2024-06-03T01:26:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:08 smithi044 ceph-mon[84214]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:08 smithi044 ceph-mon[84214]: Reconfiguring crash.smithi032 (monmap changed)... 2024-06-03T01:26:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:08 smithi044 ceph-mon[84214]: Reconfiguring daemon crash.smithi032 on smithi032 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: Reconfiguring osd.0 (monmap changed)... 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.0 on smithi032 2024-06-03T01:26:10.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:09 smithi032 ceph-mon[111919]: pgmap v19: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:10.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: Reconfiguring osd.0 (monmap changed)... 2024-06-03T01:26:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:26:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.0 on smithi032 2024-06-03T01:26:10.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:09 smithi044 ceph-mon[84214]: pgmap v19: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:11.760 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:11.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:11.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: Reconfiguring osd.1 (monmap changed)... 2024-06-03T01:26:11.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:26:11.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:11.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:11 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.1 on smithi032 2024-06-03T01:26:11.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: Reconfiguring osd.1 (monmap changed)... 2024-06-03T01:26:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:26:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:11.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:11 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.1 on smithi032 2024-06-03T01:26:12.817 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:12 smithi032 ceph-mon[111919]: pgmap v20: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:12.817 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:12.818 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:12.818 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:26:12.818 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:12.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:12 smithi044 ceph-mon[84214]: pgmap v20: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:12.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:12.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:12.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:26:12.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:13.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:13 smithi032 ceph-mon[111919]: Reconfiguring osd.2 (monmap changed)... 2024-06-03T01:26:13.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:13 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.2 on smithi032 2024-06-03T01:26:13.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:13 smithi044 ceph-mon[84214]: Reconfiguring osd.2 (monmap changed)... 2024-06-03T01:26:13.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:13 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.2 on smithi032 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: pgmap v21: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: Reconfiguring mds.cephfs.smithi032.itqzrm (monmap changed)... 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:14.705 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:14.706 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:14 smithi032 ceph-mon[111919]: Reconfiguring daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:26:14.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: pgmap v21: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: Reconfiguring mds.cephfs.smithi032.itqzrm (monmap changed)... 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:14.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:14 smithi044 ceph-mon[84214]: Reconfiguring daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:26:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:16.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: Reconfiguring mds.cephfs.smithi032.tegslf (monmap changed)... 2024-06-03T01:26:16.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:16.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:16.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:16 smithi032 ceph-mon[111919]: Reconfiguring daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:26:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: Reconfiguring mds.cephfs.smithi032.tegslf (monmap changed)... 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:16 smithi044 ceph-mon[84214]: Reconfiguring daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: Unable to update caps for client.ceph-exporter.smithi044 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi044"}]: dispatch 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:17.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:17 smithi044 ceph-mon[84214]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: Unable to update caps for client.ceph-exporter.smithi044 2024-06-03T01:26:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi044"}]: dispatch 2024-06-03T01:26:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:17 smithi032 ceph-mon[111919]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:26:18.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:18 smithi044 ceph-mon[84214]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:18 smithi032 ceph-mon[111919]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:19.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:19.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:19 smithi044 ceph-mon[84214]: Reconfiguring crash.smithi044 (monmap changed)... 2024-06-03T01:26:19.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:19 smithi044 ceph-mon[84214]: Reconfiguring daemon crash.smithi044 on smithi044 2024-06-03T01:26:20.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:19 smithi032 ceph-mon[111919]: Reconfiguring crash.smithi044 (monmap changed)... 2024-06-03T01:26:20.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:19 smithi032 ceph-mon[111919]: Reconfiguring daemon crash.smithi044 on smithi044 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: pgmap v24: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:26:20.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: pgmap v24: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.wszjlw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T01:26:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:21 smithi044 ceph-mon[84214]: Reconfiguring mgr.smithi044.wszjlw (monmap changed)... 2024-06-03T01:26:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:21 smithi044 ceph-mon[84214]: Reconfiguring daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:26:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:21 smithi032 ceph-mon[111919]: Reconfiguring mgr.smithi044.wszjlw (monmap changed)... 2024-06-03T01:26:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:21 smithi032 ceph-mon[111919]: Reconfiguring daemon mgr.smithi044.wszjlw on smithi044 2024-06-03T01:26:22.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: pgmap v25: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: Reconfiguring mon.smithi044 (monmap changed)... 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: Reconfiguring daemon mon.smithi044 on smithi044 2024-06-03T01:26:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: pgmap v25: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: Reconfiguring mon.smithi044 (monmap changed)... 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T01:26:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:23.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: Reconfiguring daemon mon.smithi044 on smithi044 2024-06-03T01:26:23.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: pgmap v26: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: Reconfiguring osd.3 (monmap changed)... 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:24.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:24 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.3 on smithi044 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: pgmap v26: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: Reconfiguring osd.3 (monmap changed)... 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:24 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.3 on smithi044 2024-06-03T01:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- 172.21.15.32:0/1900123694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101850 msgr2=0x7f3af01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/1900123694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101850 0x7f3af01058a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009910 tx=0x7f3ae0009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- 172.21.15.32:0/1900123694 shutdown_connections 2024-06-03T01:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/1900123694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101850 0x7f3af01058a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/1900123694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0100ea0 0x7f3af0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- 172.21.15.32:0/1900123694 >> 172.21.15.32:0/1900123694 conn(0x7f3af00fc750 msgr2=0x7f3af00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- 172.21.15.32:0/1900123694 shutdown_connections 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- 172.21.15.32:0/1900123694 wait complete. 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 Processor -- start 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.292+0000 7f3af7ffd700 1 -- start start 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af7ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af7ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101850 0x7f3af019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af7ffd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af019d2d0 con 0x7f3af0100ea0 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af7ffd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0196780 con 0x7f3af0101850 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55772/0 (socket says 172.21.15.32:55772) 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 -- 172.21.15.32:0/3706663023 learned_addr learned my addr 172.21.15.32:0/3706663023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5598700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101850 0x7f3af019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 -- 172.21.15.32:0/3706663023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101850 msgr2=0x7f3af019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101850 0x7f3af019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 -- 172.21.15.32:0/3706663023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00095f0 con 0x7f3af0100ea0 2024-06-03T01:26:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3af5d99700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00ef10 tx=0x7f3aec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aec00c9f0 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.293+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3aec00f040 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.294+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af0196a00 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.294+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aec0146b0 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.294+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af0196f20 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.295+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af01092a0 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.295+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f3aec00cb50 con 0x7f3af0100ea0 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.296+0000 7f3ae6ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 0x7f3adc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.296+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3aec08e6f0 con 0x7f3af0100ea0 2024-06-03T01:26:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.296+0000 7f3af5598700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 0x7f3adc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.296+0000 7f3af5598700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 0x7f3adc070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00059d0 tx=0x7f3ae0005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.300+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3aec05b370 con 0x7f3af0100ea0 2024-06-03T01:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.537+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3af0197da0 con 0x7f3adc06e340 2024-06-03T01:26:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.538+0000 7f3ae6ffd700 1 -- 172.21.15.32:0/3706663023 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3af0197da0 con 0x7f3adc06e340 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 msgr2=0x7f3adc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 0x7f3adc070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00059d0 tx=0x7f3ae0005940 comp rx=0 tx=0).stop 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 msgr2=0x7f3af019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00ef10 tx=0x7f3aec00c3a0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 shutdown_connections 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3adc06e340 0x7f3adc070800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0100ea0 0x7f3af019c700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 --2- 172.21.15.32:0/3706663023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101850 0x7f3af019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 >> 172.21.15.32:0/3706663023 conn(0x7f3af00fc750 msgr2=0x7f3af00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 shutdown_connections 2024-06-03T01:26:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.541+0000 7f3af7ffd700 1 -- 172.21.15.32:0/3706663023 wait complete. 2024-06-03T01:26:25.551 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:26:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- 172.21.15.32:0/1584692750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 msgr2=0x7efdac1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 --2- 172.21.15.32:0/1584692750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac1021e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009910 tx=0x7efd9c009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- 172.21.15.32:0/1584692750 shutdown_connections 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 --2- 172.21.15.32:0/1584692750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 0x7efdac10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 --2- 172.21.15.32:0/1584692750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac1021e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- 172.21.15.32:0/1584692750 >> 172.21.15.32:0/1584692750 conn(0x7efdac0fb650 msgr2=0x7efdac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- 172.21.15.32:0/1584692750 shutdown_connections 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- 172.21.15.32:0/1584692750 wait complete. 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 Processor -- start 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.651+0000 7efdb2756700 1 -- start start 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 0x7efdac0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac1004e0 con 0x7efdac101e00 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac100620 con 0x7efdac102720 2024-06-03T01:26:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55786/0 (socket says 172.21.15.32:55786) 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 -- 172.21.15.32:0/881007977 learned_addr learned my addr 172.21.15.32:0/881007977 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efda3fff700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 0x7efdac0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 -- 172.21.15.32:0/881007977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 msgr2=0x7efdac0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 0x7efdac0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 -- 172.21.15.32:0/881007977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0095f0 con 0x7efdac101e00 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdabfff700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005b40 tx=0x7efd9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c01d070 con 0x7efdac101e00 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd9c0048e0 con 0x7efdac101e00 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac1008a0 con 0x7efdac101e00 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c003ab0 con 0x7efdac101e00 2024-06-03T01:26:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.652+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac1a2e30 con 0x7efdac101e00 2024-06-03T01:26:25.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.654+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac108310 con 0x7efdac101e00 2024-06-03T01:26:25.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.654+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7efd9c0041f0 con 0x7efdac101e00 2024-06-03T01:26:25.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.654+0000 7efda9ffb700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 0x7efd98070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:25.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.655+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efd9c08f9a0 con 0x7efdac101e00 2024-06-03T01:26:25.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.655+0000 7efda3fff700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 0x7efd98070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:25.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.655+0000 7efda3fff700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 0x7efd98070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdac101200 tx=0x7efd94009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.659+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd9c058b10 con 0x7efdac101e00 2024-06-03T01:26:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.890+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdac0611d0 con 0x7efd9806e340 2024-06-03T01:26:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.891+0000 7efda9ffb700 1 -- 172.21.15.32:0/881007977 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efdac0611d0 con 0x7efd9806e340 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.893+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 msgr2=0x7efd98070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.893+0000 7efdb2756700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 0x7efd98070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdac101200 tx=0x7efd94009190 comp rx=0 tx=0).stop 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.893+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 msgr2=0x7efdac0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.893+0000 7efdb2756700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005b40 tx=0x7efd9c00bd70 comp rx=0 tx=0).stop 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 shutdown_connections 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efd9806e340 0x7efd98070800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdac101e00 0x7efdac0ffa60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 --2- 172.21.15.32:0/881007977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdac102720 0x7efdac0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 >> 172.21.15.32:0/881007977 conn(0x7efdac0fb650 msgr2=0x7efdac105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 shutdown_connections 2024-06-03T01:26:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:25.894+0000 7efdb2756700 1 -- 172.21.15.32:0/881007977 wait complete. 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 -- 172.21.15.32:0/4022210218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073a50 msgr2=0x7f29a4111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 --2- 172.21.15.32:0/4022210218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073a50 0x7f29a4111700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009910 tx=0x7f29a0009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 -- 172.21.15.32:0/4022210218 shutdown_connections 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 --2- 172.21.15.32:0/4022210218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073a50 0x7f29a4111700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 --2- 172.21.15.32:0/4022210218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073130 0x7f29a4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 -- 172.21.15.32:0/4022210218 >> 172.21.15.32:0/4022210218 conn(0x7f29a40fc740 msgr2=0x7f29a40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 -- 172.21.15.32:0/4022210218 shutdown_connections 2024-06-03T01:26:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 -- 172.21.15.32:0/4022210218 wait complete. 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.003+0000 7f29ab89f700 1 Processor -- start 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29ab89f700 1 -- start start 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29ab89f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29ab89f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073a50 0x7f29a419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29ab89f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a419da40 con 0x7f29a4073130 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29ab89f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a41a17d0 con 0x7f29a4073a50 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55812/0 (socket says 172.21.15.32:55812) 2024-06-03T01:26:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 -- 172.21.15.32:0/3458700910 learned_addr learned my addr 172.21.15.32:0/3458700910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a8e3a700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073a50 0x7f29a419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 -- 172.21.15.32:0/3458700910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073a50 msgr2=0x7f29a419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073a50 0x7f29a419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 -- 172.21.15.32:0/3458700910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00095f0 con 0x7f29a4073130 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.004+0000 7f29a963b700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f299400e920 tx=0x7f299400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299400c9d0 con 0x7f29a4073130 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2994010430 con 0x7f29a4073130 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a41a1ab0 con 0x7f29a4073130 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2994018630 con 0x7f29a4073130 2024-06-03T01:26:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a41a2000 con 0x7f29a4073130 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.005+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a410ee80 con 0x7f29a4073130 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.006+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f29940105a0 con 0x7f29a4073130 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.007+0000 7f299a7fc700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 0x7f2990070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.007+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2994014070 con 0x7f29a4073130 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.007+0000 7f29a8e3a700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 0x7f2990070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.007+0000 7f29a8e3a700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 0x7f2990070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29a419e440 tx=0x7f29a0005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.011+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2994053850 con 0x7f29a4073130 2024-06-03T01:26:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.229+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29a41a2330 con 0x7f299006e340 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: pgmap v27: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: Reconfiguring osd.4 (monmap changed)... 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:26:26.229 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:26.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.4 on smithi044 2024-06-03T01:26:26.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:26 smithi032 ceph-mon[111919]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.239+0000 7f299a7fc700 1 -- 172.21.15.32:0/3458700910 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f29a41a2330 con 0x7f299006e340 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (2m) 43s ago 13m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (13m) 43s ago 13m 8090k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (12m) 29s ago 12m 7936k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (13m) 43s ago 13m 6660k - 18.2.1 5be31c24972a 73741c6ba5d3 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (12m) 29s ago 12m 6660k - 18.2.1 5be31c24972a ca111c04dd18 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (2m) 43s ago 13m 76.2M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (8m) 43s ago 8m 125M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (8m) 43s ago 8m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (8m) 29s ago 8m 15.7M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:26:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (7m) 29s ago 7m 17.3M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (4m) 43s ago 15m 475M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (3m) 29s ago 12m 449M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (59s) 43s ago 15m 29.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (33s) 29s ago 12m 27.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (3m) 43s ago 13m 15.2M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (3m) 29s ago 12m 15.9M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (11m) 43s ago 11m 239M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (11m) 43s ago 11m 241M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (10m) 43s ago 10m 204M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (10m) 29s ago 10m 268M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (10m) 29s ago 10m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (9m) 29s ago 9m 231M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:26:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 43s ago 13m 71.7M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:26:26.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 msgr2=0x7f2990070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 0x7f2990070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29a419e440 tx=0x7f29a0005940 comp rx=0 tx=0).stop 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 msgr2=0x7f29a419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f299400e920 tx=0x7f299400ec30 comp rx=0 tx=0).stop 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 shutdown_connections 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f299006e340 0x7f2990070800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a4073130 0x7f29a419ce20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 --2- 172.21.15.32:0/3458700910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a4073a50 0x7f29a419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 >> 172.21.15.32:0/3458700910 conn(0x7f29a40fc740 msgr2=0x7f29a4103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 shutdown_connections 2024-06-03T01:26:26.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.242+0000 7f29ab89f700 1 -- 172.21.15.32:0/3458700910 wait complete. 2024-06-03T01:26:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 -- 172.21.15.32:0/915884893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 msgr2=0x7fe0f4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 --2- 172.21.15.32:0/915884893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f4100180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009910 tx=0x7fe0e4009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 -- 172.21.15.32:0/915884893 shutdown_connections 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 --2- 172.21.15.32:0/915884893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 0x7fe0f41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 --2- 172.21.15.32:0/915884893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f4100180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 -- 172.21.15.32:0/915884893 >> 172.21.15.32:0/915884893 conn(0x7fe0f40fb650 msgr2=0x7fe0f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 -- 172.21.15.32:0/915884893 shutdown_connections 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.349+0000 7fe0fa518700 1 -- 172.21.15.32:0/915884893 wait complete. 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 Processor -- start 2024-06-03T01:26:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 -- start start 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 0x7fe0f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f419d220 con 0x7fe0f40ffda0 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0fa518700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4196760 con 0x7fe0f4100750 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55830/0 (socket says 172.21.15.32:55830) 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 -- 172.21.15.32:0/2389666793 learned_addr learned my addr 172.21.15.32:0/2389666793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f37fe700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 0x7fe0f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 -- 172.21.15.32:0/2389666793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 msgr2=0x7fe0f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 0x7fe0f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.350+0000 7fe0f3fff700 1 -- 172.21.15.32:0/2389666793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e40095f0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0f3fff700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e401d070 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0e400f460 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f41969e0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e4017630 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.351+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f4196ed0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.352+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f404f2e0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.352+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fe0e400f5d0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.353+0000 7fe0f17fa700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 0x7fe0dc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.353+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe0e408f8c0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.353+0000 7fe0f37fe700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 0x7fe0dc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.353+0000 7fe0f37fe700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 0x7fe0dc070800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4197bd0 tx=0x7fe0e800a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.358+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe0e405c560 con 0x7fe0f40ffda0 2024-06-03T01:26:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: pgmap v27: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: Reconfiguring osd.4 (monmap changed)... 2024-06-03T01:26:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:26:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.4 on smithi044 2024-06-03T01:26:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:26 smithi044 ceph-mon[84214]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:26.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.648+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0f41970f0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.649+0000 7fe0f17fa700 1 -- 172.21.15.32:0/2389666793 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+685 (secure 0 0 0) 0x7fe0e405c0f0 con 0x7fe0f40ffda0 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:26:26.649 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:26:26.650 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 msgr2=0x7fe0dc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 0x7fe0dc070800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4197bd0 tx=0x7fe0e800a380 comp rx=0 tx=0).stop 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 msgr2=0x7fe0f419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e400bd70 comp rx=0 tx=0).stop 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 shutdown_connections 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe0dc06e340 0x7fe0dc070800 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0f40ffda0 0x7fe0f419c6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 --2- 172.21.15.32:0/2389666793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f4100750 0x7fe0f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 >> 172.21.15.32:0/2389666793 conn(0x7fe0f40fb650 msgr2=0x7fe0f4103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 shutdown_connections 2024-06-03T01:26:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.651+0000 7fe0fa518700 1 -- 172.21.15.32:0/2389666793 wait complete. 2024-06-03T01:26:26.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 -- 172.21.15.32:0/3036706807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c068580 msgr2=0x7f1c1c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/3036706807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c068580 0x7f1c1c0689e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009910 tx=0x7f1c04009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:26.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 -- 172.21.15.32:0/3036706807 shutdown_connections 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/3036706807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c068580 0x7f1c1c0689e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/3036706807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 -- 172.21.15.32:0/3036706807 >> 172.21.15.32:0/3036706807 conn(0x7f1c1c075670 msgr2=0x7f1c1c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 -- 172.21.15.32:0/3036706807 shutdown_connections 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.759+0000 7f1c22a6e700 1 -- 172.21.15.32:0/3036706807 wait complete. 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 Processor -- start 2024-06-03T01:26:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 -- start start 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c068580 0x7f1c1c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c100770 con 0x7f1c1c106cd0 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c22a6e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c1008b0 con 0x7f1c1c068580 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c2080a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c068580 0x7f1c1c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55860/0 (socket says 172.21.15.32:55860) 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.760+0000 7f1c1bfff700 1 -- 172.21.15.32:0/6254526 learned_addr learned my addr 172.21.15.32:0/6254526 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:26.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c1bfff700 1 -- 172.21.15.32:0/6254526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c068580 msgr2=0x7f1c1c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c1bfff700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c068580 0x7f1c1c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c1bfff700 1 -- 172.21.15.32:0/6254526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040095f0 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c1bfff700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c040092e0 tx=0x7f1c0400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0401d070 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c100b30 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c04005e00 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c04017400 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.761+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c1a2d10 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.762+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c1091d0 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.763+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f1c04003a60 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.763+0000 7f1c19ffb700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 0x7f1c080707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.763+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1c0408f7e0 con 0x7f1c1c106cd0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.763+0000 7f1c2080a700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 0x7f1c080707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:26.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.764+0000 7f1c2080a700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 0x7f1c080707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009550 tx=0x7f1c0c0091b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:26.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:26.768+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1c04058950 con 0x7f1c1c106cd0 2024-06-03T01:26:27.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.039+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1c1c108bb0 con 0x7f1c1c106cd0 2024-06-03T01:26:27.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.040+0000 7f1c19ffb700 1 -- 172.21.15.32:0/6254526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f1c0405c010 con 0x7f1c1c106cd0 2024-06-03T01:26:27.040 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:26:27.040 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:26:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:26:27.042 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:27.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 msgr2=0x7f1c080707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 0x7f1c080707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009550 tx=0x7f1c0c0091b0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 msgr2=0x7f1c1c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c040092e0 tx=0x7f1c0400bd70 comp rx=0 tx=0).stop 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 shutdown_connections 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f1c0806e330 0x7f1c080707f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c1c068580 0x7f1c1c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 --2- 172.21.15.32:0/6254526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c1c106cd0 0x7f1c1c100230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 >> 172.21.15.32:0/6254526 conn(0x7f1c1c075670 msgr2=0x7f1c1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 shutdown_connections 2024-06-03T01:26:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.043+0000 7f1c22a6e700 1 -- 172.21.15.32:0/6254526 wait complete. 2024-06-03T01:26:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:26:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 -- 172.21.15.32:0/1354934494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc101e60 msgr2=0x7febbc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 --2- 172.21.15.32:0/1354934494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc101e60 0x7febbc102240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febb800b0a0 tx=0x7febb800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 -- 172.21.15.32:0/1354934494 shutdown_connections 2024-06-03T01:26:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 --2- 172.21.15.32:0/1354934494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc102780 0x7febbc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 --2- 172.21.15.32:0/1354934494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc101e60 0x7febbc102240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 -- 172.21.15.32:0/1354934494 >> 172.21.15.32:0/1354934494 conn(0x7febbc0fb650 msgr2=0x7febbc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 -- 172.21.15.32:0/1354934494 shutdown_connections 2024-06-03T01:26:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.153+0000 7febc3a42700 1 -- 172.21.15.32:0/1354934494 wait complete. 2024-06-03T01:26:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 Processor -- start 2024-06-03T01:26:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 -- start start 2024-06-03T01:26:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc101e60 0x7febbc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc19f4a0 con 0x7febbc102780 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc3a42700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc1989a0 con 0x7febbc101e60 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40144/0 (socket says 172.21.15.32:40144) 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc17de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc101e60 0x7febbc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 -- 172.21.15.32:0/2392313341 learned_addr learned my addr 172.21.15.32:0/2392313341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 -- 172.21.15.32:0/2392313341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc101e60 msgr2=0x7febbc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc101e60 0x7febbc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.154+0000 7febc0fdd700 1 -- 172.21.15.32:0/2392313341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8009d00 con 0x7febbc102780 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febc0fdd700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febac00d8d0 tx=0x7febac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac0041f0 con 0x7febbc102780 2024-06-03T01:26:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febac007620 con 0x7febbc102780 2024-06-03T01:26:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc198c80 con 0x7febbc102780 2024-06-03T01:26:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac0056b0 con 0x7febbc102780 2024-06-03T01:26:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.155+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc1991d0 con 0x7febbc102780 2024-06-03T01:26:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.156+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc10a770 con 0x7febbc102780 2024-06-03T01:26:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.156+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7febac00f7d0 con 0x7febbc102780 2024-06-03T01:26:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.157+0000 7febb27fc700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 0x7feba80707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.157+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7febac08e240 con 0x7febbc102780 2024-06-03T01:26:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.157+0000 7febc17de700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 0x7feba80707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.157+0000 7febc17de700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 0x7feba80707f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febb8000f80 tx=0x7febb80061f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:27.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.162+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7febac05ae40 con 0x7febbc102780 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2389666793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:27 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/6254526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:26:27.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.387+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febbc1993f0 con 0x7feba806e330 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.388+0000 7febb27fc700 1 -- 172.21.15.32:0/2392313341 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7febbc1993f0 con 0x7feba806e330 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:26:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "4/23 daemons upgraded", 2024-06-03T01:26:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mon daemons", 2024-06-03T01:26:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:26:27.389 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:26:27.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 msgr2=0x7feba80707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 0x7feba80707f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febb8000f80 tx=0x7febb80061f0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 msgr2=0x7febbc19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febac00d8d0 tx=0x7febac00dbe0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 shutdown_connections 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7feba806e330 0x7feba80707f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febbc101e60 0x7febbc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 --2- 172.21.15.32:0/2392313341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc102780 0x7febbc19ee10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 >> 172.21.15.32:0/2392313341 conn(0x7febbc0fb650 msgr2=0x7febbc100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 shutdown_connections 2024-06-03T01:26:27.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.390+0000 7febc3a42700 1 -- 172.21.15.32:0/2392313341 wait complete. 2024-06-03T01:26:27.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:27.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2389666793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:27.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:27 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/6254526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:26:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/1583374088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 msgr2=0x7f0a78100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/1583374088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a78100080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009910 tx=0x7f0a68009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/1583374088 shutdown_connections 2024-06-03T01:26:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/1583374088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a78100080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/1583374088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 0x7f0a78105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/1583374088 >> 172.21.15.32:0/1583374088 conn(0x7f0a78075670 msgr2=0x7f0a78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/1583374088 shutdown_connections 2024-06-03T01:26:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.498+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/1583374088 wait complete. 2024-06-03T01:26:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 Processor -- start 2024-06-03T01:26:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 -- start start 2024-06-03T01:26:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 0x7f0a7819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819d200 con 0x7f0a780ffc20 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7f4e6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781966b0 con 0x7f0a78105bd0 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7d282700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7d282700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40152/0 (socket says 172.21.15.32:40152) 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7d282700 1 -- 172.21.15.32:0/778363336 learned_addr learned my addr 172.21.15.32:0/778363336 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.499+0000 7f0a7ca81700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 0x7f0a7819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7d282700 1 -- 172.21.15.32:0/778363336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 msgr2=0x7f0a7819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7d282700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 0x7f0a7819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7d282700 1 -- 172.21.15.32:0/778363336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680095f0 con 0x7f0a780ffc20 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7d282700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e920 tx=0x7f0a7400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7400c9d0 con 0x7f0a780ffc20 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a74010430 con 0x7f0a780ffc20 2024-06-03T01:26:27.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a78196990 con 0x7f0a780ffc20 2024-06-03T01:26:27.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a74018630 con 0x7f0a780ffc20 2024-06-03T01:26:27.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.500+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a78196ee0 con 0x7f0a780ffc20 2024-06-03T01:26:27.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.501+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7804f2e0 con 0x7f0a780ffc20 2024-06-03T01:26:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.501+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0a740105a0 con 0x7f0a780ffc20 2024-06-03T01:26:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.502+0000 7f0a6e7fc700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 0x7f0a64070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.502+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0a74014070 con 0x7f0a780ffc20 2024-06-03T01:26:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.502+0000 7f0a7ca81700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 0x7f0a64070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.503+0000 7f0a7ca81700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 0x7f0a64070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a780ffa60 tx=0x7f0a68005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:27.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.507+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0a74053850 con 0x7f0a780ffc20 2024-06-03T01:26:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.800+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0a78197150 con 0x7f0a780ffc20 2024-06-03T01:26:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.801+0000 7f0a6e7fc700 1 -- 172.21.15.32:0/778363336 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0a7405b2a0 con 0x7f0a780ffc20 2024-06-03T01:26:27.801 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:26:27.801 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:26:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:26:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 msgr2=0x7f0a64070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 0x7f0a64070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a780ffa60 tx=0x7f0a68005940 comp rx=0 tx=0).stop 2024-06-03T01:26:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 msgr2=0x7f0a7819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e920 tx=0x7f0a7400ec30 comp rx=0 tx=0).stop 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 shutdown_connections 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0a6406e340 0x7f0a64070800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a780ffc20 0x7f0a7819c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 --2- 172.21.15.32:0/778363336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a78105bd0 0x7f0a7819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 >> 172.21.15.32:0/778363336 conn(0x7f0a78075670 msgr2=0x7f0a78068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 shutdown_connections 2024-06-03T01:26:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:27.803+0000 7f0a7f4e6700 1 -- 172.21.15.32:0/778363336 wait complete. 2024-06-03T01:26:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:28 smithi032 ceph-mon[111919]: Reconfiguring osd.5 (monmap changed)... 2024-06-03T01:26:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:28 smithi032 ceph-mon[111919]: Reconfiguring daemon osd.5 on smithi044 2024-06-03T01:26:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:28 smithi032 ceph-mon[111919]: pgmap v28: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:28 smithi032 ceph-mon[111919]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:28 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/778363336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:26:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:28 smithi044 ceph-mon[84214]: Reconfiguring osd.5 (monmap changed)... 2024-06-03T01:26:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:28 smithi044 ceph-mon[84214]: Reconfiguring daemon osd.5 on smithi044 2024-06-03T01:26:28.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:28 smithi044 ceph-mon[84214]: pgmap v28: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:28.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:28 smithi044 ceph-mon[84214]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:28.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:28 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/778363336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:26:29.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:29 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:29.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:29 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:29.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:29 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:29.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:29 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:29 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:29 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:29 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:30.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:29 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: Reconfiguring mds.cephfs.smithi044.gkcyfr (monmap changed)... 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: Reconfiguring daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: pgmap v29: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:31.043 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:31.044 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:31.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: Reconfiguring mds.cephfs.smithi044.gkcyfr (monmap changed)... 2024-06-03T01:26:31.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: Reconfiguring daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:26:31.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: pgmap v29: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:31.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:31.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:31.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:26:31.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:31.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:31 smithi044 ceph-mon[84214]: Reconfiguring mds.cephfs.smithi044.mjxnhh (monmap changed)... 2024-06-03T01:26:31.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:31 smithi044 ceph-mon[84214]: Reconfiguring daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:26:32.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:31 smithi032 ceph-mon[111919]: Reconfiguring mds.cephfs.smithi044.mjxnhh (monmap changed)... 2024-06-03T01:26:32.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:31 smithi032 ceph-mon[111919]: Reconfiguring daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:26:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: pgmap v30: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:33.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all mon 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]: dispatch 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]': finished 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]: dispatch 2024-06-03T01:26:33.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:32 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]': finished 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: pgmap v30: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all mon 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]: dispatch 2024-06-03T01:26:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]': finished 2024-06-03T01:26:33.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]: dispatch 2024-06-03T01:26:33.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:32 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]': finished 2024-06-03T01:26:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:33 smithi032 ceph-mon[111919]: pgmap v31: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:33 smithi044 ceph-mon[84214]: pgmap v31: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:35.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:34 smithi032 ceph-mon[111919]: Upgrade: Updating crash.smithi032 (1/2) 2024-06-03T01:26:35.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:34 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:35.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:34 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:35.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:34 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:35.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:34 smithi032 ceph-mon[111919]: Deploying daemon crash.smithi032 on smithi032 2024-06-03T01:26:35.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:34 smithi044 ceph-mon[84214]: Upgrade: Updating crash.smithi032 (1/2) 2024-06-03T01:26:35.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:34 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:35.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:34 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:35.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:34 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:35.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:34 smithi044 ceph-mon[84214]: Deploying daemon crash.smithi032 on smithi032 2024-06-03T01:26:36.134 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:35 smithi032 ceph-mon[111919]: pgmap v32: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:35 smithi044 ceph-mon[84214]: pgmap v32: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:38.332 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:37 smithi044 ceph-mon[84214]: pgmap v33: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:38.332 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:38.332 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:38.332 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:37 smithi032 ceph-mon[111919]: pgmap v33: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: pgmap v34: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: Upgrade: Updating crash.smithi044 (2/2) 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:40.328 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:40 smithi032 ceph-mon[111919]: Deploying daemon crash.smithi044 on smithi044 2024-06-03T01:26:40.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: pgmap v34: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:40.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: Upgrade: Updating crash.smithi044 (2/2) 2024-06-03T01:26:40.464 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:40.464 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-03T01:26:40.464 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:40.464 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:40 smithi044 ceph-mon[84214]: Deploying daemon crash.smithi044 on smithi044 2024-06-03T01:26:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:42 smithi044 ceph-mon[84214]: pgmap v35: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:42 smithi032 ceph-mon[111919]: pgmap v35: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:44.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:44 smithi044 ceph-mon[84214]: pgmap v36: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:44.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:44.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:44.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:44 smithi032 ceph-mon[111919]: pgmap v36: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:44.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:46.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:46 smithi032 ceph-mon[111919]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:46.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:46 smithi044 ceph-mon[84214]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:47.534 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:47.534 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:47.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:47 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:47.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:47 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:48 smithi044 ceph-mon[84214]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:48.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:48 smithi032 ceph-mon[111919]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:48.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:48.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:50.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:50 smithi032 ceph-mon[111919]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:50.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:50.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:50.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:50 smithi044 ceph-mon[84214]: pgmap v39: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:50.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:50.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:52.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:51 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:52.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:51 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:52.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:51 smithi032 ceph-mon[111919]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:51 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:51 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:52.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:51 smithi044 ceph-mon[84214]: pgmap v40: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:53.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:26:53.903 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:53.903 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:53.903 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:53.903 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: pgmap v41: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all crash 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]: dispatch 2024-06-03T01:26:53.904 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]': finished 2024-06-03T01:26:53.905 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]: dispatch 2024-06-03T01:26:53.905 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]': finished 2024-06-03T01:26:53.905 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-03T01:26:53.905 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-03T01:26:53.905 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:53 smithi032 ceph-mon[111919]: Upgrade: osd.0 is safe to restart 2024-06-03T01:26:54.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: pgmap v41: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all crash 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:54.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]: dispatch 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]': finished 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]: dispatch 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]': finished 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-03T01:26:54.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:53 smithi044 ceph-mon[84214]: Upgrade: osd.0 is safe to restart 2024-06-03T01:26:55.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:55 smithi044 ceph-mon[84214]: Upgrade: Updating osd.0 2024-06-03T01:26:55.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:55.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:26:55.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:55.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:55 smithi044 ceph-mon[84214]: Deploying daemon osd.0 on smithi032 2024-06-03T01:26:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:55 smithi032 ceph-mon[111919]: Upgrade: Updating osd.0 2024-06-03T01:26:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:26:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T01:26:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:26:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:55 smithi032 ceph-mon[111919]: Deploying daemon osd.0 on smithi032 2024-06-03T01:26:56.704 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:56 smithi032 ceph-mon[111919]: pgmap v42: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:56.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:56 smithi044 ceph-mon[84214]: pgmap v42: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:57.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:57 smithi032 systemd[1]: Stopping Ceph osd.0 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:26:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 -- 172.21.15.32:0/1880856654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a30100750 msgr2=0x7f8a301047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 --2- 172.21.15.32:0/1880856654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a30100750 0x7f8a301047a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009910 tx=0x7f8a24009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 -- 172.21.15.32:0/1880856654 shutdown_connections 2024-06-03T01:26:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 --2- 172.21.15.32:0/1880856654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a30100750 0x7f8a301047a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 --2- 172.21.15.32:0/1880856654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a300ffda0 0x7f8a30100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 -- 172.21.15.32:0/1880856654 >> 172.21.15.32:0/1880856654 conn(0x7f8a300fb650 msgr2=0x7f8a300fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 -- 172.21.15.32:0/1880856654 shutdown_connections 2024-06-03T01:26:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.917+0000 7f8a35891700 1 -- 172.21.15.32:0/1880856654 wait complete. 2024-06-03T01:26:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.918+0000 7f8a35891700 1 Processor -- start 2024-06-03T01:26:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.918+0000 7f8a35891700 1 -- start start 2024-06-03T01:26:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a35891700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a35891700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 0x7f8a3019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a35891700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019f500 con 0x7f8a300ffda0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a35891700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a301989b0 con 0x7f8a30100750 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45080/0 (socket says 172.21.15.32:45080) 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 -- 172.21.15.32:0/580498161 learned_addr learned my addr 172.21.15.32:0/580498161 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2e7fc700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 0x7f8a3019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 -- 172.21.15.32:0/580498161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 msgr2=0x7f8a3019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 0x7f8a3019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 -- 172.21.15.32:0/580498161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240095f0 con 0x7f8a300ffda0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2e7fc700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 0x7f8a3019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.919+0000 7f8a2effd700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000d8d0 tx=0x7f8a2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.920+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a20009940 con 0x7f8a300ffda0 2024-06-03T01:26:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.920+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a30198c90 con 0x7f8a300ffda0 2024-06-03T01:26:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.920+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a301991e0 con 0x7f8a300ffda0 2024-06-03T01:26:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.920+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a20010460 con 0x7f8a300ffda0 2024-06-03T01:26:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.920+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2000f5f0 con 0x7f8a300ffda0 2024-06-03T01:26:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.922+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c0052f0 con 0x7f8a300ffda0 2024-06-03T01:26:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.922+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8a20009aa0 con 0x7f8a300ffda0 2024-06-03T01:26:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.923+0000 7f8a3488f700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 0x7f8a180707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.923+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8a2008e290 con 0x7f8a300ffda0 2024-06-03T01:26:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.923+0000 7f8a2e7fc700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 0x7f8a180707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.924+0000 7f8a2e7fc700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 0x7f8a180707f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a300fbec0 tx=0x7f8a2401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:57.928+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8a2005ac00 con 0x7f8a300ffda0 2024-06-03T01:26:58.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.157+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a1c000bc0 con 0x7f8a1806e330 2024-06-03T01:26:58.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.158+0000 7f8a3488f700 1 -- 172.21.15.32:0/580498161 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8a1c000bc0 con 0x7f8a1806e330 2024-06-03T01:26:58.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 msgr2=0x7f8a180707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 0x7f8a180707f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a300fbec0 tx=0x7f8a2401a040 comp rx=0 tx=0).stop 2024-06-03T01:26:58.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 msgr2=0x7f8a3019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000d8d0 tx=0x7f8a2000dc90 comp rx=0 tx=0).stop 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 shutdown_connections 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8a1806e330 0x7f8a180707f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a300ffda0 0x7f8a3019e930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 --2- 172.21.15.32:0/580498161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a30100750 0x7f8a3019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 >> 172.21.15.32:0/580498161 conn(0x7f8a300fb650 msgr2=0x7f8a30103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 shutdown_connections 2024-06-03T01:26:58.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.160+0000 7f8a35891700 1 -- 172.21.15.32:0/580498161 wait complete. 2024-06-03T01:26:58.171 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:26:58.171 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:57 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[48123]: 2024-06-03T01:26:57.922+0000 7f5ff5acb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:58.171 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:57 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[48123]: 2024-06-03T01:26:57.922+0000 7f5ff5acb700 -1 osd.0 49 *** Got signal Terminated *** 2024-06-03T01:26:58.172 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:57 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[48123]: 2024-06-03T01:26:57.922+0000 7f5ff5acb700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:58.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 -- 172.21.15.32:0/1609013234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340fff50 msgr2=0x7f6d34105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 --2- 172.21.15.32:0/1609013234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340fff50 0x7f6d34105840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009910 tx=0x7f6d28009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 -- 172.21.15.32:0/1609013234 shutdown_connections 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 --2- 172.21.15.32:0/1609013234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340fff50 0x7f6d34105840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 --2- 172.21.15.32:0/1609013234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340ff5a0 0x7f6d340ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 -- 172.21.15.32:0/1609013234 >> 172.21.15.32:0/1609013234 conn(0x7f6d34074e30 msgr2=0x7f6d34075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 -- 172.21.15.32:0/1609013234 shutdown_connections 2024-06-03T01:26:58.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.271+0000 7f6d38c49700 1 -- 172.21.15.32:0/1609013234 wait complete. 2024-06-03T01:26:58.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 Processor -- start 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 -- start start 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340fff50 0x7f6d3419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419d210 con 0x7f6d340ff5a0 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d38c49700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341966c0 con 0x7f6d340fff50 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45098/0 (socket says 172.21.15.32:45098) 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.272+0000 7f6d3259c700 1 -- 172.21.15.32:0/3872783041 learned_addr learned my addr 172.21.15.32:0/3872783041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d3259c700 1 -- 172.21.15.32:0/3872783041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340fff50 msgr2=0x7f6d3419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d31d9b700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340fff50 0x7f6d3419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d3259c700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340fff50 0x7f6d3419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d3259c700 1 -- 172.21.15.32:0/3872783041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280095f0 con 0x7f6d340ff5a0 2024-06-03T01:26:58.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d3259c700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400e920 tx=0x7f6d2400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d24009790 con 0x7f6d340ff5a0 2024-06-03T01:26:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d341969a0 con 0x7f6d340ff5a0 2024-06-03T01:26:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d2400cba0 con 0x7f6d340ff5a0 2024-06-03T01:26:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d24018650 con 0x7f6d340ff5a0 2024-06-03T01:26:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.273+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d34196ef0 con 0x7f6d340ff5a0 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.274+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d34109200 con 0x7f6d340ff5a0 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.275+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6d2400cd10 con 0x7f6d340ff5a0 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.275+0000 7f6d237fe700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 0x7f6d1c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.275+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6d24014070 con 0x7f6d340ff5a0 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.275+0000 7f6d31d9b700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 0x7f6d1c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.276+0000 7f6d31d9b700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 0x7f6d1c0707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6d34197b30 tx=0x7f6d28009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.280+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6d24053850 con 0x7f6d340ff5a0 2024-06-03T01:26:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.507+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d34197160 con 0x7f6d1c06e330 2024-06-03T01:26:58.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.508+0000 7f6d237fe700 1 -- 172.21.15.32:0/3872783041 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6d34197160 con 0x7f6d1c06e330 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 msgr2=0x7f6d1c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 0x7f6d1c0707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6d34197b30 tx=0x7f6d28009f70 comp rx=0 tx=0).stop 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 msgr2=0x7f6d3419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400e920 tx=0x7f6d2400ece0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 shutdown_connections 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6d1c06e330 0x7f6d1c0707f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d340ff5a0 0x7f6d3419c640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 --2- 172.21.15.32:0/3872783041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d340fff50 0x7f6d3419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.510+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 >> 172.21.15.32:0/3872783041 conn(0x7f6d34074e30 msgr2=0x7f6d340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.511+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 shutdown_connections 2024-06-03T01:26:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.511+0000 7f6d38c49700 1 -- 172.21.15.32:0/3872783041 wait complete. 2024-06-03T01:26:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 -- 172.21.15.32:0/2570141832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 msgr2=0x7fa9c8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/2570141832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c8100180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800b0a0 tx=0x7fa9b800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 -- 172.21.15.32:0/2570141832 shutdown_connections 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/2570141832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/2570141832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c8100180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 -- 172.21.15.32:0/2570141832 >> 172.21.15.32:0/2570141832 conn(0x7fa9c80fb650 msgr2=0x7fa9c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 -- 172.21.15.32:0/2570141832 shutdown_connections 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.627+0000 7fa9cce6f700 1 -- 172.21.15.32:0/2570141832 wait complete. 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 Processor -- start 2024-06-03T01:26:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 -- start start 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c819d3b0 con 0x7fa9c80ffda0 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9cce6f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8196860 con 0x7fa9c8100750 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.628+0000 7fa9c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45112/0 (socket says 172.21.15.32:45112) 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 -- 172.21.15.32:0/4000162627 learned_addr learned my addr 172.21.15.32:0/4000162627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c5d9b700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 -- 172.21.15.32:0/4000162627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 msgr2=0x7fa9c819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 -- 172.21.15.32:0/4000162627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b8009d00 con 0x7fa9c80ffda0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c5d9b700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9c659c700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000f80 tx=0x7fa9b800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9b800e040 con 0x7fa9c80ffda0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9b8003e60 con 0x7fa9c80ffda0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c8196ae0 con 0x7fa9c80ffda0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9b801d910 con 0x7fa9c80ffda0 2024-06-03T01:26:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.629+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c8196fd0 con 0x7fa9c80ffda0 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.630+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c804f2e0 con 0x7fa9c80ffda0 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.631+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa9b8004820 con 0x7fa9c80ffda0 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.631+0000 7fa9b77fe700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 0x7fa9b0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.631+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa9b808fb70 con 0x7fa9c80ffda0 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.631+0000 7fa9c5d9b700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 0x7fa9b0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.632+0000 7fa9c5d9b700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 0x7fa9b0070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8106760 tx=0x7fa9bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.636+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa9b805c890 con 0x7fa9c80ffda0 2024-06-03T01:26:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:58 smithi032 ceph-mon[111919]: pgmap v43: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:58 smithi032 ceph-mon[111919]: osd.0 marked itself down and dead 2024-06-03T01:26:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.855+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9c8197240 con 0x7fa9b006e340 2024-06-03T01:26:58.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.867+0000 7fa9b77fe700 1 -- 172.21.15.32:0/4000162627 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa9c8197240 con 0x7fa9b006e340 2024-06-03T01:26:58.867 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:26:58.867 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (3m) 10s ago 14m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:26:58.867 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (14m) 10s ago 14m 8204k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:26:58.867 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (13m) 12s ago 13m 8002k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (21s) 10s ago 14m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (16s) 12s ago 13m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (2m) 10s ago 13m 81.0M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (8m) 10s ago 8m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (8m) 10s ago 8m 123M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (8m) 12s ago 8m 16.0M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (8m) 12s ago 8m 17.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (5m) 10s ago 15m 497M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (4m) 12s ago 13m 450M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (92s) 10s ago 15m 44.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (65s) 12s ago 13m 37.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (3m) 10s ago 14m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:26:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (3m) 12s ago 13m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (12m) 10s ago 12m 240M 1217M 18.2.1 5be31c24972a 0431debd7714 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (11m) 10s ago 11m 240M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (11m) 10s ago 11m 204M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (10m) 12s ago 10m 269M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (10m) 12s ago 10m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (10m) 12s ago 10m 231M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:26:58.869 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 10s ago 13m 76.0M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 msgr2=0x7fa9b0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 0x7fa9b0070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8106760 tx=0x7fa9bc008040 comp rx=0 tx=0).stop 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 msgr2=0x7fa9c819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000f80 tx=0x7fa9b800cdd0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 shutdown_connections 2024-06-03T01:26:58.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa9b006e340 0x7fa9b0070800 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c80ffda0 0x7fa9c819c7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 --2- 172.21.15.32:0/4000162627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9c8100750 0x7fa9c819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 >> 172.21.15.32:0/4000162627 conn(0x7fa9c80fb650 msgr2=0x7fa9c8103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 shutdown_connections 2024-06-03T01:26:58.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.870+0000 7fa9cce6f700 1 -- 172.21.15.32:0/4000162627 wait complete. 2024-06-03T01:26:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:58 smithi044 ceph-mon[84214]: pgmap v43: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:26:58.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:58 smithi044 ceph-mon[84214]: osd.0 marked itself down and dead 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 -- 172.21.15.32:0/3062411387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 msgr2=0x7ff7301047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 --2- 172.21.15.32:0/3062411387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff7301047b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009910 tx=0x7ff71c009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 -- 172.21.15.32:0/3062411387 shutdown_connections 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 --2- 172.21.15.32:0/3062411387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff7301047b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 --2- 172.21.15.32:0/3062411387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 0x7ff730100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 -- 172.21.15.32:0/3062411387 >> 172.21.15.32:0/3062411387 conn(0x7ff730074e30 msgr2=0x7ff730075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 -- 172.21.15.32:0/3062411387 shutdown_connections 2024-06-03T01:26:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 -- 172.21.15.32:0/3062411387 wait complete. 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.978+0000 7ff736b2e700 1 Processor -- start 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff736b2e700 1 -- start start 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff736b2e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 0x7ff73019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff736b2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff736b2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73019c0f0 con 0x7ff730101130 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff736b2e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730195630 con 0x7ff730100780 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45142/0 (socket says 172.21.15.32:45142) 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 -- 172.21.15.32:0/2650513785 learned_addr learned my addr 172.21.15.32:0/2650513785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff7348ca700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 0x7ff73019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 -- 172.21.15.32:0/2650513785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 msgr2=0x7ff73019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 0x7ff73019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 -- 172.21.15.32:0/2650513785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0095f0 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.979+0000 7ff72ffff700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00bda0 tx=0x7ff71c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.980+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71c01d070 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.980+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff71c00f460 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.980+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7301958b0 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.980+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71c017630 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.980+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff730195da0 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.981+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730108090 con 0x7ff730101130 2024-06-03T01:26:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.981+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff71c00fab0 con 0x7ff730101130 2024-06-03T01:26:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.982+0000 7ff72dffb700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 0x7ff720070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.982+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff71c08f780 con 0x7ff730101130 2024-06-03T01:26:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.982+0000 7ff7348ca700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 0x7ff720070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.982+0000 7ff7348ca700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 0x7ff720070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ac50 tx=0x7ff72400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:58.987+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff71c05c4a0 con 0x7ff730101130 2024-06-03T01:26:59.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.287+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff730196b40 con 0x7ff730101130 2024-06-03T01:26:59.288 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:59 smithi032 podman[122615]: 2024-06-03 01:26:59.090894566 +0000 UTC m=+1.290670952 container died 0431debd7714353c178819cae4a1ea7576d8f2e0711a50d61334359f50e8420c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0) 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.287+0000 7ff72dffb700 1 -- 172.21.15.32:0/2650513785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+684 (secure 0 0 0) 0x7ff71c05c030 con 0x7ff730101130 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:59.288 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:26:59.289 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:26:59.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 msgr2=0x7ff720070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 0x7ff720070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ac50 tx=0x7ff72400a380 comp rx=0 tx=0).stop 2024-06-03T01:26:59.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 msgr2=0x7ff73019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00bda0 tx=0x7ff71c00bdd0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 shutdown_connections 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff72006e340 0x7ff720070800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff730100780 0x7ff73019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 --2- 172.21.15.32:0/2650513785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff730101130 0x7ff73019baf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 >> 172.21.15.32:0/2650513785 conn(0x7ff730074e30 msgr2=0x7ff7300fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 shutdown_connections 2024-06-03T01:26:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.290+0000 7ff736b2e700 1 -- 172.21.15.32:0/2650513785 wait complete. 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 -- 172.21.15.32:0/483151716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 msgr2=0x7fd548101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/483151716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd548101260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd544009910 tx=0x7fd544009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 -- 172.21.15.32:0/483151716 shutdown_connections 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/483151716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 0x7fd548105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/483151716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd548101260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 -- 172.21.15.32:0/483151716 >> 172.21.15.32:0/483151716 conn(0x7fd5480fc730 msgr2=0x7fd5480feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 -- 172.21.15.32:0/483151716 shutdown_connections 2024-06-03T01:26:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.398+0000 7fd54f2c2700 1 -- 172.21.15.32:0/483151716 wait complete. 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 Processor -- start 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 -- start start 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 0x7fd54819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54819d220 con 0x7fd548100e80 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54f2c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481966d0 con 0x7fd548101830 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45162/0 (socket says 172.21.15.32:45162) 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 -- 172.21.15.32:0/3276045660 learned_addr learned my addr 172.21.15.32:0/3276045660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54c85d700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 0x7fd54819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 -- 172.21.15.32:0/3276045660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 msgr2=0x7fd54819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 0x7fd54819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 -- 172.21.15.32:0/3276045660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440095f0 con 0x7fd548100e80 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.399+0000 7fd54d05e700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.400+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54401d070 con 0x7fd548100e80 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.400+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd54400f460 con 0x7fd548100e80 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.400+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd544017630 con 0x7fd548100e80 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.400+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd548196950 con 0x7fd548100e80 2024-06-03T01:26:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.400+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd548196df0 con 0x7fd548100e80 2024-06-03T01:26:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.401+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd548109220 con 0x7fd548100e80 2024-06-03T01:26:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.401+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fd54400f5d0 con 0x7fd548100e80 2024-06-03T01:26:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.402+0000 7fd53e7fc700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 0x7fd534070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.402+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd54408f830 con 0x7fd548100e80 2024-06-03T01:26:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.402+0000 7fd54c85d700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 0x7fd534070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.402+0000 7fd54c85d700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 0x7fd534070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd538005950 tx=0x7fd5380058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.407+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd54405c4d0 con 0x7fd548100e80 2024-06-03T01:26:59.568 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:59 smithi032 podman[122615]: 2024-06-03 01:26:59.513924947 +0000 UTC m=+1.713701322 container cleanup 0431debd7714353c178819cae4a1ea7576d8f2e0711a50d61334359f50e8420c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-03T01:26:59.568 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:59 smithi032 bash[122615]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0 2024-06-03T01:26:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.669+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd548197ce0 con 0x7fd548100e80 2024-06-03T01:26:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.669+0000 7fd53e7fc700 1 -- 172.21.15.32:0/3276045660 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7fd548197ce0 con 0x7fd548100e80 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:26:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:26:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:59.672 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 msgr2=0x7fd534070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 0x7fd534070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd538005950 tx=0x7fd5380058e0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 msgr2=0x7fd54819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54400be20 comp rx=0 tx=0).stop 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 shutdown_connections 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd53406e340 0x7fd534070800 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd548100e80 0x7fd54819c650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 --2- 172.21.15.32:0/3276045660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd548101830 0x7fd54819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 >> 172.21.15.32:0/3276045660 conn(0x7fd5480fc730 msgr2=0x7fd5480fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 shutdown_connections 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.672+0000 7fd54f2c2700 1 -- 172.21.15.32:0/3276045660 wait complete. 2024-06-03T01:26:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 -- 172.21.15.32:0/1483698368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa504105bf0 msgr2=0x7fa504105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/1483698368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa504105bf0 0x7fa504105fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa500009910 tx=0x7fa500009c20 comp rx=0 tx=0).stop 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 -- 172.21.15.32:0/1483698368 shutdown_connections 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/1483698368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5040ffb40 0x7fa5040fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/1483698368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa504105bf0 0x7fa504105fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 -- 172.21.15.32:0/1483698368 >> 172.21.15.32:0/1483698368 conn(0x7fa5040fb610 msgr2=0x7fa5040fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 -- 172.21.15.32:0/1483698368 shutdown_connections 2024-06-03T01:26:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.781+0000 7fa50bcd9700 1 -- 172.21.15.32:0/1483698368 wait complete. 2024-06-03T01:26:59.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 Processor -- start 2024-06-03T01:26:59.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 -- start start 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 0x7fa50419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 0x7fa50419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50419d270 con 0x7fa5040ffb40 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.782+0000 7fa50bcd9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504196720 con 0x7fa504105bf0 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 0x7fa50419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509a75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 0x7fa50419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509274700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 0x7fa50419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:47640/0 (socket says 172.21.15.32:47640) 2024-06-03T01:26:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509274700 1 -- 172.21.15.32:0/3992715511 learned_addr learned my addr 172.21.15.32:0/3992715511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509a75700 1 -- 172.21.15.32:0/3992715511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 msgr2=0x7fa50419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509a75700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 0x7fa50419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509a75700 1 -- 172.21.15.32:0/3992715511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000095f0 con 0x7fa5040ffb40 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa509a75700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 0x7fa50419c6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa500009910 tx=0x7fa50000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.783+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa50001d070 con 0x7fa5040ffb40 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.784+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa500005dc0 con 0x7fa5040ffb40 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.784+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5041969a0 con 0x7fa5040ffb40 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.784+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa50000fe80 con 0x7fa5040ffb40 2024-06-03T01:26:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.784+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa504196e90 con 0x7fa5040ffb40 2024-06-03T01:26:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.785+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50404f2e0 con 0x7fa5040ffb40 2024-06-03T01:26:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.785+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa500003a60 con 0x7fa5040ffb40 2024-06-03T01:26:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.786+0000 7fa4faffd700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 0x7fa4f0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:26:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.786+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa50008f660 con 0x7fa5040ffb40 2024-06-03T01:26:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.786+0000 7fa509274700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 0x7fa4f0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:26:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.786+0000 7fa509274700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 0x7fa4f0070800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa504197b90 tx=0x7fa4f400b500 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:26:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:26:59.791+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa50005c300 con 0x7fa5040ffb40 2024-06-03T01:26:59.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:26:59 smithi032 podman[122726]: 2024-06-03 01:26:59.746716775 +0000 UTC m=+0.652245766 container remove 0431debd7714353c178819cae4a1ea7576d8f2e0711a50d61334359f50e8420c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=HEAD) 2024-06-03T01:26:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:59 smithi032 ceph-mon[111919]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:59 smithi032 ceph-mon[111919]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:59 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:26:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:59 smithi032 ceph-mon[111919]: osdmap e50: 6 total, 5 up, 6 in 2024-06-03T01:26:59.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:26:59 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2650513785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:26:59.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:59 smithi044 ceph-mon[84214]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:59.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:59 smithi044 ceph-mon[84214]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:26:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:59 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:26:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:59 smithi044 ceph-mon[84214]: osdmap e50: 6 total, 5 up, 6 in 2024-06-03T01:26:59.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:26:59 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2650513785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.057+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa504197150 con 0x7fa4f006e340 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.058+0000 7fa4faffd700 1 -- 172.21.15.32:0/3992715511 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa504197150 con 0x7fa4f006e340 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:27:00.058 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:27:00.059 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:27:00.059 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "6/23 daemons upgraded", 2024-06-03T01:27:00.059 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:27:00.059 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:27:00.059 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:27:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 msgr2=0x7fa4f0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 0x7fa4f0070800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa504197b90 tx=0x7fa4f400b500 comp rx=0 tx=0).stop 2024-06-03T01:27:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 msgr2=0x7fa50419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 0x7fa50419c6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa500009910 tx=0x7fa50000bd70 comp rx=0 tx=0).stop 2024-06-03T01:27:00.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 shutdown_connections 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa4f006e340 0x7fa4f0070800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5040ffb40 0x7fa50419c6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 --2- 172.21.15.32:0/3992715511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504105bf0 0x7fa50419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 >> 172.21.15.32:0/3992715511 conn(0x7fa5040fb610 msgr2=0x7fa504103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 shutdown_connections 2024-06-03T01:27:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.060+0000 7fa50bcd9700 1 -- 172.21.15.32:0/3992715511 wait complete. 2024-06-03T01:27:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 -- 172.21.15.32:0/39980392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 msgr2=0x7f60a0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 --2- 172.21.15.32:0/39980392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a0105fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f609c009910 tx=0x7f609c009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 -- 172.21.15.32:0/39980392 shutdown_connections 2024-06-03T01:27:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 --2- 172.21.15.32:0/39980392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 0x7f60a0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 --2- 172.21.15.32:0/39980392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a0105fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 -- 172.21.15.32:0/39980392 >> 172.21.15.32:0/39980392 conn(0x7f60a0075670 msgr2=0x7f60a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 -- 172.21.15.32:0/39980392 shutdown_connections 2024-06-03T01:27:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.176+0000 7f60a8503700 1 -- 172.21.15.32:0/39980392 wait complete. 2024-06-03T01:27:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 Processor -- start 2024-06-03T01:27:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 -- start start 2024-06-03T01:27:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 0x7f60a019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a019d1f0 con 0x7f60a0105bd0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a8503700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01966a0 con 0x7f60a00ffc20 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45194/0 (socket says 172.21.15.32:45194) 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 -- 172.21.15.32:0/2482006928 learned_addr learned my addr 172.21.15.32:0/2482006928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a629f700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 0x7f60a019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 -- 172.21.15.32:0/2482006928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 msgr2=0x7f60a019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 0x7f60a019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.177+0000 7f60a5a9e700 1 -- 172.21.15.32:0/2482006928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0095f0 con 0x7f60a0105bd0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60a5a9e700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f609000d8d0 tx=0x7f609000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60900098a0 con 0x7f60a0105bd0 2024-06-03T01:27:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a0196980 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6090010460 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609000f5f0 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.178+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a0196ed0 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.179+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a004f2e0 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.179+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6090009a00 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.180+0000 7f60977fe700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 0x7f608c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.180+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f609008e250 con 0x7f60a0105bd0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.180+0000 7f60a629f700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 0x7f608c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.180+0000 7f60a629f700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 0x7f608c0707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b5c0 tx=0x7f609c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:00.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.185+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f609005aef0 con 0x7f60a0105bd0 2024-06-03T01:27:00.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.130842325 +0000 UTC m=+0.018309784 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:00.336 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.233615519 +0000 UTC m=+0.121082958 container create 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T01:27:00.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.512+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60a01970f0 con 0x7f60a0105bd0 2024-06-03T01:27:00.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.512+0000 7f60977fe700 1 -- 172.21.15.32:0/2482006928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+290 (secure 0 0 0) 0x7f609005aa80 con 0x7f60a0105bd0 2024-06-03T01:27:00.513 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-06-03T01:27:00.513 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:27:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:27:00.513 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] OSD_DOWN: 1 osds down 2024-06-03T01:27:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: osd.0 (root=default,host=smithi032) is down 2024-06-03T01:27:00.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 msgr2=0x7f608c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 0x7f608c0707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b5c0 tx=0x7f609c01a040 comp rx=0 tx=0).stop 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 msgr2=0x7f60a019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f609000d8d0 tx=0x7f609000dbe0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 shutdown_connections 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f608c06e330 0x7f608c0707f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a00ffc20 0x7f60a019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 --2- 172.21.15.32:0/2482006928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a0105bd0 0x7f60a019cb60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.514+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 >> 172.21.15.32:0/2482006928 conn(0x7f60a0075670 msgr2=0x7f60a00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.515+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 shutdown_connections 2024-06-03T01:27:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:00.515+0000 7f60a8503700 1 -- 172.21.15.32:0/2482006928 wait complete. 2024-06-03T01:27:00.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:00 smithi032 ceph-mon[111919]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:00.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:00 smithi032 ceph-mon[111919]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:00.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:00 smithi032 ceph-mon[111919]: osdmap e51: 6 total, 5 up, 6 in 2024-06-03T01:27:00.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:00 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3276045660' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:27:00.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:00 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2482006928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:27:00.769 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.445422507 +0000 UTC m=+0.332889962 container init 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:00.769 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.449421744 +0000 UTC m=+0.336889198 container start 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:00.769 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.502861993 +0000 UTC m=+0.390329433 container attach 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-06-03T01:27:00.872 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:00 smithi044 ceph-mon[84214]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:00.872 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:00 smithi044 ceph-mon[84214]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:00.872 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:00 smithi044 ceph-mon[84214]: osdmap e51: 6 total, 5 up, 6 in 2024-06-03T01:27:00.872 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:00 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3276045660' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:27:00.872 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:00 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2482006928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:27:01.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:00 smithi032 podman[122860]: 2024-06-03 01:27:00.769354776 +0000 UTC m=+0.656822238 container died 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-03T01:27:01.700 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:01 smithi032 ceph-mon[111919]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:01.701 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 podman[122909]: 2024-06-03 01:27:01.449169534 +0000 UTC m=+0.673225599 container remove 9bc69d199b5e387b2d042c8adf7ff1e3fe860f328e44d1f09fe65aa5c7578b0a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:27:01.701 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.0.service: Deactivated successfully. 2024-06-03T01:27:01.701 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 systemd[1]: Stopped Ceph osd.0 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:27:01.701 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.0.service: Consumed 1min 608ms CPU time. 2024-06-03T01:27:01.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:01 smithi044 ceph-mon[84214]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:02.031 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 systemd[1]: Starting Ceph osd.0 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:27:02.031 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 podman[123009]: 2024-06-03 01:27:01.823371461 +0000 UTC m=+0.017984881 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:02.031 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:01 smithi032 podman[123009]: 2024-06-03 01:27:01.933670474 +0000 UTC m=+0.128283886 container create 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:27:02.313 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 podman[123009]: 2024-06-03 01:27:02.136285647 +0000 UTC m=+0.330899060 container init 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T01:27:02.313 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 podman[123009]: 2024-06-03 01:27:02.139853138 +0000 UTC m=+0.334466554 container start 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:02.313 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 podman[123009]: 2024-06-03 01:27:02.24460234 +0000 UTC m=+0.439215769 container attach 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:02.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-mon[111919]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:02.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-03T01:27:02.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-03T01:27:02.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-03T01:27:02.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 bash[123009]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate[123025]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-03T01:27:02.836 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:02 smithi032 podman[123009]: 2024-06-03 01:27:02.632053672 +0000 UTC m=+0.826667085 container died 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:02.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:02 smithi044 ceph-mon[84214]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:03.559 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 podman[123159]: 2024-06-03 01:27:03.20729717 +0000 UTC m=+0.566801659 container remove 8494a1e8253dad2a499ec77d1e69be7b655bd176716761fbf6dc6fba5baa2377 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-activate, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-03T01:27:03.559 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 podman[123220]: 2024-06-03 01:27:03.41656777 +0000 UTC m=+0.101815702 container create d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T01:27:03.559 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 podman[123220]: 2024-06-03 01:27:03.333207846 +0000 UTC m=+0.018455780 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:03.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:03 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 72/366 objects degraded (19.672%), 19 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:04.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:03 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 72/366 objects degraded (19.672%), 19 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:04.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 podman[123220]: 2024-06-03 01:27:03.669185229 +0000 UTC m=+0.354433147 container init d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T01:27:04.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 podman[123220]: 2024-06-03 01:27:03.672537001 +0000 UTC m=+0.357784915 container start d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-03T01:27:04.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 ceph-osd[123239]: -- 172.21.15.32:0/6956918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559582bf7680 con 0x559582bc1000 2024-06-03T01:27:04.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 bash[123220]: d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b 2024-06-03T01:27:04.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:03 smithi032 systemd[1]: Started Ceph osd.0 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:27:04.585 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:04 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:27:04.219+0000 7f094a2ad7c0 -1 Falling back to public interface 2024-06-03T01:27:04.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:04 smithi044 ceph-mon[84214]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:04.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:04 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:05.072 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:04 smithi032 ceph-mon[111919]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:05.072 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:05.072 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:05.073 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:04 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:06.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:06 smithi044 ceph-mon[84214]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:06.982 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:06 smithi032 ceph-mon[111919]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:08 smithi032 ceph-mon[111919]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:08 smithi044 ceph-mon[84214]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/366 objects degraded (19.672%) 2024-06-03T01:27:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:08.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:08 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:27:08.458+0000 7f094a2ad7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-06-03T01:27:09.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:09 smithi032 ceph-mon[111919]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T01:27:09.287 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:08 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:27:08.900+0000 7f094a2ad7c0 -1 osd.0 49 log_to_monitors true 2024-06-03T01:27:09.287 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:27:09 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:27:09.031+0000 7f093b704700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:27:09.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:09 smithi044 ceph-mon[84214]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T01:27:10.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 409 B/s wr, 0 op/s; 72/366 objects degraded (19.672%) 2024-06-03T01:27:10.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T01:27:10.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: osdmap e52: 6 total, 5 up, 6 in 2024-06-03T01:27:10.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: mgrmap e38: smithi032.kfcfaa(active, since 92s), standbys: smithi044.wszjlw 2024-06-03T01:27:10.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:27:10.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:10.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 409 B/s wr, 0 op/s; 72/366 objects degraded (19.672%) 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: osdmap e52: 6 total, 5 up, 6 in 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: mgrmap e38: smithi032.kfcfaa(active, since 92s), standbys: smithi044.wszjlw 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: from='osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995] boot 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: osdmap e53: 6 total, 6 up, 6 in 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:11.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:11 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:11.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:27:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: osd.0 [v2:172.21.15.32:6802/3149126995,v1:172.21.15.32:6803/3149126995] boot 2024-06-03T01:27:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: osdmap e53: 6 total, 6 up, 6 in 2024-06-03T01:27:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:27:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:11.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:11 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:12.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:12 smithi032 ceph-mon[111919]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 72/366 objects degraded (19.672%) 2024-06-03T01:27:12.236 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:12 smithi032 ceph-mon[111919]: osdmap e54: 6 total, 6 up, 6 in 2024-06-03T01:27:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:12 smithi044 ceph-mon[84214]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 72/366 objects degraded (19.672%) 2024-06-03T01:27:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:12 smithi044 ceph-mon[84214]: osdmap e54: 6 total, 6 up, 6 in 2024-06-03T01:27:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:13 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:13.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:13 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 14/366 objects degraded (3.825%); 23 B/s, 0 objects/s recovering 2024-06-03T01:27:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 14/366 objects degraded (3.825%); 23 B/s, 0 objects/s recovering 2024-06-03T01:27:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:15 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:15 smithi032 ceph-mon[111919]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-03T01:27:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:15 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:15.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:15 smithi044 ceph-mon[84214]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-06-03T01:27:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:16 smithi032 ceph-mon[111919]: pgmap v57: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-06-03T01:27:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:16 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded) 2024-06-03T01:27:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:16 smithi044 ceph-mon[84214]: pgmap v57: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-06-03T01:27:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:16 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded) 2024-06-03T01:27:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:18 smithi032 ceph-mon[111919]: pgmap v58: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-06-03T01:27:19.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:18 smithi044 ceph-mon[84214]: pgmap v58: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-06-03T01:27:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:20 smithi032 ceph-mon[111919]: pgmap v59: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 B/s, 0 objects/s recovering 2024-06-03T01:27:21.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:20 smithi044 ceph-mon[84214]: pgmap v59: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 B/s, 0 objects/s recovering 2024-06-03T01:27:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:22 smithi032 ceph-mon[111919]: pgmap v60: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-06-03T01:27:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:22 smithi044 ceph-mon[84214]: pgmap v60: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-06-03T01:27:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:25.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:24 smithi032 ceph-mon[111919]: pgmap v61: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-06-03T01:27:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:24 smithi044 ceph-mon[84214]: pgmap v61: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-06-03T01:27:27.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:26 smithi032 ceph-mon[111919]: pgmap v62: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-06-03T01:27:27.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:26 smithi044 ceph-mon[84214]: pgmap v62: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-06-03T01:27:29.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:28 smithi032 ceph-mon[111919]: pgmap v63: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:28 smithi044 ceph-mon[84214]: pgmap v63: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:29.896 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:29 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:30.123 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:29 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:30.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 -- 172.21.15.32:0/252225745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600fff50 msgr2=0x7f0f60105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 --2- 172.21.15.32:0/252225745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600fff50 0x7f0f60105840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009910 tx=0x7f0f54009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 -- 172.21.15.32:0/252225745 shutdown_connections 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 --2- 172.21.15.32:0/252225745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600fff50 0x7f0f60105840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 --2- 172.21.15.32:0/252225745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600ff5a0 0x7f0f600ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 -- 172.21.15.32:0/252225745 >> 172.21.15.32:0/252225745 conn(0x7f0f60074e30 msgr2=0x7f0f60075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 -- 172.21.15.32:0/252225745 shutdown_connections 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.637+0000 7f0f653aa700 1 -- 172.21.15.32:0/252225745 wait complete. 2024-06-03T01:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 Processor -- start 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 -- start start 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600fff50 0x7f0f6019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6019d190 con 0x7f0f600ff5a0 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f653aa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601966d0 con 0x7f0f600fff50 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51248/0 (socket says 172.21.15.32:51248) 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f5effd700 1 -- 172.21.15.32:0/1060294502 learned_addr learned my addr 172.21.15.32:0/1060294502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.638+0000 7f0f5e7fc700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600fff50 0x7f0f6019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f5effd700 1 -- 172.21.15.32:0/1060294502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600fff50 msgr2=0x7f0f6019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:30.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f5effd700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600fff50 0x7f0f6019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f5effd700 1 -- 172.21.15.32:0/1060294502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f540095f0 con 0x7f0f600ff5a0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f5effd700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c930 tx=0x7f0f5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f50007ad0 con 0x7f0f600ff5a0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f5000ce80 con 0x7f0f600ff5a0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f601969b0 con 0x7f0f600ff5a0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f50018630 con 0x7f0f600ff5a0 2024-06-03T01:27:30.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.639+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f60196f00 con 0x7f0f600ff5a0 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.640+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f60109200 con 0x7f0f600ff5a0 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.641+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f50007c30 con 0x7f0f600ff5a0 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.641+0000 7f0f47fff700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 0x7f0f480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.641+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0f5008e060 con 0x7f0f600ff5a0 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.641+0000 7f0f5e7fc700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 0x7f0f480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:30.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.642+0000 7f0f5e7fc700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 0x7f0f480706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f60197b40 tx=0x7f0f5401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:30.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.646+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0f5005af20 con 0x7f0f600ff5a0 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: pgmap v64: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: Upgrade: osd.1 is safe to restart 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:27:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.876+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f60197cd0 con 0x7f0f4806e200 2024-06-03T01:27:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.878+0000 7f0f47fff700 1 -- 172.21.15.32:0/1060294502 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0f60197cd0 con 0x7f0f4806e200 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 msgr2=0x7f0f480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 0x7f0f480706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f60197b40 tx=0x7f0f5401a040 comp rx=0 tx=0).stop 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 msgr2=0x7f0f6019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000c930 tx=0x7f0f5000cc40 comp rx=0 tx=0).stop 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 shutdown_connections 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0f4806e200 0x7f0f480706c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f600ff5a0 0x7f0f6019c650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 --2- 172.21.15.32:0/1060294502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f600fff50 0x7f0f6019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 >> 172.21.15.32:0/1060294502 conn(0x7f0f60074e30 msgr2=0x7f0f600fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 shutdown_connections 2024-06-03T01:27:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:30.880+0000 7f0f653aa700 1 -- 172.21.15.32:0/1060294502 wait complete. 2024-06-03T01:27:30.892 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 -- 172.21.15.32:0/2027535314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 msgr2=0x7f53d8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 --2- 172.21.15.32:0/2027535314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8100080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009910 tx=0x7f53c8009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 -- 172.21.15.32:0/2027535314 shutdown_connections 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 --2- 172.21.15.32:0/2027535314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8100080 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 --2- 172.21.15.32:0/2027535314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 -- 172.21.15.32:0/2027535314 >> 172.21.15.32:0/2027535314 conn(0x7f53d8075670 msgr2=0x7f53d8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:31.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 -- 172.21.15.32:0/2027535314 shutdown_connections 2024-06-03T01:27:31.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.001+0000 7f53df74c700 1 -- 172.21.15.32:0/2027535314 wait complete. 2024-06-03T01:27:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 Processor -- start 2024-06-03T01:27:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 -- start start 2024-06-03T01:27:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81022c0 con 0x7f53d80ffc20 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53df74c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8102400 con 0x7f53d8105bd0 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51278/0 (socket says 172.21.15.32:51278) 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 -- 172.21.15.32:0/4096412269 learned_addr learned my addr 172.21.15.32:0/4096412269 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dcce7700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 -- 172.21.15.32:0/4096412269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 msgr2=0x7f53d8101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dd4e8700 1 -- 172.21.15.32:0/4096412269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80095f0 con 0x7f53d80ffc20 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.002+0000 7f53dcce7700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.003+0000 7f53dd4e8700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53d400d8d0 tx=0x7f53d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.003+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d4009940 con 0x7f53d80ffc20 2024-06-03T01:27:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.003+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53d4010460 con 0x7f53d80ffc20 2024-06-03T01:27:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.003+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d81a2a60 con 0x7f53d80ffc20 2024-06-03T01:27:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.004+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d400f5f0 con 0x7f53d80ffc20 2024-06-03T01:27:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.004+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d81a2e70 con 0x7f53d80ffc20 2024-06-03T01:27:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.004+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d804f2e0 con 0x7f53d80ffc20 2024-06-03T01:27:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.004+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f53d40105d0 con 0x7f53d80ffc20 2024-06-03T01:27:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.005+0000 7f53ce7fc700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 0x7f53c40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.005+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f53d408dfe0 con 0x7f53d80ffc20 2024-06-03T01:27:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.005+0000 7f53dcce7700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 0x7f53c40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.006+0000 7f53dcce7700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 0x7f53c40706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b5c0 tx=0x7f53c8005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.010+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f53d40573f0 con 0x7f53d80ffc20 2024-06-03T01:27:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: pgmap v64: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:31.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-03T01:27:31.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: Upgrade: osd.1 is safe to restart 2024-06-03T01:27:31.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:31.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T01:27:31.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.249+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53d8068db0 con 0x7f53c406e200 2024-06-03T01:27:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.251+0000 7f53ce7fc700 1 -- 172.21.15.32:0/4096412269 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f53d8068db0 con 0x7f53c406e200 2024-06-03T01:27:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 msgr2=0x7f53c40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 0x7f53c40706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b5c0 tx=0x7f53c8005960 comp rx=0 tx=0).stop 2024-06-03T01:27:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 msgr2=0x7f53d8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53d400d8d0 tx=0x7f53d400dc90 comp rx=0 tx=0).stop 2024-06-03T01:27:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 shutdown_connections 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f53c406e200 0x7f53c40706c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53d80ffc20 0x7f53d8101840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 --2- 172.21.15.32:0/4096412269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8105bd0 0x7f53d8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 >> 172.21.15.32:0/4096412269 conn(0x7f53d8075670 msgr2=0x7f53d80686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 shutdown_connections 2024-06-03T01:27:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.253+0000 7f53df74c700 1 -- 172.21.15.32:0/4096412269 wait complete. 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 -- 172.21.15.32:0/188651715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01640ffb60 msgr2=0x7f01640fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 --2- 172.21.15.32:0/188651715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01640ffb60 0x7f01640fffc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0154009910 tx=0x7f0154009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 -- 172.21.15.32:0/188651715 shutdown_connections 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 --2- 172.21.15.32:0/188651715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01640ffb60 0x7f01640fffc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 --2- 172.21.15.32:0/188651715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0164105a20 0x7f0164105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 -- 172.21.15.32:0/188651715 >> 172.21.15.32:0/188651715 conn(0x7f01640754a0 msgr2=0x7f01640758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 -- 172.21.15.32:0/188651715 shutdown_connections 2024-06-03T01:27:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 -- 172.21.15.32:0/188651715 wait complete. 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.364+0000 7f016b606700 1 Processor -- start 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f016b606700 1 -- start start 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f016b606700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 0x7f01641015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f016b606700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f016b606700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0164102070 con 0x7f0164105a20 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f016b606700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01641021b0 con 0x7f01640ffb60 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51296/0 (socket says 172.21.15.32:51296) 2024-06-03T01:27:31.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 -- 172.21.15.32:0/155373439 learned_addr learned my addr 172.21.15.32:0/155373439 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f01693a2700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 0x7f01641015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 -- 172.21.15.32:0/155373439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 msgr2=0x7f01641015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 0x7f01641015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 -- 172.21.15.32:0/155373439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540095f0 con 0x7f0164105a20 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f01693a2700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 0x7f01641015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.365+0000 7f0168ba1700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0154005f50 tx=0x7f015400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.366+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f015401d070 con 0x7f0164105a20 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.366+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f015400b610 con 0x7f0164105a20 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.366+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0164102430 con 0x7f0164105a20 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.366+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0154022470 con 0x7f0164105a20 2024-06-03T01:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.366+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01641a2cb0 con 0x7f0164105a20 2024-06-03T01:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.367+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0164108fc0 con 0x7f0164105a20 2024-06-03T01:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.367+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f015400fca0 con 0x7f0164105a20 2024-06-03T01:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.368+0000 7f015a7fc700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 0x7f0150070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.368+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f015408f550 con 0x7f0164105a20 2024-06-03T01:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.368+0000 7f01693a2700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 0x7f0150070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.369+0000 7f01693a2700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 0x7f0150070710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0160009cb0 tx=0x7f0160009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.373+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f01540588c0 con 0x7f0164105a20 2024-06-03T01:27:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.595+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01640611d0 con 0x7f015006e250 2024-06-03T01:27:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.606+0000 7f015a7fc700 1 -- 172.21.15.32:0/155373439 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f01640611d0 con 0x7f015006e250 2024-06-03T01:27:31.606 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:27:31.606 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (3m) 22s ago 14m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:27:31.606 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (15m) 22s ago 14m 8228k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:27:31.606 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (13m) 45s ago 13m 8002k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (54s) 22s ago 14m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (49s) 45s ago 13m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (3m) 22s ago 14m 80.2M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (9m) 22s ago 9m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (9m) 22s ago 9m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (9m) 45s ago 9m 16.0M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (9m) 45s ago 9m 17.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (5m) 22s ago 16m 501M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (4m) 45s ago 13m 450M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (2m) 22s ago 16m 45.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (98s) 45s ago 13m 37.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (4m) 22s ago 14m 15.2M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:27:31.607 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 45s ago 13m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (28s) 22s ago 12m 15.0M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (12m) 22s ago 12m 241M 1217M 18.2.1 5be31c24972a 43b833a83c7a 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (11m) 22s ago 11m 205M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (11m) 45s ago 11m 269M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (11m) 45s ago 11m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (10m) 45s ago 10m 231M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:27:31.608 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (4m) 22s ago 14m 76.0M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 msgr2=0x7f0150070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 0x7f0150070710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0160009cb0 tx=0x7f0160009260 comp rx=0 tx=0).stop 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 msgr2=0x7f0164101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0154005f50 tx=0x7f015400f740 comp rx=0 tx=0).stop 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 shutdown_connections 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f015006e250 0x7f0150070710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01640ffb60 0x7f01641015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 --2- 172.21.15.32:0/155373439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0164105a20 0x7f0164101b30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 >> 172.21.15.32:0/155373439 conn(0x7f01640754a0 msgr2=0x7f0164068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 shutdown_connections 2024-06-03T01:27:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.608+0000 7f016b606700 1 -- 172.21.15.32:0/155373439 wait complete. 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.715+0000 7fac78201700 1 -- 172.21.15.32:0/2138777587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 msgr2=0x7fac70105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.715+0000 7fac78201700 1 --2- 172.21.15.32:0/2138777587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70105840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fac60009910 tx=0x7fac60009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 -- 172.21.15.32:0/2138777587 shutdown_connections 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 --2- 172.21.15.32:0/2138777587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70105840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 --2- 172.21.15.32:0/2138777587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac700ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 -- 172.21.15.32:0/2138777587 >> 172.21.15.32:0/2138777587 conn(0x7fac70074e30 msgr2=0x7fac70075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 -- 172.21.15.32:0/2138777587 shutdown_connections 2024-06-03T01:27:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 -- 172.21.15.32:0/2138777587 wait complete. 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.716+0000 7fac78201700 1 Processor -- start 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac78201700 1 -- start start 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac78201700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac70196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac78201700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac78201700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac70197560 con 0x7fac700fff50 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac78201700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7019b370 con 0x7fac700ff5a0 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51314/0 (socket says 172.21.15.32:51314) 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 -- 172.21.15.32:0/3042967508 learned_addr learned my addr 172.21.15.32:0/3042967508 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac75f9d700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac70196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 -- 172.21.15.32:0/3042967508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 msgr2=0x7fac70196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac70196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 -- 172.21.15.32:0/3042967508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600095f0 con 0x7fac700fff50 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac75f9d700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac70196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac7579c700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f50 tx=0x7fac6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.717+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6001d070 con 0x7fac700fff50 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.718+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac6000f460 con 0x7fac700fff50 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.718+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7019b5f0 con 0x7fac700fff50 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.718+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac60017630 con 0x7fac700fff50 2024-06-03T01:27:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.718+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7019bae0 con 0x7fac700fff50 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.719+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac70109220 con 0x7fac700fff50 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.719+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fac6000f5d0 con 0x7fac700fff50 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.720+0000 7fac66ffd700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 0x7fac5c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.720+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fac6008f6f0 con 0x7fac700fff50 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.720+0000 7fac75f9d700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 0x7fac5c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.720+0000 7fac75f9d700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 0x7fac5c0706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0099d0 tx=0x7fac6c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:31.725+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fac6005c5b0 con 0x7fac700fff50 2024-06-03T01:27:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:31 smithi032 ceph-mon[111919]: Upgrade: Updating osd.1 2024-06-03T01:27:31.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:31 smithi032 ceph-mon[111919]: Deploying daemon osd.1 on smithi032 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.024+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fac70197cf0 con 0x7fac700fff50 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.025+0000 7fac66ffd700 1 -- 172.21.15.32:0/3042967508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+787 (secure 0 0 0) 0x7fac6005c140 con 0x7fac700fff50 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:27:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 1 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:27:32.026 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 msgr2=0x7fac5c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 0x7fac5c0706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0099d0 tx=0x7fac6c008040 comp rx=0 tx=0).stop 2024-06-03T01:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 msgr2=0x7fac70196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f50 tx=0x7fac6000be20 comp rx=0 tx=0).stop 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 shutdown_connections 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac5c06e200 0x7fac5c0706c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac700ff5a0 0x7fac70196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 --2- 172.21.15.32:0/3042967508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac700fff50 0x7fac70196ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 >> 172.21.15.32:0/3042967508 conn(0x7fac70074e30 msgr2=0x7fac700fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 shutdown_connections 2024-06-03T01:27:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.027+0000 7fac78201700 1 -- 172.21.15.32:0/3042967508 wait complete. 2024-06-03T01:27:32.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 -- 172.21.15.32:0/3440031484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54101850 msgr2=0x7f0d541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 --2- 172.21.15.32:0/3440031484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54101850 0x7f0d541058a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009910 tx=0x7f0d3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 -- 172.21.15.32:0/3440031484 shutdown_connections 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 --2- 172.21.15.32:0/3440031484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54101850 0x7f0d541058a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 --2- 172.21.15.32:0/3440031484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54100ea0 0x7f0d54101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 -- 172.21.15.32:0/3440031484 >> 172.21.15.32:0/3440031484 conn(0x7f0d540fc750 msgr2=0x7f0d540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 -- 172.21.15.32:0/3440031484 shutdown_connections 2024-06-03T01:27:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.136+0000 7f0d59271700 1 -- 172.21.15.32:0/3440031484 wait complete. 2024-06-03T01:27:32.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 Processor -- start 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 -- start start 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54101850 0x7f0d5419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5419d2d0 con 0x7f0d54100ea0 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d59271700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54196780 con 0x7f0d54101850 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d52ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d52ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51328/0 (socket says 172.21.15.32:51328) 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.137+0000 7f0d52ffd700 1 -- 172.21.15.32:0/1207019662 learned_addr learned my addr 172.21.15.32:0/1207019662 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d527fc700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54101850 0x7f0d5419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d52ffd700 1 -- 172.21.15.32:0/1207019662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54101850 msgr2=0x7f0d5419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d52ffd700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54101850 0x7f0d5419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d52ffd700 1 -- 172.21.15.32:0/1207019662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c0095f0 con 0x7f0d54100ea0 2024-06-03T01:27:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d52ffd700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400d8d0 tx=0x7f0d4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d44009940 con 0x7f0d54100ea0 2024-06-03T01:27:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d44010460 con 0x7f0d54100ea0 2024-06-03T01:27:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d54196a60 con 0x7f0d54100ea0 2024-06-03T01:27:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4400f5f0 con 0x7f0d54100ea0 2024-06-03T01:27:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.138+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d54196fb0 con 0x7f0d54100ea0 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.139+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d541092a0 con 0x7f0d54100ea0 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.140+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0d440105d0 con 0x7f0d54100ea0 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.140+0000 7f0d4bfff700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 0x7f0d400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.140+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0d4408dfe0 con 0x7f0d54100ea0 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.140+0000 7f0d527fc700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 0x7f0d400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.141+0000 7f0d527fc700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 0x7f0d400706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d54197bf0 tx=0x7f0d3c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.145+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0d4405af20 con 0x7f0d54100ea0 2024-06-03T01:27:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:31 smithi044 ceph-mon[84214]: Upgrade: Updating osd.1 2024-06-03T01:27:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:31 smithi044 ceph-mon[84214]: Deploying daemon osd.1 on smithi032 2024-06-03T01:27:32.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.408+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d5404f2e0 con 0x7f0d54100ea0 2024-06-03T01:27:32.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.408+0000 7f0d4bfff700 1 -- 172.21.15.32:0/1207019662 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f0d44016770 con 0x7f0d54100ea0 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:27:32.409 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:27:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:27:32.411 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:27:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:27:32.412 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:27:32.412 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:27:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 msgr2=0x7f0d400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 0x7f0d400706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d54197bf0 tx=0x7f0d3c01a040 comp rx=0 tx=0).stop 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 msgr2=0x7f0d5419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400d8d0 tx=0x7f0d4400dc90 comp rx=0 tx=0).stop 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 shutdown_connections 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0d4006e200 0x7f0d400706c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d54100ea0 0x7f0d5419c700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 --2- 172.21.15.32:0/1207019662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d54101850 0x7f0d5419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.411+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 >> 172.21.15.32:0/1207019662 conn(0x7f0d540fc750 msgr2=0x7f0d540fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.412+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 shutdown_connections 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.412+0000 7f0d59271700 1 -- 172.21.15.32:0/1207019662 wait complete. 2024-06-03T01:27:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 -- 172.21.15.32:0/4045961716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 msgr2=0x7fe3fc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 --2- 172.21.15.32:0/4045961716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc101280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009910 tx=0x7fe3ec009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 -- 172.21.15.32:0/4045961716 shutdown_connections 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 --2- 172.21.15.32:0/4045961716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101850 0x7fe3fc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 --2- 172.21.15.32:0/4045961716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc101280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 -- 172.21.15.32:0/4045961716 >> 172.21.15.32:0/4045961716 conn(0x7fe3fc0fc750 msgr2=0x7fe3fc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 -- 172.21.15.32:0/4045961716 shutdown_connections 2024-06-03T01:27:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.524+0000 7fe401101700 1 -- 172.21.15.32:0/4045961716 wait complete. 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 Processor -- start 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 -- start start 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101850 0x7fe3fc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc19d2c0 con 0x7fe3fc100ea0 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe401101700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc196770 con 0x7fe3fc101850 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51346/0 (socket says 172.21.15.32:51346) 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 -- 172.21.15.32:0/1493126011 learned_addr learned my addr 172.21.15.32:0/1493126011 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 -- 172.21.15.32:0/1493126011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101850 msgr2=0x7fe3fc19cc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101850 0x7fe3fc19cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.525+0000 7fe3fad9d700 1 -- 172.21.15.32:0/1493126011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0095f0 con 0x7fe3fc100ea0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe3fad9d700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ec01d070 con 0x7fe3fc100ea0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc1969f0 con 0x7fe3fc100ea0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3ec00f460 con 0x7fe3fc100ea0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ec017630 con 0x7fe3fc100ea0 2024-06-03T01:27:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.526+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc196ee0 con 0x7fe3fc100ea0 2024-06-03T01:27:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.528+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc109240 con 0x7fe3fc100ea0 2024-06-03T01:27:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.528+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe3ec00f5d0 con 0x7fe3fc100ea0 2024-06-03T01:27:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.528+0000 7fe3f8d99700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 0x7fe3e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.529+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe3ec08f630 con 0x7fe3fc100ea0 2024-06-03T01:27:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.529+0000 7fe3f3fff700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 0x7fe3e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.529+0000 7fe3f3fff700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 0x7fe3e80706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc197be0 tx=0x7fe3e400b3f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.533+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe3ec05c450 con 0x7fe3fc100ea0 2024-06-03T01:27:32.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.768+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3fc197d60 con 0x7fe3e806e200 2024-06-03T01:27:32.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.769+0000 7fe3f8d99700 1 -- 172.21.15.32:0/1493126011 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe3fc197d60 con 0x7fe3e806e200 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:27:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "7/23 daemons upgraded", 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:27:32.770 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:27:32.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 msgr2=0x7fe3e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 0x7fe3e80706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc197be0 tx=0x7fe3e400b3f0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 msgr2=0x7fe3fc19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec00be20 comp rx=0 tx=0).stop 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 shutdown_connections 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe3e806e200 0x7fe3e80706c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc100ea0 0x7fe3fc19c6f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 --2- 172.21.15.32:0/1493126011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101850 0x7fe3fc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 >> 172.21.15.32:0/1493126011 conn(0x7fe3fc0fc750 msgr2=0x7fe3fc0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 shutdown_connections 2024-06-03T01:27:32.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.771+0000 7fe401101700 1 -- 172.21.15.32:0/1493126011 wait complete. 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: pgmap v65: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3042967508' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:32 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1207019662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:27:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 -- 172.21.15.32:0/3347794991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8068580 msgr2=0x7fc1f80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/3347794991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8068580 0x7fc1f80689e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009910 tx=0x7fc1ec009c20 comp rx=0 tx=0).stop 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 -- 172.21.15.32:0/3347794991 shutdown_connections 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/3347794991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8068580 0x7fc1f80689e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/3347794991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8106cd0 0x7fc1f81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 -- 172.21.15.32:0/3347794991 >> 172.21.15.32:0/3347794991 conn(0x7fc1f8075670 msgr2=0x7fc1f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 -- 172.21.15.32:0/3347794991 shutdown_connections 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.883+0000 7fc1fea0d700 1 -- 172.21.15.32:0/3347794991 wait complete. 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 Processor -- start 2024-06-03T01:27:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 -- start start 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8068580 0x7fc1f819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f819f400 con 0x7fc1f8106cd0 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1fea0d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81988b0 con 0x7fc1f8068580 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.884+0000 7fc1f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51374/0 (socket says 172.21.15.32:51374) 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f77fe700 1 -- 172.21.15.32:0/2532025412 learned_addr learned my addr 172.21.15.32:0/2532025412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f7fff700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8068580 0x7fc1f819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f77fe700 1 -- 172.21.15.32:0/2532025412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8068580 msgr2=0x7fc1f819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f77fe700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8068580 0x7fc1f819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f77fe700 1 -- 172.21.15.32:0/2532025412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec0095f0 con 0x7fc1f8106cd0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f77fe700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005f50 tx=0x7fc1ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec01d070 con 0x7fc1f8106cd0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f8198b30 con 0x7fc1f8106cd0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1ec00b610 con 0x7fc1f8106cd0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec022470 con 0x7fc1f8106cd0 2024-06-03T01:27:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.885+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f8199020 con 0x7fc1f8106cd0 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.886+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f804f2e0 con 0x7fc1f8106cd0 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.887+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc1ec003780 con 0x7fc1f8106cd0 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.888+0000 7fc1f57fa700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 0x7fc1e00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.888+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc1ec08f3a0 con 0x7fc1f8106cd0 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.888+0000 7fc1f7fff700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 0x7fc1e00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:27:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.888+0000 7fc1f7fff700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 0x7fc1e00706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009c60 tx=0x7fc1e8009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:27:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:32.893+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc1ec05c2e0 con 0x7fc1f8106cd0 2024-06-03T01:27:33.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.192+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc1f8199dc0 con 0x7fc1f8106cd0 2024-06-03T01:27:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.192+0000 7fc1f57fa700 1 -- 172.21.15.32:0/2532025412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc1ec05be70 con 0x7fc1f8106cd0 2024-06-03T01:27:33.193 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:27:33.193 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:27:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:27:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 msgr2=0x7fc1e00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 0x7fc1e00706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009c60 tx=0x7fc1e8009260 comp rx=0 tx=0).stop 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 msgr2=0x7fc1f819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005f50 tx=0x7fc1ec00f690 comp rx=0 tx=0).stop 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 shutdown_connections 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc1e006e200 0x7fc1e00706c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1f8068580 0x7fc1f819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 --2- 172.21.15.32:0/2532025412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1f8106cd0 0x7fc1f819ed70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 >> 172.21.15.32:0/2532025412 conn(0x7fc1f8075670 msgr2=0x7fc1f80fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 shutdown_connections 2024-06-03T01:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:27:33.195+0000 7fc1fea0d700 1 -- 172.21.15.32:0/2532025412 wait complete. 2024-06-03T01:27:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: pgmap v65: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3042967508' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:33.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:32 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1207019662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:27:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:33 smithi032 ceph-mon[111919]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:34.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:33 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2532025412' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:27:34.086 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:33 smithi032 systemd[1]: Stopping Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:27:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:33 smithi044 ceph-mon[84214]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:27:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:33 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2532025412' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:27:34.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:34 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:27:34.195+0000 7f0d6cf77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:34.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:34 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:27:34.195+0000 7f0d6cf77700 -1 osd.1 54 *** Got signal Terminated *** 2024-06-03T01:27:34.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:34 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[55991]: 2024-06-03T01:27:34.195+0000 7f0d6cf77700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:27:35.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:34 smithi032 ceph-mon[111919]: pgmap v66: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:35.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:34 smithi032 ceph-mon[111919]: osd.1 marked itself down and dead 2024-06-03T01:27:35.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:34 smithi044 ceph-mon[84214]: pgmap v66: 65 pgs: 65 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:35.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:34 smithi044 ceph-mon[84214]: osd.1 marked itself down and dead 2024-06-03T01:27:35.430 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:35 smithi032 podman[128452]: 2024-06-03 01:27:35.145927556 +0000 UTC m=+1.099830300 container died 43b833a83c7add99b51cc31b960f8430be3e601520486cd5ca53fc41b1e40ae0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:35 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:27:36.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:35 smithi032 ceph-mon[111919]: osdmap e55: 6 total, 5 up, 6 in 2024-06-03T01:27:36.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:35 smithi032 podman[128452]: 2024-06-03 01:27:35.762674422 +0000 UTC m=+1.716577150 container cleanup 43b833a83c7add99b51cc31b960f8430be3e601520486cd5ca53fc41b1e40ae0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-06-03T01:27:36.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:35 smithi032 bash[128452]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1 2024-06-03T01:27:36.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:35 smithi032 podman[128468]: 2024-06-03 01:27:35.981088206 +0000 UTC m=+0.830691254 container remove 43b833a83c7add99b51cc31b960f8430be3e601520486cd5ca53fc41b1e40ae0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, RELEASE=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-06-03T01:27:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:35 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:27:36.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:35 smithi044 ceph-mon[84214]: osdmap e55: 6 total, 5 up, 6 in 2024-06-03T01:27:36.700 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.30845633 +0000 UTC m=+0.018242211 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:36.701 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.407154287 +0000 UTC m=+0.116940154 container create 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-06-03T01:27:36.701 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.618114029 +0000 UTC m=+0.327899897 container init 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:36.701 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.621632357 +0000 UTC m=+0.331418225 container start 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-03T01:27:36.701 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.701190683 +0000 UTC m=+0.410976573 container attach 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-03T01:27:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:36 smithi032 ceph-mon[111919]: pgmap v68: 65 pgs: 12 stale+active+clean, 53 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:36 smithi032 ceph-mon[111919]: osdmap e56: 6 total, 5 up, 6 in 2024-06-03T01:27:37.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:36 smithi032 podman[128559]: 2024-06-03 01:27:36.927803944 +0000 UTC m=+0.637589813 container died 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-03T01:27:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:36 smithi044 ceph-mon[84214]: pgmap v68: 65 pgs: 12 stale+active+clean, 53 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:36 smithi044 ceph-mon[84214]: osdmap e56: 6 total, 5 up, 6 in 2024-06-03T01:27:37.838 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:37 smithi032 podman[128584]: 2024-06-03 01:27:37.5886724 +0000 UTC m=+0.654175574 container remove 31553d07a97fb38ee138500a0165f33efb0efc41c3cfea5efd75f2205994cd26 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-03T01:27:37.838 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:37 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service: Deactivated successfully. 2024-06-03T01:27:37.839 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:37 smithi032 systemd[1]: Stopped Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:27:37.839 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:37 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service: Consumed 1min 5.995s CPU time. 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 1]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1, 0, 5]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [3, 5]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 2]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]: dispatch 2024-06-03T01:27:37.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 0]}]: dispatch 2024-06-03T01:27:37.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:37.840 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:38.157 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:37 smithi032 systemd[1]: Starting Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:27:38.158 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:37.962114595 +0000 UTC m=+0.018610895 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:38.158 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:38.064745515 +0000 UTC m=+0.121241794 container create 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-03T01:27:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-06-03T01:27:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 1]}]: dispatch 2024-06-03T01:27:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1, 0, 5]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [3, 5]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 2]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 0]}]: dispatch 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:38.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:38.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:38.267509678 +0000 UTC m=+0.324005958 container init 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T01:27:38.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:38.27091108 +0000 UTC m=+0.327407369 container start 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-03T01:27:38.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:38.350709781 +0000 UTC m=+0.407206076 container attach 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 1]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1, 0, 5]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [3, 5]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 2]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]': finished 2024-06-03T01:27:39.063 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 0]}]': finished 2024-06-03T01:27:39.064 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-mon[111919]: osdmap e57: 6 total, 5 up, 6 in 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 bash[128682]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-03T01:27:39.064 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-03T01:27:39.065 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-03T01:27:39.065 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-03T01:27:39.065 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-03T01:27:39.065 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate[128696]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-03T01:27:39.065 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:38 smithi032 podman[128682]: 2024-06-03 01:27:38.7609937 +0000 UTC m=+0.817489983 container died 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:27:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-06-03T01:27:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 1]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1, 0, 5]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [3, 5]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 2]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 0]}]': finished 2024-06-03T01:27:39.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:38 smithi044 ceph-mon[84214]: osdmap e57: 6 total, 5 up, 6 in 2024-06-03T01:27:39.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 podman[128835]: 2024-06-03 01:27:39.321284024 +0000 UTC m=+0.553774798 container remove 74e2ac4360a31b1c39ea7062ce8400bf642672df9d330e938c2d0c3018a0044f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T01:27:39.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 podman[128895]: 2024-06-03 01:27:39.530507713 +0000 UTC m=+0.103053210 container create 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-03T01:27:39.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 podman[128895]: 2024-06-03 01:27:39.445833175 +0000 UTC m=+0.018378702 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:39.924 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:39 smithi032 ceph-mon[111919]: osdmap e58: 6 total, 5 up, 6 in 2024-06-03T01:27:39.924 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:39 smithi032 ceph-mon[111919]: osdmap e59: 6 total, 5 up, 6 in 2024-06-03T01:27:39.924 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 podman[128895]: 2024-06-03 01:27:39.724641816 +0000 UTC m=+0.297187332 container init 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-03T01:27:39.925 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 podman[128895]: 2024-06-03 01:27:39.729420791 +0000 UTC m=+0.301966303 container start 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-03T01:27:39.925 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 ceph-osd[128915]: -- 172.21.15.32:0/3903334695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b799f9680 con 0x560b799c3000 2024-06-03T01:27:39.925 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 bash[128895]: 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd 2024-06-03T01:27:39.925 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:39 smithi032 systemd[1]: Started Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:27:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:39 smithi044 ceph-mon[84214]: osdmap e58: 6 total, 5 up, 6 in 2024-06-03T01:27:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:39 smithi044 ceph-mon[84214]: osdmap e59: 6 total, 5 up, 6 in 2024-06-03T01:27:40.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:27:40.270+0000 7f59dceb07c0 -1 Falling back to public interface 2024-06-03T01:27:41.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-mon[111919]: pgmap v73: 65 pgs: 4 unknown, 2 peering, 13 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 30 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/345 objects degraded (13.333%) 2024-06-03T01:27:41.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 46/345 objects degraded (13.333%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:41.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:41.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:41.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:40 smithi044 ceph-mon[84214]: pgmap v73: 65 pgs: 4 unknown, 2 peering, 13 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 30 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/345 objects degraded (13.333%) 2024-06-03T01:27:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:40 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 46/345 objects degraded (13.333%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:41.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:41.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:41 smithi032 ceph-mon[111919]: osdmap e60: 6 total, 5 up, 6 in 2024-06-03T01:27:41.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:41 smithi032 ceph-mon[111919]: osdmap e61: 6 total, 5 up, 6 in 2024-06-03T01:27:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:41 smithi044 ceph-mon[84214]: osdmap e60: 6 total, 5 up, 6 in 2024-06-03T01:27:42.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:41 smithi044 ceph-mon[84214]: osdmap e61: 6 total, 5 up, 6 in 2024-06-03T01:27:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:42 smithi032 ceph-mon[111919]: pgmap v76: 65 pgs: 2 remapped+peering, 27 active+clean, 16 active+undersized, 1 stale+active+clean, 14 active+undersized+degraded, 2 peering, 3 unknown; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50/357 objects degraded (14.006%) 2024-06-03T01:27:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:42 smithi032 ceph-mon[111919]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:27:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:42 smithi032 ceph-mon[111919]: osdmap e62: 6 total, 5 up, 6 in 2024-06-03T01:27:43.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:42 smithi044 ceph-mon[84214]: pgmap v76: 65 pgs: 2 remapped+peering, 27 active+clean, 16 active+undersized, 1 stale+active+clean, 14 active+undersized+degraded, 2 peering, 3 unknown; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50/357 objects degraded (14.006%) 2024-06-03T01:27:43.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:42 smithi044 ceph-mon[84214]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:27:43.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:42 smithi044 ceph-mon[84214]: osdmap e62: 6 total, 5 up, 6 in 2024-06-03T01:27:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:44 smithi032 ceph-mon[111919]: pgmap v79: 65 pgs: 6 active+clean+remapped, 2 remapped+peering, 26 active+clean, 16 active+undersized, 1 stale+active+clean, 14 active+undersized+degraded; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 MiB/s rd, 63 MiB/s wr, 716 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 6.0 MiB/s, 1 objects/s recovering 2024-06-03T01:27:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:44 smithi032 ceph-mon[111919]: osdmap e63: 6 total, 5 up, 6 in 2024-06-03T01:27:45.085 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:44 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:27:44.774+0000 7f59dceb07c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-06-03T01:27:45.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:44 smithi044 ceph-mon[84214]: pgmap v79: 65 pgs: 6 active+clean+remapped, 2 remapped+peering, 26 active+clean, 16 active+undersized, 1 stale+active+clean, 14 active+undersized+degraded; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 MiB/s rd, 63 MiB/s wr, 716 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 6.0 MiB/s, 1 objects/s recovering 2024-06-03T01:27:45.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:44 smithi044 ceph-mon[84214]: osdmap e63: 6 total, 5 up, 6 in 2024-06-03T01:27:45.835 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:45 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:27:45.488+0000 7f59dceb07c0 -1 osd.1 54 log_to_monitors true 2024-06-03T01:27:46.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-mon[111919]: pgmap v81: 65 pgs: 8 active+clean+remapped, 1 remapped+peering, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 MiB/s rd, 56 MiB/s wr, 437 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 5.8 MiB/s, 172 objects/s recovering 2024-06-03T01:27:46.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:46.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:46.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-mon[111919]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T01:27:46.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-mon[111919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:27:46.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:46 smithi044 ceph-mon[84214]: pgmap v81: 65 pgs: 8 active+clean+remapped, 1 remapped+peering, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 316 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 MiB/s rd, 56 MiB/s wr, 437 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 5.8 MiB/s, 172 objects/s recovering 2024-06-03T01:27:46.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:46.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:46.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:46 smithi044 ceph-mon[84214]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T01:27:46.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:46 smithi044 ceph-mon[84214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:27:47.268 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:27:46 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:27:46.932+0000 7f59ca2ff700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:27:47.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T01:27:47.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: osdmap e64: 6 total, 5 up, 6 in 2024-06-03T01:27:47.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:27:47.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:47.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:47.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 50/366 objects degraded (13.661%), 14 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:47.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:47 smithi032 ceph-mon[111919]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' 2024-06-03T01:27:47.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: osdmap e64: 6 total, 5 up, 6 in 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 50/366 objects degraded (13.661%), 14 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:47.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:47 smithi044 ceph-mon[84214]: from='osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249]' entity='osd.1' 2024-06-03T01:27:48.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:48 smithi032 ceph-mon[111919]: pgmap v83: 65 pgs: 8 active+clean+remapped, 1 remapped+peering, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 MiB/s rd, 45 MiB/s wr, 348 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 4.6 MiB/s, 137 objects/s recovering 2024-06-03T01:27:48.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:48 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:27:48.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:48 smithi032 ceph-mon[111919]: osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249] boot 2024-06-03T01:27:48.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:48 smithi032 ceph-mon[111919]: osdmap e65: 6 total, 6 up, 6 in 2024-06-03T01:27:48.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:27:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:48 smithi044 ceph-mon[84214]: pgmap v83: 65 pgs: 8 active+clean+remapped, 1 remapped+peering, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 MiB/s rd, 45 MiB/s wr, 348 op/s; 50/366 objects degraded (13.661%); 15/366 objects misplaced (4.098%); 4.6 MiB/s, 137 objects/s recovering 2024-06-03T01:27:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:48 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:27:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:48 smithi044 ceph-mon[84214]: osd.1 [v2:172.21.15.32:6810/3877677249,v1:172.21.15.32:6811/3877677249] boot 2024-06-03T01:27:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:48 smithi044 ceph-mon[84214]: osdmap e65: 6 total, 6 up, 6 in 2024-06-03T01:27:48.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:27:50.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:49 smithi044 ceph-mon[84214]: osdmap e66: 6 total, 6 up, 6 in 2024-06-03T01:27:50.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:49 smithi044 ceph-mon[84214]: pgmap v86: 65 pgs: 19 peering, 2 active+clean+remapped, 7 remapped+peering, 26 active+clean, 6 active+undersized, 5 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%); 0 B/s, 278 objects/s recovering 2024-06-03T01:27:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:49 smithi032 ceph-mon[111919]: osdmap e66: 6 total, 6 up, 6 in 2024-06-03T01:27:50.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:49 smithi032 ceph-mon[111919]: pgmap v86: 65 pgs: 19 peering, 2 active+clean+remapped, 7 remapped+peering, 26 active+clean, 6 active+undersized, 5 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%); 0 B/s, 278 objects/s recovering 2024-06-03T01:27:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: osdmap e67: 6 total, 6 up, 6 in 2024-06-03T01:27:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:27:51.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:27:51.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:50 smithi044 ceph-mon[84214]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-03T01:27:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: osdmap e67: 6 total, 6 up, 6 in 2024-06-03T01:27:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:27:51.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:50 smithi032 ceph-mon[111919]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-03T01:27:52.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:51 smithi044 ceph-mon[84214]: osdmap e68: 6 total, 6 up, 6 in 2024-06-03T01:27:52.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:51 smithi044 ceph-mon[84214]: pgmap v89: 65 pgs: 19 peering, 1 active+clean+remapped, 8 remapped+peering, 28 active+clean, 4 active+undersized, 5 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%); 0 B/s, 185 objects/s recovering 2024-06-03T01:27:52.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:51 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:52.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:51 smithi044 ceph-mon[84214]: osdmap e69: 6 total, 6 up, 6 in 2024-06-03T01:27:52.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:51 smithi032 ceph-mon[111919]: osdmap e68: 6 total, 6 up, 6 in 2024-06-03T01:27:52.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:51 smithi032 ceph-mon[111919]: pgmap v89: 65 pgs: 19 peering, 1 active+clean+remapped, 8 remapped+peering, 28 active+clean, 4 active+undersized, 5 active+undersized+degraded; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%); 0 B/s, 185 objects/s recovering 2024-06-03T01:27:52.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:51 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded (PG_DEGRADED) 2024-06-03T01:27:52.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:51 smithi032 ceph-mon[111919]: osdmap e69: 6 total, 6 up, 6 in 2024-06-03T01:27:53.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:53.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:53 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:53.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:53 smithi032 ceph-mon[111919]: osdmap e70: 6 total, 6 up, 6 in 2024-06-03T01:27:53.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:27:53.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:53 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:27:53.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:53 smithi044 ceph-mon[84214]: osdmap e70: 6 total, 6 up, 6 in 2024-06-03T01:27:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:54 smithi032 ceph-mon[111919]: pgmap v92: 65 pgs: 3 peering, 1 remapped+peering, 61 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 196 objects/s recovering 2024-06-03T01:27:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:54 smithi032 ceph-mon[111919]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:27:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:54 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded) 2024-06-03T01:27:54.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:54 smithi032 ceph-mon[111919]: osdmap e71: 6 total, 6 up, 6 in 2024-06-03T01:27:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:54 smithi044 ceph-mon[84214]: pgmap v92: 65 pgs: 3 peering, 1 remapped+peering, 61 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 196 objects/s recovering 2024-06-03T01:27:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:54 smithi044 ceph-mon[84214]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:27:54.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:54 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded) 2024-06-03T01:27:54.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:54 smithi044 ceph-mon[84214]: osdmap e71: 6 total, 6 up, 6 in 2024-06-03T01:27:56.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:56 smithi032 ceph-mon[111919]: pgmap v94: 65 pgs: 3 peering, 1 remapped+peering, 61 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 195 objects/s recovering 2024-06-03T01:27:56.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:56 smithi044 ceph-mon[84214]: pgmap v94: 65 pgs: 3 peering, 1 remapped+peering, 61 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 195 objects/s recovering 2024-06-03T01:27:57.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:57 smithi032 ceph-mon[111919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:27:57.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:57 smithi044 ceph-mon[84214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:27:58.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:27:58 smithi032 ceph-mon[111919]: pgmap v95: 65 pgs: 1 peering, 64 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s, 250 objects/s recovering 2024-06-03T01:27:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:27:58 smithi044 ceph-mon[84214]: pgmap v95: 65 pgs: 1 peering, 64 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s, 250 objects/s recovering 2024-06-03T01:28:00.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:00 smithi032 ceph-mon[111919]: pgmap v96: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 206 objects/s recovering 2024-06-03T01:28:00.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:00 smithi044 ceph-mon[84214]: pgmap v96: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 206 objects/s recovering 2024-06-03T01:28:02.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:02 smithi032 ceph-mon[111919]: pgmap v97: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 181 objects/s recovering 2024-06-03T01:28:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:02 smithi044 ceph-mon[84214]: pgmap v97: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 181 objects/s recovering 2024-06-03T01:28:03.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 -- 172.21.15.32:0/2886603848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 msgr2=0x7fb3b8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/2886603848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b8100080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009910 tx=0x7fb3ac009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 -- 172.21.15.32:0/2886603848 shutdown_connections 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/2886603848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b8100080 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/2886603848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 0x7fb3b8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 -- 172.21.15.32:0/2886603848 >> 172.21.15.32:0/2886603848 conn(0x7fb3b8075670 msgr2=0x7fb3b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 -- 172.21.15.32:0/2886603848 shutdown_connections 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.319+0000 7fb3bcea3700 1 -- 172.21.15.32:0/2886603848 wait complete. 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 Processor -- start 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 -- start start 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 0x7fb3b819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b819d2b0 con 0x7fb3b80ffc20 2024-06-03T01:28:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3bcea3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8196730 con 0x7fb3b8105bd0 2024-06-03T01:28:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40558/0 (socket says 172.21.15.32:40558) 2024-06-03T01:28:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.320+0000 7fb3b659c700 1 -- 172.21.15.32:0/1593127128 learned_addr learned my addr 172.21.15.32:0/1593127128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3b5d9b700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 0x7fb3b819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3b659c700 1 -- 172.21.15.32:0/1593127128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 msgr2=0x7fb3b819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3b659c700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 0x7fb3b819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3b659c700 1 -- 172.21.15.32:0/1593127128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0095f0 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3b659c700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800e920 tx=0x7fb3a800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a800c9d0 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b8196a10 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a8010430 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a8018630 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.321+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b8196f60 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.322+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b81080d0 con 0x7fb3b80ffc20 2024-06-03T01:28:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.323+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb3a80105a0 con 0x7fb3b80ffc20 2024-06-03T01:28:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.323+0000 7fb3a77fe700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 0x7fb3a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.323+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fb3a8014070 con 0x7fb3b80ffc20 2024-06-03T01:28:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.323+0000 7fb3b5d9b700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 0x7fb3a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.324+0000 7fb3b5d9b700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 0x7fb3a00706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8197f20 tx=0x7fb3ac005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.328+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb3a8053850 con 0x7fb3b80ffc20 2024-06-03T01:28:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.555+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b8197da0 con 0x7fb3a006e200 2024-06-03T01:28:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.557+0000 7fb3a77fe700 1 -- 172.21.15.32:0/1593127128 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb3ac01e040 con 0x7fb3a006e200 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 msgr2=0x7fb3a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 0x7fb3a00706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8197f20 tx=0x7fb3ac005940 comp rx=0 tx=0).stop 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 msgr2=0x7fb3b819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800e920 tx=0x7fb3a800ec30 comp rx=0 tx=0).stop 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 shutdown_connections 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb3a006e200 0x7fb3a00706c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b80ffc20 0x7fb3b819c690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 --2- 172.21.15.32:0/1593127128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b8105bd0 0x7fb3b819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 >> 172.21.15.32:0/1593127128 conn(0x7fb3b8075670 msgr2=0x7fb3b80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 shutdown_connections 2024-06-03T01:28:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.559+0000 7fb3bcea3700 1 -- 172.21.15.32:0/1593127128 wait complete. 2024-06-03T01:28:03.571 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:28:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/2711583158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 msgr2=0x7f4fd41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/2711583158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd41058e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009910 tx=0x7f4fc4009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/2711583158 shutdown_connections 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/2711583158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd41058e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/2711583158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 0x7f4fd40ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/2711583158 >> 172.21.15.32:0/2711583158 conn(0x7f4fd4074df0 msgr2=0x7f4fd4075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/2711583158 shutdown_connections 2024-06-03T01:28:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.669+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/2711583158 wait complete. 2024-06-03T01:28:03.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 Processor -- start 2024-06-03T01:28:03.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 -- start start 2024-06-03T01:28:03.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 0x7f4fd419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419d290 con 0x7f4fd40fffb0 2024-06-03T01:28:03.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fdb5d5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4196740 con 0x7f4fd40ff600 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40584/0 (socket says 172.21.15.32:40584) 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 -- 172.21.15.32:0/3681588226 learned_addr learned my addr 172.21.15.32:0/3681588226 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd9371700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 0x7f4fd419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 -- 172.21.15.32:0/3681588226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 msgr2=0x7f4fd419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 0x7f4fd419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.670+0000 7f4fd8b70700 1 -- 172.21.15.32:0/3681588226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40095f0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fd8b70700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40092e0 tx=0x7f4fc400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc401d070 con 0x7f4fd40fffb0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd41969c0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fc4005dc0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc4017400 con 0x7f4fd40fffb0 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.671+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd4196eb0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.672+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd41091a0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.672+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4fc400f6f0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.673+0000 7f4fca7fc700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 0x7f4fc00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.673+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f4fc408f5d0 con 0x7f4fd40fffb0 2024-06-03T01:28:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.673+0000 7f4fd9371700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 0x7f4fc00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:03.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.673+0000 7f4fd9371700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 0x7f4fc00706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005950 tx=0x7f4fd000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.677+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4fc405c380 con 0x7f4fd40fffb0 2024-06-03T01:28:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.905+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fd4197cc0 con 0x7f4fc006e200 2024-06-03T01:28:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.906+0000 7f4fca7fc700 1 -- 172.21.15.32:0/3681588226 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4fd4197cc0 con 0x7f4fc006e200 2024-06-03T01:28:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 msgr2=0x7f4fc00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 0x7f4fc00706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005950 tx=0x7f4fd000b410 comp rx=0 tx=0).stop 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 msgr2=0x7f4fd419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40092e0 tx=0x7f4fc400bd70 comp rx=0 tx=0).stop 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 shutdown_connections 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4fc006e200 0x7f4fc00706c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd40ff600 0x7f4fd419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 --2- 172.21.15.32:0/3681588226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fd40fffb0 0x7f4fd419cc00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 >> 172.21.15.32:0/3681588226 conn(0x7f4fd4074df0 msgr2=0x7f4fd40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 shutdown_connections 2024-06-03T01:28:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:03.908+0000 7f4fdb5d5700 1 -- 172.21.15.32:0/3681588226 wait complete. 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 -- 172.21.15.32:0/1806892799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e580ffc20 msgr2=0x7f3e58100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 --2- 172.21.15.32:0/1806892799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e580ffc20 0x7f3e58100080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009910 tx=0x7f3e4c009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 -- 172.21.15.32:0/1806892799 shutdown_connections 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 --2- 172.21.15.32:0/1806892799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e580ffc20 0x7f3e58100080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 --2- 172.21.15.32:0/1806892799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e58105bd0 0x7f3e58105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 -- 172.21.15.32:0/1806892799 >> 172.21.15.32:0/1806892799 conn(0x7f3e58075670 msgr2=0x7f3e58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 -- 172.21.15.32:0/1806892799 shutdown_connections 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.016+0000 7f3e5e913700 1 -- 172.21.15.32:0/1806892799 wait complete. 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 Processor -- start 2024-06-03T01:28:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 -- start start 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e580ffc20 0x7f3e58101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e581022c0 con 0x7f3e58105bd0 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e5e913700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e58102400 con 0x7f3e580ffc20 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40598/0 (socket says 172.21.15.32:40598) 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.017+0000 7f3e577fe700 1 -- 172.21.15.32:0/533911400 learned_addr learned my addr 172.21.15.32:0/533911400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e57fff700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e580ffc20 0x7f3e58101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e577fe700 1 -- 172.21.15.32:0/533911400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e580ffc20 msgr2=0x7f3e58101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e577fe700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e580ffc20 0x7f3e58101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e577fe700 1 -- 172.21.15.32:0/533911400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0095f0 con 0x7f3e58105bd0 2024-06-03T01:28:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e577fe700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c01d070 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e58102680 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e4c00f460 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c017610 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.018+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e581a2d90 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.019+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5804f2e0 con 0x7f3e58105bd0 2024-06-03T01:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.020+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e4c00f5d0 con 0x7f3e58105bd0 2024-06-03T01:28:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.021+0000 7f3e557fa700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 0x7f3e400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.021+0000 7f3e57fff700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 0x7f3e400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.021+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f3e4c08f810 con 0x7f3e58105bd0 2024-06-03T01:28:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.021+0000 7f3e57fff700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 0x7f3e400706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005950 tx=0x7f3e4800a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.025+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3e4c0589f0 con 0x7f3e58105bd0 2024-06-03T01:28:04.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.244+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e58068db0 con 0x7f3e4006e200 2024-06-03T01:28:04.246 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:04 smithi032 ceph-mon[111919]: pgmap v98: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 71 objects/s recovering 2024-06-03T01:28:04.246 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:04 smithi032 ceph-mon[111919]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.256+0000 7f3e557fa700 1 -- 172.21.15.32:0/533911400 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3e58068db0 con 0x7f3e4006e200 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (4m) 19s ago 15m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (15m) 19s ago 15m 8556k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (14m) 78s ago 14m 8002k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (86s) 19s ago 15m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (81s) 78s ago 14m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (3m) 19s ago 15m 81.3M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:28:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (9m) 19s ago 9m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (9m) 19s ago 9m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (9m) 78s ago 9m 16.0M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (9m) 78s ago 9m 17.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (6m) 19s ago 16m 504M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (5m) 78s ago 14m 450M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (2m) 19s ago 16m 47.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (2m) 78s ago 14m 37.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 19s ago 15m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 78s ago 14m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (60s) 19s ago 13m 120M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (24s) 19s ago 12m 15.0M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (12m) 19s ago 12m 234M 1217M 18.2.1 5be31c24972a 804fd2e4f195 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (12m) 78s ago 12m 269M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:28:04.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (11m) 78s ago 11m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (11m) 78s ago 11m 231M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (4m) 19s ago 14m 78.3M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 msgr2=0x7f3e400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 0x7f3e400706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005950 tx=0x7f3e4800a300 comp rx=0 tx=0).stop 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 msgr2=0x7f3e58101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c00bd70 comp rx=0 tx=0).stop 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 shutdown_connections 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3e4006e200 0x7f3e400706c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e580ffc20 0x7f3e58101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 --2- 172.21.15.32:0/533911400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e58105bd0 0x7f3e58101d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 >> 172.21.15.32:0/533911400 conn(0x7f3e58075670 msgr2=0x7f3e580686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 shutdown_connections 2024-06-03T01:28:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.258+0000 7f3e5e913700 1 -- 172.21.15.32:0/533911400 wait complete. 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 -- 172.21.15.32:0/1893487076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24101de0 msgr2=0x7fab241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 --2- 172.21.15.32:0/1893487076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24101de0 0x7fab241021c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab20009910 tx=0x7fab20009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 -- 172.21.15.32:0/1893487076 shutdown_connections 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 --2- 172.21.15.32:0/1893487076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24102700 0x7fab2410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 --2- 172.21.15.32:0/1893487076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24101de0 0x7fab241021c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.364+0000 7fab2c750700 1 -- 172.21.15.32:0/1893487076 >> 172.21.15.32:0/1893487076 conn(0x7fab240fb650 msgr2=0x7fab240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 -- 172.21.15.32:0/1893487076 shutdown_connections 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 -- 172.21.15.32:0/1893487076 wait complete. 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 Processor -- start 2024-06-03T01:28:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 -- start start 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24101de0 0x7fab240ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab241004c0 con 0x7fab24102700 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.365+0000 7fab2c750700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24100600 con 0x7fab24101de0 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40622/0 (socket says 172.21.15.32:40622) 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 -- 172.21.15.32:0/3959689513 learned_addr learned my addr 172.21.15.32:0/3959689513 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab2a4ec700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24101de0 0x7fab240ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 -- 172.21.15.32:0/3959689513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24101de0 msgr2=0x7fab240ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24101de0 0x7fab240ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 -- 172.21.15.32:0/3959689513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200095f0 con 0x7fab24102700 2024-06-03T01:28:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab29ceb700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e940 tx=0x7fab1400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab140096e0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab14004cf0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1400eee0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab241a2aa0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.366+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab241a2eb0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.367+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab241082f0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.368+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fab1400cba0 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.368+0000 7fab1b7fe700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 0x7fab100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.369+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fab14014070 con 0x7fab24102700 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.369+0000 7fab2a4ec700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 0x7fab100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.369+0000 7fab2a4ec700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 0x7fab100706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab2000b760 tx=0x7fab20009f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.374+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fab14057720 con 0x7fab24102700 2024-06-03T01:28:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:04 smithi044 ceph-mon[84214]: pgmap v98: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 71 objects/s recovering 2024-06-03T01:28:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:04 smithi044 ceph-mon[84214]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.670+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab24107cd0 con 0x7fab24102700 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.671+0000 7fab1b7fe700 1 -- 172.21.15.32:0/3959689513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+787 (secure 0 0 0) 0x7fab1405ade0 con 0x7fab24102700 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:28:04.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:28:04.673 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:28:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 msgr2=0x7fab100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 0x7fab100706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab2000b760 tx=0x7fab20009f90 comp rx=0 tx=0).stop 2024-06-03T01:28:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 msgr2=0x7fab240fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e940 tx=0x7fab1400ed00 comp rx=0 tx=0).stop 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 shutdown_connections 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fab1006e200 0x7fab100706c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab24101de0 0x7fab240ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 --2- 172.21.15.32:0/3959689513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab24102700 0x7fab240fff80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 >> 172.21.15.32:0/3959689513 conn(0x7fab240fb650 msgr2=0x7fab24105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 shutdown_connections 2024-06-03T01:28:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.673+0000 7fab2c750700 1 -- 172.21.15.32:0/3959689513 wait complete. 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- 172.21.15.32:0/228057752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc101e40 msgr2=0x7f46fc102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 --2- 172.21.15.32:0/228057752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc101e40 0x7f46fc102220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009910 tx=0x7f46f8009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- 172.21.15.32:0/228057752 shutdown_connections 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 --2- 172.21.15.32:0/228057752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102760 0x7f46fc10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 --2- 172.21.15.32:0/228057752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc101e40 0x7f46fc102220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- 172.21.15.32:0/228057752 >> 172.21.15.32:0/228057752 conn(0x7f46fc0fb650 msgr2=0x7f46fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- 172.21.15.32:0/228057752 shutdown_connections 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- 172.21.15.32:0/228057752 wait complete. 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 Processor -- start 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- start start 2024-06-03T01:28:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 0x7f46fc19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc19d260 con 0x7f46fc102760 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4704394700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc196710 con 0x7f46fc101e40 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40642/0 (socket says 172.21.15.32:40642) 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 -- 172.21.15.32:0/3387645852 learned_addr learned my addr 172.21.15.32:0/3387645852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4702130700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 0x7f46fc19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 -- 172.21.15.32:0/3387645852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 msgr2=0x7f46fc19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 0x7f46fc19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 -- 172.21.15.32:0/3387645852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80095f0 con 0x7f46fc102760 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f4702130700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 0x7f46fc19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.783+0000 7f470192f700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00d8d0 tx=0x7f46ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.784+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46ec0041f0 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.784+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc1969f0 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.784+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46ec004350 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.784+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46ec003d60 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.784+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc196f40 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.785+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc108350 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.785+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f46ec009730 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.786+0000 7f46f37fe700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 0x7f46e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.786+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f46ec08de30 con 0x7f46fc102760 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.786+0000 7f4702130700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 0x7f46e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.786+0000 7f4702130700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 0x7f46e80706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005b40 tx=0x7f46f801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:04.790+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f46ec05abe0 con 0x7f46fc102760 2024-06-03T01:28:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.044+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46fc04f2e0 con 0x7f46fc102760 2024-06-03T01:28:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.044+0000 7f46f37fe700 1 -- 172.21.15.32:0/3387645852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f46ec017770 con 0x7f46fc102760 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:28:05.046 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:28:05.047 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:28:05.048 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 msgr2=0x7f46e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 0x7f46e80706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005b40 tx=0x7f46f801a040 comp rx=0 tx=0).stop 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 msgr2=0x7f46fc19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00d8d0 tx=0x7f46ec00dc90 comp rx=0 tx=0).stop 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 shutdown_connections 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f46e806e200 0x7f46e80706c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc101e40 0x7f46fc19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 --2- 172.21.15.32:0/3387645852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46fc102760 0x7f46fc19cbd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 >> 172.21.15.32:0/3387645852 conn(0x7f46fc0fb650 msgr2=0x7f46fc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:05.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 shutdown_connections 2024-06-03T01:28:05.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.047+0000 7f4704394700 1 -- 172.21.15.32:0/3387645852 wait complete. 2024-06-03T01:28:05.051 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 -- 172.21.15.32:0/1197066128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 msgr2=0x7f2b081047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1197066128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b081047b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009910 tx=0x7f2af8009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 -- 172.21.15.32:0/1197066128 shutdown_connections 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1197066128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b081047b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1197066128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 0x7f2b08100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 -- 172.21.15.32:0/1197066128 >> 172.21.15.32:0/1197066128 conn(0x7f2b08074e30 msgr2=0x7f2b08075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 -- 172.21.15.32:0/1197066128 shutdown_connections 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.162+0000 7f2b1020a700 1 -- 172.21.15.32:0/1197066128 wait complete. 2024-06-03T01:28:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 Processor -- start 2024-06-03T01:28:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 -- start start 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 0x7f2b0819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0819d210 con 0x7f2b08101060 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b1020a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b081966c0 con 0x7f2b081006b0 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b0d7a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.163+0000 7f2b0d7a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40650/0 (socket says 172.21.15.32:40650) 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0d7a5700 1 -- 172.21.15.32:0/1675354357 learned_addr learned my addr 172.21.15.32:0/1675354357 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0dfa6700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 0x7f2b0819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0d7a5700 1 -- 172.21.15.32:0/1675354357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 msgr2=0x7f2b0819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0d7a5700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 0x7f2b0819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0d7a5700 1 -- 172.21.15.32:0/1675354357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2b08101060 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b0d7a5700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2af80092e0 tx=0x7f2af800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af801d070 con 0x7f2b08101060 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08196940 con 0x7f2b08101060 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2af8005dc0 con 0x7f2b08101060 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af8017400 con 0x7f2b08101060 2024-06-03T01:28:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.164+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08196e30 con 0x7f2b08101060 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.165+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b081090a0 con 0x7f2b08101060 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.166+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2af8017670 con 0x7f2b08101060 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.167+0000 7f2afeffd700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 0x7f2af40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.167+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f2af808f760 con 0x7f2b08101060 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.167+0000 7f2b0dfa6700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 0x7f2af40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.167+0000 7f2b0dfa6700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 0x7f2af40706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005950 tx=0x7f2b040058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.171+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2af805c4e0 con 0x7f2b08101060 2024-06-03T01:28:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:05 smithi032 ceph-mon[111919]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:05 smithi032 ceph-mon[111919]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:05 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3959689513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:05 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3387645852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:28:05.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.404+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b08197110 con 0x7f2af406e200 2024-06-03T01:28:05.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.406+0000 7f2afeffd700 1 -- 172.21.15.32:0/1675354357 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2b08197110 con 0x7f2af406e200 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "8/23 daemons upgraded", 2024-06-03T01:28:05.406 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:28:05.407 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:28:05.407 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 msgr2=0x7f2af40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 0x7f2af40706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005950 tx=0x7f2b040058e0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 msgr2=0x7f2b0819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2af80092e0 tx=0x7f2af800be20 comp rx=0 tx=0).stop 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 shutdown_connections 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2af406e200 0x7f2af40706c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b081006b0 0x7f2b0819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 --2- 172.21.15.32:0/1675354357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b08101060 0x7f2b0819cb80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 >> 172.21.15.32:0/1675354357 conn(0x7f2b08074e30 msgr2=0x7f2b08103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:05.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 shutdown_connections 2024-06-03T01:28:05.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.408+0000 7f2b1020a700 1 -- 172.21.15.32:0/1675354357 wait complete. 2024-06-03T01:28:05.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:05 smithi044 ceph-mon[84214]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:05.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:05 smithi044 ceph-mon[84214]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:05.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:05 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3959689513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:05.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:05 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3387645852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:28:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 -- 172.21.15.32:0/2548694245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998101840 msgr2=0x7ff998105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 --2- 172.21.15.32:0/2548694245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998101840 0x7ff998105890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff980009910 tx=0x7ff980009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 -- 172.21.15.32:0/2548694245 shutdown_connections 2024-06-03T01:28:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 --2- 172.21.15.32:0/2548694245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998101840 0x7ff998105890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 --2- 172.21.15.32:0/2548694245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998100e90 0x7ff998101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 -- 172.21.15.32:0/2548694245 >> 172.21.15.32:0/2548694245 conn(0x7ff9980fc740 msgr2=0x7ff9980feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 -- 172.21.15.32:0/2548694245 shutdown_connections 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.518+0000 7ff99e331700 1 -- 172.21.15.32:0/2548694245 wait complete. 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 Processor -- start 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 -- start start 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 0x7ff9980735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff998073b10 con 0x7ff998100e90 2024-06-03T01:28:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff99e331700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff998073c50 con 0x7ff998101840 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff997fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff997fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40670/0 (socket says 172.21.15.32:40670) 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.519+0000 7ff997fff700 1 -- 172.21.15.32:0/1366272103 learned_addr learned my addr 172.21.15.32:0/1366272103 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff997fff700 1 -- 172.21.15.32:0/1366272103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 msgr2=0x7ff9980735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff9977fe700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 0x7ff9980735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff997fff700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 0x7ff9980735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff997fff700 1 -- 172.21.15.32:0/1366272103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800095f0 con 0x7ff998100e90 2024-06-03T01:28:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff9977fe700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 0x7ff9980735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff997fff700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c930 tx=0x7ff98800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff988007ad0 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff988007c30 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9981a2a70 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff988018610 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.520+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9981a2dd0 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.522+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998109290 con 0x7ff998100e90 2024-06-03T01:28:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.522+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff98800f450 con 0x7ff998100e90 2024-06-03T01:28:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.522+0000 7ff9957fa700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 0x7ff9840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.523+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6683+0+0 (secure 0 0 0) 0x7ff98808e440 con 0x7ff998100e90 2024-06-03T01:28:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.523+0000 7ff9977fe700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 0x7ff9840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.523+0000 7ff9977fe700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 0x7ff9840706c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff98000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.527+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff9880576c0 con 0x7ff998100e90 2024-06-03T01:28:05.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.827+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff998073e10 con 0x7ff998100e90 2024-06-03T01:28:05.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.827+0000 7ff9957fa700 1 -- 172.21.15.32:0/1366272103 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff98805ad80 con 0x7ff998100e90 2024-06-03T01:28:05.827 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:28:05.827 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:28:05.827 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:28:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 msgr2=0x7ff9840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 0x7ff9840706c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff98000b540 comp rx=0 tx=0).stop 2024-06-03T01:28:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 msgr2=0x7ff998073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c930 tx=0x7ff98800ccf0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 shutdown_connections 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff98406e200 0x7ff9840706c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff998100e90 0x7ff998073090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 --2- 172.21.15.32:0/1366272103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff998101840 0x7ff9980735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 >> 172.21.15.32:0/1366272103 conn(0x7ff9980fc740 msgr2=0x7ff9980fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 shutdown_connections 2024-06-03T01:28:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:05.829+0000 7ff99e331700 1 -- 172.21.15.32:0/1366272103 wait complete. 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: pgmap v99: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 63 objects/s recovering 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: Upgrade: osd.2 is safe to restart 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:06.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:06 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1366272103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:28:06.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: pgmap v99: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 63 objects/s recovering 2024-06-03T01:28:06.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:28:06.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-03T01:28:06.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: Upgrade: osd.2 is safe to restart 2024-06-03T01:28:06.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:06.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:06 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1366272103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:28:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: pgmap v100: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 59 objects/s recovering 2024-06-03T01:28:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: Upgrade: Updating osd.2 2024-06-03T01:28:08.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:28:08.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:08.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:08 smithi032 ceph-mon[111919]: Deploying daemon osd.2 on smithi032 2024-06-03T01:28:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: pgmap v100: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 59 objects/s recovering 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: Upgrade: Updating osd.2 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:08 smithi044 ceph-mon[84214]: Deploying daemon osd.2 on smithi032 2024-06-03T01:28:10.317 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:10 smithi032 ceph-mon[111919]: pgmap v101: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:10 smithi044 ceph-mon[84214]: pgmap v101: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:10.584 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:10 smithi032 systemd[1]: Stopping Ceph osd.2 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:28:10.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:10 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[63345]: 2024-06-03T01:28:10.583+0000 7fa85e2ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:28:10.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:10 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[63345]: 2024-06-03T01:28:10.583+0000 7fa85e2ad700 -1 osd.2 71 *** Got signal Terminated *** 2024-06-03T01:28:10.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:10 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[63345]: 2024-06-03T01:28:10.583+0000 7fa85e2ad700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:28:11.331 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:11 smithi032 ceph-mon[111919]: osd.2 marked itself down and dead 2024-06-03T01:28:11.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:11 smithi044 ceph-mon[84214]: osd.2 marked itself down and dead 2024-06-03T01:28:11.585 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:11 smithi032 podman[134294]: 2024-06-03 01:28:11.553421501 +0000 UTC m=+1.082608055 container died 804fd2e4f1951fe4fcc6d6acc6291de683bd6f25ce7fce34b49ec35316d623b1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T01:28:12.068 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:12 smithi032 ceph-mon[111919]: pgmap v102: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:12.120 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:12 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:28:12.120 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:12 smithi032 ceph-mon[111919]: osdmap e72: 6 total, 5 up, 6 in 2024-06-03T01:28:12.120 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:11 smithi032 podman[134294]: 2024-06-03 01:28:11.866844834 +0000 UTC m=+1.396031387 container cleanup 804fd2e4f1951fe4fcc6d6acc6291de683bd6f25ce7fce34b49ec35316d623b1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T01:28:12.121 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:11 smithi032 bash[134294]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2 2024-06-03T01:28:12.335 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134312]: 2024-06-03 01:28:12.068381113 +0000 UTC m=+0.506918133 container remove 804fd2e4f1951fe4fcc6d6acc6291de683bd6f25ce7fce34b49ec35316d623b1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-06-03T01:28:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:12 smithi044 ceph-mon[84214]: pgmap v102: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:12 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:28:12.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:12 smithi044 ceph-mon[84214]: osdmap e72: 6 total, 5 up, 6 in 2024-06-03T01:28:12.688 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.377764115 +0000 UTC m=+0.093713634 container create c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-03T01:28:12.688 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.301860762 +0000 UTC m=+0.017810291 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:12.689 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.605392012 +0000 UTC m=+0.321341545 container init c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T01:28:12.689 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.609059471 +0000 UTC m=+0.325008994 container start c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-03T01:28:12.689 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.688723723 +0000 UTC m=+0.404673240 container attach c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-03T01:28:13.024 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:12 smithi032 podman[134398]: 2024-06-03 01:28:12.928531996 +0000 UTC m=+0.644481520 container died c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-03T01:28:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:13 smithi032 ceph-mon[111919]: osdmap e73: 6 total, 5 up, 6 in 2024-06-03T01:28:13.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:13 smithi044 ceph-mon[84214]: osdmap e73: 6 total, 5 up, 6 in 2024-06-03T01:28:13.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 podman[134425]: 2024-06-03 01:28:13.53442851 +0000 UTC m=+0.600011640 container remove c31482594948c3c6ab580d2e634e0825f92faa17b8573149cd753e81b814dc06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T01:28:13.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.2.service: Deactivated successfully. 2024-06-03T01:28:13.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 systemd[1]: Stopped Ceph osd.2 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:28:13.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.2.service: Consumed 54.813s CPU time. 2024-06-03T01:28:13.835 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 systemd[1]: Starting Ceph osd.2 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:28:14.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-mon[111919]: pgmap v105: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 9 peering, 3 stale+active+clean, 41 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-mon[111919]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 podman[134525]: 2024-06-03 01:28:13.985571196 +0000 UTC m=+0.101880973 container create 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:13 smithi032 podman[134525]: 2024-06-03 01:28:13.901778783 +0000 UTC m=+0.018088537 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:14.254 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 podman[134525]: 2024-06-03 01:28:14.155041479 +0000 UTC m=+0.271351248 container init 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 podman[134525]: 2024-06-03 01:28:14.158627465 +0000 UTC m=+0.274937206 container start 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:28:14.254 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 podman[134525]: 2024-06-03 01:28:14.254652511 +0000 UTC m=+0.370962247 container attach 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-03T01:28:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:14 smithi044 ceph-mon[84214]: pgmap v105: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 9 peering, 3 stale+active+clean, 41 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-06-03T01:28:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:14 smithi044 ceph-mon[84214]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-06-03T01:28:14.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:14 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 17/366 objects degraded (4.645%), 5 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-03T01:28:14.908 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 bash[134525]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate[134539]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-03T01:28:14.909 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:14 smithi032 podman[134525]: 2024-06-03 01:28:14.64896697 +0000 UTC m=+0.765276717 container died 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-03T01:28:15.568 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 podman[134673]: 2024-06-03 01:28:15.234351266 +0000 UTC m=+0.574563798 container remove 7f80d0ac720a3dcae7ff09e9d16172002d50cb71b565e6e12bf199fa39fe0285 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-activate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-03T01:28:15.568 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 podman[134733]: 2024-06-03 01:28:15.443603393 +0000 UTC m=+0.098754068 container create 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-03T01:28:15.568 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 podman[134733]: 2024-06-03 01:28:15.363766273 +0000 UTC m=+0.018916914 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:16.028 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 podman[134733]: 2024-06-03 01:28:15.679393848 +0000 UTC m=+0.334544473 container init 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-03T01:28:16.029 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 podman[134733]: 2024-06-03 01:28:15.684074765 +0000 UTC m=+0.339225399 container start 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-06-03T01:28:16.029 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 ceph-osd[134752]: -- 172.21.15.32:0/1198869235 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56252bca3680 con 0x56252bc6d000 2024-06-03T01:28:16.029 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 bash[134733]: 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 2024-06-03T01:28:16.029 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:15 smithi032 systemd[1]: Started Ceph osd.2 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:28:16.335 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:16 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:28:16.224+0000 7fb5811bc7c0 -1 Falling back to public interface 2024-06-03T01:28:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:16 smithi032 ceph-mon[111919]: pgmap v106: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 9 peering, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32/366 objects degraded (8.743%) 2024-06-03T01:28:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:16.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:16 smithi044 ceph-mon[84214]: pgmap v106: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 9 peering, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32/366 objects degraded (8.743%) 2024-06-03T01:28:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:16.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:17.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:17 smithi032 ceph-mon[111919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-06-03T01:28:17.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:17 smithi044 ceph-mon[84214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-06-03T01:28:18.181 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:18 smithi032 ceph-mon[111919]: pgmap v107: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:18 smithi044 ceph-mon[84214]: pgmap v107: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:20.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:20 smithi032 ceph-mon[111919]: pgmap v108: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:20.335 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:20 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:28:20.167+0000 7fb5811bc7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-06-03T01:28:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:20 smithi044 ceph-mon[84214]: pgmap v108: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:21.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:20 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:28:20.721+0000 7fb5811bc7c0 -1 osd.2 71 log_to_monitors true 2024-06-03T01:28:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:21 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:21 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:21.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:21 smithi044 ceph-mon[84214]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T01:28:22.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:28:21 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:28:21.749+0000 7fb572613700 -1 osd.2 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:28:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:21 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:21 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:21 smithi032 ceph-mon[111919]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: pgmap v109: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: osdmap e74: 6 total, 5 up, 6 in 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 49/366 objects degraded (13.388%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: pgmap v109: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/366 objects degraded (13.388%) 2024-06-03T01:28:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: osdmap e74: 6 total, 5 up, 6 in 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 49/366 objects degraded (13.388%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:24.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:23 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:28:24.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:23 smithi032 ceph-mon[111919]: osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958] boot 2024-06-03T01:28:24.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:23 smithi032 ceph-mon[111919]: osdmap e75: 6 total, 6 up, 6 in 2024-06-03T01:28:24.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:23 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:28:24.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:23 smithi032 ceph-mon[111919]: osdmap e76: 6 total, 6 up, 6 in 2024-06-03T01:28:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:23 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:28:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:23 smithi044 ceph-mon[84214]: osd.2 [v2:172.21.15.32:6818/1912654958,v1:172.21.15.32:6819/1912654958] boot 2024-06-03T01:28:24.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:23 smithi044 ceph-mon[84214]: osdmap e75: 6 total, 6 up, 6 in 2024-06-03T01:28:24.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:23 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:28:24.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:23 smithi044 ceph-mon[84214]: osdmap e76: 6 total, 6 up, 6 in 2024-06-03T01:28:24.918 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:24 smithi032 ceph-mon[111919]: pgmap v112: 65 pgs: 10 peering, 11 active+undersized, 8 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32/366 objects degraded (8.743%) 2024-06-03T01:28:25.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:24 smithi044 ceph-mon[84214]: pgmap v112: 65 pgs: 10 peering, 11 active+undersized, 8 active+undersized+degraded, 36 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32/366 objects degraded (8.743%) 2024-06-03T01:28:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: pgmap v114: 65 pgs: 10 peering, 8 active+undersized, 6 active+undersized+degraded, 41 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22/366 objects degraded (6.011%) 2024-06-03T01:28:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:26.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:26 smithi032 ceph-mon[111919]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-06-03T01:28:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: pgmap v114: 65 pgs: 10 peering, 8 active+undersized, 6 active+undersized+degraded, 41 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22/366 objects degraded (6.011%) 2024-06-03T01:28:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:26 smithi044 ceph-mon[84214]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-06-03T01:28:27.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:27 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 22/366 objects degraded (6.011%), 6 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:27.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:27 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 22/366 objects degraded (6.011%), 6 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:28 smithi032 ceph-mon[111919]: pgmap v115: 65 pgs: 10 peering, 55 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:28 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/366 objects degraded (6.011%), 6 pgs degraded) 2024-06-03T01:28:28.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:28 smithi044 ceph-mon[84214]: pgmap v115: 65 pgs: 10 peering, 55 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:28.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:28 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/366 objects degraded (6.011%), 6 pgs degraded) 2024-06-03T01:28:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:30 smithi032 ceph-mon[111919]: pgmap v116: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:30.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:30 smithi044 ceph-mon[84214]: pgmap v116: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:32 smithi032 ceph-mon[111919]: pgmap v117: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:32.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:32 smithi044 ceph-mon[84214]: pgmap v117: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:34.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:34 smithi032 ceph-mon[111919]: pgmap v118: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:34.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:34 smithi044 ceph-mon[84214]: pgmap v118: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 -- 172.21.15.32:0/2219438910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8100ea0 msgr2=0x7f0dd8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2219438910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8101280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009910 tx=0x7f0dd4009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 -- 172.21.15.32:0/2219438910 shutdown_connections 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2219438910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8101850 0x7f0dd81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2219438910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8101280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 -- 172.21.15.32:0/2219438910 >> 172.21.15.32:0/2219438910 conn(0x7f0dd80fc750 msgr2=0x7f0dd80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 -- 172.21.15.32:0/2219438910 shutdown_connections 2024-06-03T01:28:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.976+0000 7f0de04c4700 1 -- 172.21.15.32:0/2219438910 wait complete. 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 Processor -- start 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 -- start start 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8073b10 con 0x7f0dd8101850 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0de04c4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8073c50 con 0x7f0dd8100ea0 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0ddda5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.977+0000 7f0ddda5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51628/0 (socket says 172.21.15.32:51628) 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0ddda5f700 1 -- 172.21.15.32:0/2419548635 learned_addr learned my addr 172.21.15.32:0/2419548635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0dde260700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0ddda5f700 1 -- 172.21.15.32:0/2419548635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8100ea0 msgr2=0x7f0dd8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0ddda5f700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0ddda5f700 1 -- 172.21.15.32:0/2419548635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40095f0 con 0x7f0dd8101850 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0ddda5f700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c960 tx=0x7f0dcc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dcc007a70 con 0x7f0dd8101850 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dcc007bd0 con 0x7f0dd8101850 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd81a2af0 con 0x7f0dd8101850 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dcc018630 con 0x7f0dd8101850 2024-06-03T01:28:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.978+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd81a2e50 con 0x7f0dd8101850 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.980+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd8109240 con 0x7f0dd8101850 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.980+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0dcc00f450 con 0x7f0dd8101850 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.981+0000 7f0dcb7fe700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 0x7f0dc40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.981+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f0dcc08e0c0 con 0x7f0dd8101850 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.981+0000 7f0dde260700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 0x7f0dc40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.981+0000 7f0dde260700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 0x7f0dc40706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4006010 tx=0x7f0dd401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:35.986+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0dcc057470 con 0x7f0dd8101850 2024-06-03T01:28:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.213+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd80611d0 con 0x7f0dc406e200 2024-06-03T01:28:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.214+0000 7f0dcb7fe700 1 -- 172.21.15.32:0/2419548635 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0dd80611d0 con 0x7f0dc406e200 2024-06-03T01:28:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.216+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 msgr2=0x7f0dc40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.216+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 0x7f0dc40706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4006010 tx=0x7f0dd401a040 comp rx=0 tx=0).stop 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.216+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 msgr2=0x7f0dd80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.216+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c960 tx=0x7f0dcc00cc70 comp rx=0 tx=0).stop 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 shutdown_connections 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f0dc406e200 0x7f0dc40706c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dd8100ea0 0x7f0dd8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 --2- 172.21.15.32:0/2419548635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dd8101850 0x7f0dd80735d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 >> 172.21.15.32:0/2419548635 conn(0x7f0dd80fc750 msgr2=0x7f0dd80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 shutdown_connections 2024-06-03T01:28:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.217+0000 7f0de04c4700 1 -- 172.21.15.32:0/2419548635 wait complete. 2024-06-03T01:28:36.228 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:28:36.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.326+0000 7f5323194700 1 -- 172.21.15.32:0/3204944901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c101850 msgr2=0x7f531c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.326+0000 7f5323194700 1 --2- 172.21.15.32:0/3204944901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c101850 0x7f531c1058a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5308009910 tx=0x7f5308009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 -- 172.21.15.32:0/3204944901 shutdown_connections 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 --2- 172.21.15.32:0/3204944901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c101850 0x7f531c1058a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 --2- 172.21.15.32:0/3204944901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c100ea0 0x7f531c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 -- 172.21.15.32:0/3204944901 >> 172.21.15.32:0/3204944901 conn(0x7f531c0fc750 msgr2=0x7f531c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 -- 172.21.15.32:0/3204944901 shutdown_connections 2024-06-03T01:28:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 -- 172.21.15.32:0/3204944901 wait complete. 2024-06-03T01:28:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 Processor -- start 2024-06-03T01:28:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.327+0000 7f5323194700 1 -- start start 2024-06-03T01:28:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5323194700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5323194700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 0x7f531c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5323194700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19d360 con 0x7f531c100ea0 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5323194700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c196810 con 0x7f531c101850 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51642/0 (socket says 172.21.15.32:51642) 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 -- 172.21.15.32:0/503837296 learned_addr learned my addr 172.21.15.32:0/503837296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f531bfff700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 0x7f531c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 -- 172.21.15.32:0/503837296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 msgr2=0x7f531c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 0x7f531c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 -- 172.21.15.32:0/503837296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080095f0 con 0x7f531c100ea0 2024-06-03T01:28:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f531bfff700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 0x7f531c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5320f30700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f531000b4b0 tx=0x7f531000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.328+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5310011800 con 0x7f531c100ea0 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.329+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5310011e20 con 0x7f531c100ea0 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.329+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c196af0 con 0x7f531c100ea0 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.329+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531000f2e0 con 0x7f531c100ea0 2024-06-03T01:28:36.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.329+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c197040 con 0x7f531c100ea0 2024-06-03T01:28:36.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.330+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c1091d0 con 0x7f531c100ea0 2024-06-03T01:28:36.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.330+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5310010380 con 0x7f531c100ea0 2024-06-03T01:28:36.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.331+0000 7f5319ffb700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 0x7f530c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.331+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f531008dde0 con 0x7f531c100ea0 2024-06-03T01:28:36.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.331+0000 7f531bfff700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 0x7f530c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.331+0000 7f531bfff700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 0x7f530c0706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f531c0fd160 tx=0x7f530801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:36.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.336+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f531005ab90 con 0x7f531c100ea0 2024-06-03T01:28:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.561+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f531c1972c0 con 0x7f530c06e200 2024-06-03T01:28:36.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:36 smithi032 ceph-mon[111919]: pgmap v119: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.563+0000 7f5319ffb700 1 -- 172.21.15.32:0/503837296 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f531c1972c0 con 0x7f530c06e200 2024-06-03T01:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 msgr2=0x7f530c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 0x7f530c0706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f531c0fd160 tx=0x7f530801a040 comp rx=0 tx=0).stop 2024-06-03T01:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 msgr2=0x7f531c19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f531000b4b0 tx=0x7f531000b870 comp rx=0 tx=0).stop 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 shutdown_connections 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f530c06e200 0x7f530c0706c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f531c100ea0 0x7f531c19c790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 --2- 172.21.15.32:0/503837296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f531c101850 0x7f531c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 >> 172.21.15.32:0/503837296 conn(0x7f531c0fc750 msgr2=0x7f531c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 shutdown_connections 2024-06-03T01:28:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.565+0000 7f5323194700 1 -- 172.21.15.32:0/503837296 wait complete. 2024-06-03T01:28:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- 172.21.15.32:0/3570125929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc101027a0 msgr2=0x7fdc1010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 --2- 172.21.15.32:0/3570125929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc101027a0 0x7fdc1010ac90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009910 tx=0x7fdc00009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- 172.21.15.32:0/3570125929 shutdown_connections 2024-06-03T01:28:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 --2- 172.21.15.32:0/3570125929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc101027a0 0x7fdc1010ac90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 --2- 172.21.15.32:0/3570125929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc10101e80 0x7fdc10102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- 172.21.15.32:0/3570125929 >> 172.21.15.32:0/3570125929 conn(0x7fdc100fb650 msgr2=0x7fdc100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- 172.21.15.32:0/3570125929 shutdown_connections 2024-06-03T01:28:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- 172.21.15.32:0/3570125929 wait complete. 2024-06-03T01:28:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 Processor -- start 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.681+0000 7fdc17330700 1 -- start start 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc17330700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc17330700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc101027a0 0x7fdc1019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc17330700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1019d220 con 0x7fdc10101e80 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc17330700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101966d0 con 0x7fdc101027a0 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51658/0 (socket says 172.21.15.32:51658) 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 -- 172.21.15.32:0/1223314204 learned_addr learned my addr 172.21.15.32:0/1223314204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc148cb700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc101027a0 0x7fdc1019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 -- 172.21.15.32:0/1223314204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc101027a0 msgr2=0x7fdc1019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc101027a0 0x7fdc1019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 -- 172.21.15.32:0/1223314204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000095f0 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc150cc700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d8d0 tx=0x7fdc0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.682+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0c009940 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.683+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc0c010460 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.683+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc101969b0 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.683+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0c00f5f0 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.683+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc10196f00 con 0x7fdc10101e80 2024-06-03T01:28:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.684+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc10108320 con 0x7fdc10101e80 2024-06-03T01:28:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.684+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc0c0105d0 con 0x7fdc10101e80 2024-06-03T01:28:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.685+0000 7fdc067fc700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.685+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fdc0c08e200 con 0x7fdc10101e80 2024-06-03T01:28:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.685+0000 7fdc148cb700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.685+0000 7fdc148cb700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 0x7fdbfc0706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc000059d0 tx=0x7fdc00005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.690+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdc0c05afb0 con 0x7fdc10101e80 2024-06-03T01:28:36.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:36 smithi044 ceph-mon[84214]: pgmap v119: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.923+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc10197cc0 con 0x7fdbfc06e200 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.934+0000 7fdc067fc700 1 -- 172.21.15.32:0/1223314204 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdc10197cc0 con 0x7fdbfc06e200 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (4m) 16s ago 15m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (16m) 16s ago 16m 8656k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (14m) 110s ago 14m 8002k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (119s) 16s ago 16m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:28:36.934 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (114s) 110s ago 14m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (4m) 16s ago 15m 81.4M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (10m) 16s ago 10m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (10m) 16s ago 10m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (10m) 110s ago 10m 16.0M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (10m) 110s ago 10m 17.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (6m) 16s ago 17m 505M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (5m) 110s ago 14m 450M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (3m) 16s ago 17m 49.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (2m) 110s ago 14m 37.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 16s ago 15m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:28:36.935 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (5m) 110s ago 14m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (93s) 16s ago 13m 138M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (57s) 16s ago 13m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (21s) 16s ago 13m 12.1M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (12m) 110s ago 12m 269M 2241M 18.2.1 5be31c24972a 659834fab216 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (12m) 110s ago 12m 249M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (11m) 110s ago 11m 231M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:28:36.936 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 16s ago 15m 78.3M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 msgr2=0x7fdbfc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 0x7fdbfc0706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc000059d0 tx=0x7fdc00005960 comp rx=0 tx=0).stop 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 msgr2=0x7fdc1019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d8d0 tx=0x7fdc0c00dc90 comp rx=0 tx=0).stop 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 shutdown_connections 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc10101e80 0x7fdc1019c650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 --2- 172.21.15.32:0/1223314204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc101027a0 0x7fdc1019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 >> 172.21.15.32:0/1223314204 conn(0x7fdc100fb650 msgr2=0x7fdc10105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 shutdown_connections 2024-06-03T01:28:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:36.936+0000 7fdc17330700 1 -- 172.21.15.32:0/1223314204 wait complete. 2024-06-03T01:28:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 -- 172.21.15.32:0/1459553497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 msgr2=0x7f0890100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 --2- 172.21.15.32:0/1459553497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890100080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0878009910 tx=0x7f0878009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 -- 172.21.15.32:0/1459553497 shutdown_connections 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 --2- 172.21.15.32:0/1459553497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890100080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 --2- 172.21.15.32:0/1459553497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 0x7f0890105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 -- 172.21.15.32:0/1459553497 >> 172.21.15.32:0/1459553497 conn(0x7f0890075670 msgr2=0x7f0890075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 -- 172.21.15.32:0/1459553497 shutdown_connections 2024-06-03T01:28:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.047+0000 7f08949d5700 1 -- 172.21.15.32:0/1459553497 wait complete. 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.048+0000 7f08949d5700 1 Processor -- start 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.048+0000 7f08949d5700 1 -- start start 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.048+0000 7f08949d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.048+0000 7f08949d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 0x7f0890101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08949d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901022c0 con 0x7f08900ffc20 2024-06-03T01:28:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08949d5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890102400 con 0x7f0890105bd0 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47378/0 (socket says 172.21.15.32:47378) 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 -- 172.21.15.32:0/1205005836 learned_addr learned my addr 172.21.15.32:0/1205005836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088dd9b700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 0x7f0890101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 -- 172.21.15.32:0/1205005836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 msgr2=0x7f0890101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 0x7f0890101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 -- 172.21.15.32:0/1205005836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780095f0 con 0x7f08900ffc20 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f088e59c700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f088000e920 tx=0x7f088000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088000c9d0 con 0x7f08900ffc20 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0880010430 con 0x7f08900ffc20 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901a2a30 con 0x7f08900ffc20 2024-06-03T01:28:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0880018630 con 0x7f08900ffc20 2024-06-03T01:28:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.049+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08901a2e40 con 0x7f08900ffc20 2024-06-03T01:28:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.050+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089004f2e0 con 0x7f08900ffc20 2024-06-03T01:28:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.051+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f08800105a0 con 0x7f08900ffc20 2024-06-03T01:28:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.052+0000 7f08877fe700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 0x7f087c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.052+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f0880014070 con 0x7f08900ffc20 2024-06-03T01:28:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.052+0000 7f088dd9b700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 0x7f087c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.052+0000 7f088dd9b700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 0x7f087c0706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0890103030 tx=0x7f0878005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.056+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0880053850 con 0x7f08900ffc20 2024-06-03T01:28:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.357+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08901025c0 con 0x7f08900ffc20 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.358+0000 7f08877fe700 1 -- 172.21.15.32:0/1205005836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+787 (secure 0 0 0) 0x7f0880057750 con 0x7f08900ffc20 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:37.358 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 3 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 7 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:28:37.359 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 msgr2=0x7f087c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 0x7f087c0706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0890103030 tx=0x7f0878005940 comp rx=0 tx=0).stop 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 msgr2=0x7f0890101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f088000e920 tx=0x7f088000ec30 comp rx=0 tx=0).stop 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 shutdown_connections 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f087c06e200 0x7f087c0706c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08900ffc20 0x7f0890101840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 --2- 172.21.15.32:0/1205005836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0890105bd0 0x7f0890101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 >> 172.21.15.32:0/1205005836 conn(0x7f0890075670 msgr2=0x7f0890068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 shutdown_connections 2024-06-03T01:28:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.360+0000 7f08949d5700 1 -- 172.21.15.32:0/1205005836 wait complete. 2024-06-03T01:28:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 -- 172.21.15.32:0/3575261438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 msgr2=0x7fa3dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 --2- 172.21.15.32:0/3575261438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc1058a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009910 tx=0x7fa3cc009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 -- 172.21.15.32:0/3575261438 shutdown_connections 2024-06-03T01:28:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 --2- 172.21.15.32:0/3575261438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc1058a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 --2- 172.21.15.32:0/3575261438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 -- 172.21.15.32:0/3575261438 >> 172.21.15.32:0/3575261438 conn(0x7fa3dc0fc750 msgr2=0x7fa3dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 -- 172.21.15.32:0/3575261438 shutdown_connections 2024-06-03T01:28:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 -- 172.21.15.32:0/3575261438 wait complete. 2024-06-03T01:28:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.469+0000 7fa3e45be700 1 Processor -- start 2024-06-03T01:28:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e45be700 1 -- start start 2024-06-03T01:28:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e45be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e45be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e45be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc19d2d0 con 0x7fa3dc101850 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e45be700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc196780 con 0x7fa3dc100ea0 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47402/0 (socket says 172.21.15.32:47402) 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 -- 172.21.15.32:0/1126681261 learned_addr learned my addr 172.21.15.32:0/1126681261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e235a700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 -- 172.21.15.32:0/1126681261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 msgr2=0x7fa3dc19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 -- 172.21.15.32:0/1126681261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0095f0 con 0x7fa3dc101850 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e235a700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.470+0000 7fa3e1b59700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0092e0 tx=0x7fa3cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.471+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc01d070 con 0x7fa3dc101850 2024-06-03T01:28:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.471+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3cc005dc0 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.471+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc196a00 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.471+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc017400 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.471+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc196ef0 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.472+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc1092a0 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.472+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3cc00f460 con 0x7fa3dc101850 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.473+0000 7fa3d37fe700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 0x7fa3c80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.473+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fa3cc08f630 con 0x7fa3dc101850 2024-06-03T01:28:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.473+0000 7fa3e235a700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 0x7fa3c80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.473+0000 7fa3e235a700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 0x7fa3c80706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800bc10 tx=0x7fa3d800b5f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.477+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa3cc05c460 con 0x7fa3dc101850 2024-06-03T01:28:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:37 smithi032 ceph-mon[111919]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:37 smithi032 ceph-mon[111919]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:37.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:37 smithi044 ceph-mon[84214]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:37.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:37 smithi044 ceph-mon[84214]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:37.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.730+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa3dc04f2e0 con 0x7fa3dc101850 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.731+0000 7fa3d37fe700 1 -- 172.21.15.32:0/1126681261 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7fa3cc05bff0 con 0x7fa3dc101850 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:37.731 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:28:37.732 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:28:37.733 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:37.734 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:28:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 msgr2=0x7fa3c80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 0x7fa3c80706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800bc10 tx=0x7fa3d800b5f0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 msgr2=0x7fa3dc19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0092e0 tx=0x7fa3cc00be20 comp rx=0 tx=0).stop 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 shutdown_connections 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa3c806e200 0x7fa3c80706c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100ea0 0x7fa3dc19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 --2- 172.21.15.32:0/1126681261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc101850 0x7fa3dc19cc40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 >> 172.21.15.32:0/1126681261 conn(0x7fa3dc0fc750 msgr2=0x7fa3dc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 shutdown_connections 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.734+0000 7fa3e45be700 1 -- 172.21.15.32:0/1126681261 wait complete. 2024-06-03T01:28:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.844+0000 7f7562d0b700 1 -- 172.21.15.32:0/3321398192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 msgr2=0x7f755c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.844+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3321398192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c1047a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7548009910 tx=0x7f7548009c20 comp rx=0 tx=0).stop 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 -- 172.21.15.32:0/3321398192 shutdown_connections 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3321398192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c1047a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3321398192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 0x7f755c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 -- 172.21.15.32:0/3321398192 >> 172.21.15.32:0/3321398192 conn(0x7f755c0fb650 msgr2=0x7f755c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 -- 172.21.15.32:0/3321398192 shutdown_connections 2024-06-03T01:28:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 -- 172.21.15.32:0/3321398192 wait complete. 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 Processor -- start 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.845+0000 7f7562d0b700 1 -- start start 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7562d0b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 0x7f755c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7562d0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7562d0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c198320 con 0x7f755c100750 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7562d0b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c198460 con 0x7f755c0ffda0 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47424/0 (socket says 172.21.15.32:47424) 2024-06-03T01:28:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 -- 172.21.15.32:0/3682770349 learned_addr learned my addr 172.21.15.32:0/3682770349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7560aa7700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 0x7f755c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 -- 172.21.15.32:0/3682770349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 msgr2=0x7f755c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 0x7f755c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 -- 172.21.15.32:0/3682770349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480095f0 con 0x7f755c100750 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f755bfff700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7548009fd0 tx=0x7f754800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754801d070 con 0x7f755c100750 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7548022470 con 0x7f755c100750 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1a0830 con 0x7f755c100750 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.846+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7548017610 con 0x7f755c100750 2024-06-03T01:28:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.847+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1a0be0 con 0x7f755c100750 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.848+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c1081a0 con 0x7f755c100750 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.848+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7548005080 con 0x7f755c100750 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.849+0000 7f7559ffb700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 0x7f754c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.849+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f754808f940 con 0x7f755c100750 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.849+0000 7f7560aa7700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 0x7f754c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:37.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.849+0000 7f7560aa7700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 0x7f754c0706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7550009bf0 tx=0x7f7550009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:37.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:37.853+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7548058b20 con 0x7f755c100750 2024-06-03T01:28:38.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.084+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f755c0611d0 con 0x7f754c06e200 2024-06-03T01:28:38.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.085+0000 7f7559ffb700 1 -- 172.21.15.32:0/3682770349 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f755c0611d0 con 0x7f754c06e200 2024-06-03T01:28:38.085 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:28:38.085 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:28:38.085 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:28:38.085 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "9/23 daemons upgraded", 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:28:38.086 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:28:38.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 msgr2=0x7f754c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 0x7f754c0706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7550009bf0 tx=0x7f7550009260 comp rx=0 tx=0).stop 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 msgr2=0x7f755c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7548009fd0 tx=0x7f754800f690 comp rx=0 tx=0).stop 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 shutdown_connections 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f754c06e200 0x7f754c0706c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f755c0ffda0 0x7f755c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 --2- 172.21.15.32:0/3682770349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f755c100750 0x7f755c197d00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.087+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 >> 172.21.15.32:0/3682770349 conn(0x7f755c0fb650 msgr2=0x7f755c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.088+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 shutdown_connections 2024-06-03T01:28:38.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.088+0000 7f7562d0b700 1 -- 172.21.15.32:0/3682770349 wait complete. 2024-06-03T01:28:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/1518106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 msgr2=0x7fd9f4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/1518106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4101280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400b0a0 tx=0x7fd9e400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/1518106281 shutdown_connections 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/1518106281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 0x7fd9f41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/1518106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4101280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/1518106281 >> 172.21.15.32:0/1518106281 conn(0x7fd9f40fc750 msgr2=0x7fd9f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/1518106281 shutdown_connections 2024-06-03T01:28:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.196+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/1518106281 wait complete. 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 Processor -- start 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 -- start start 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 0x7fd9f40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4073b10 con 0x7fd9f4100ea0 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.197+0000 7fd9f8bdb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4073c50 con 0x7fd9f4101850 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47434/0 (socket says 172.21.15.32:47434) 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 -- 172.21.15.32:0/2153569919 learned_addr learned my addr 172.21.15.32:0/2153569919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:28:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f1d9b700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 0x7fd9f40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 -- 172.21.15.32:0/2153569919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 msgr2=0x7fd9f40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 0x7fd9f40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 -- 172.21.15.32:0/2153569919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4009d00 con 0x7fd9f4100ea0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f259c700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f80 tx=0x7fd9e400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9e400e040 con 0x7fd9f4100ea0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f4073ed0 con 0x7fd9f4100ea0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9e4003de0 con 0x7fd9f4100ea0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9e401d790 con 0x7fd9f4100ea0 2024-06-03T01:28:38.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.198+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f41a2e20 con 0x7fd9f4100ea0 2024-06-03T01:28:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.199+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f4109240 con 0x7fd9f4100ea0 2024-06-03T01:28:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.200+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd9e4004040 con 0x7fd9f4100ea0 2024-06-03T01:28:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.200+0000 7fd9eb7fe700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 0x7fd9e00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:28:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.200+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fd9e408fac0 con 0x7fd9f4100ea0 2024-06-03T01:28:38.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.200+0000 7fd9f1d9b700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 0x7fd9e00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:28:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.201+0000 7fd9f1d9b700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 0x7fd9e00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005950 tx=0x7fd9dc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:28:38.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.205+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd9e4058ca0 con 0x7fd9f4100ea0 2024-06-03T01:28:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.499+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd9f40745c0 con 0x7fd9f4100ea0 2024-06-03T01:28:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.499+0000 7fd9eb7fe700 1 -- 172.21.15.32:0/2153569919 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd9e405c360 con 0x7fd9f4100ea0 2024-06-03T01:28:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:28:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:28:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:28:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 msgr2=0x7fd9e00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 0x7fd9e00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005950 tx=0x7fd9dc0058e0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 msgr2=0x7fd9f4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:28:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f80 tx=0x7fd9e400cdd0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 shutdown_connections 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd9e006e200 0x7fd9e00706c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9f4100ea0 0x7fd9f4073090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 --2- 172.21.15.32:0/2153569919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f4101850 0x7fd9f40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 >> 172.21.15.32:0/2153569919 conn(0x7fd9f40fc750 msgr2=0x7fd9f40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 shutdown_connections 2024-06-03T01:28:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:28:38.501+0000 7fd9f8bdb700 1 -- 172.21.15.32:0/2153569919 wait complete. 2024-06-03T01:28:38.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:38 smithi032 ceph-mon[111919]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:38.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:38 smithi032 ceph-mon[111919]: pgmap v120: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:38.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:38 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1205005836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:38.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:38 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1126681261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:28:38.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:38 smithi044 ceph-mon[84214]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:38.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:38 smithi044 ceph-mon[84214]: pgmap v120: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:38.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:38 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1205005836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:28:38.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:38 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1126681261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:28:39.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:39 smithi032 ceph-mon[111919]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:39.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:39 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2153569919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:28:39.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:39 smithi044 ceph-mon[84214]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:28:39.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:39 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2153569919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:28:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:40 smithi032 ceph-mon[111919]: pgmap v121: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:40.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:40 smithi044 ceph-mon[84214]: pgmap v121: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:41.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:41 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:41.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:41 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:41.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:41 smithi032 ceph-mon[111919]: Upgrade: osd.3 is safe to restart 2024-06-03T01:28:41.678 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:41 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:41.678 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:41 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-03T01:28:41.679 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:41 smithi044 ceph-mon[84214]: Upgrade: osd.3 is safe to restart 2024-06-03T01:28:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:42 smithi032 ceph-mon[111919]: pgmap v122: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:42.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:42 smithi044 ceph-mon[84214]: pgmap v122: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:42.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:43.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:43 smithi044 ceph-mon[84214]: Upgrade: Updating osd.3 2024-06-03T01:28:43.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:28:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:43.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:43 smithi044 ceph-mon[84214]: Deploying daemon osd.3 on smithi044 2024-06-03T01:28:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:43 smithi032 ceph-mon[111919]: Upgrade: Updating osd.3 2024-06-03T01:28:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T01:28:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:28:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:43 smithi032 ceph-mon[111919]: Deploying daemon osd.3 on smithi044 2024-06-03T01:28:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:44 smithi044 ceph-mon[84214]: pgmap v123: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:44 smithi032 ceph-mon[111919]: pgmap v123: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:45.936 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:45 smithi044 systemd[1]: Stopping Ceph osd.3 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:28:46.221 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:45 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[42964]: 2024-06-03T01:28:45.940+0000 7fdd72aff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:28:46.221 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:45 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[42964]: 2024-06-03T01:28:45.940+0000 7fdd72aff700 -1 osd.3 76 *** Got signal Terminated *** 2024-06-03T01:28:46.222 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:45 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[42964]: 2024-06-03T01:28:45.940+0000 7fdd72aff700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:28:46.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:46 smithi032 ceph-mon[111919]: pgmap v124: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:46.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:46 smithi032 ceph-mon[111919]: osd.3 marked itself down and dead 2024-06-03T01:28:46.692 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:46 smithi044 ceph-mon[84214]: pgmap v124: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:46.692 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:46 smithi044 ceph-mon[84214]: osd.3 marked itself down and dead 2024-06-03T01:28:46.958 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:46 smithi044 podman[93431]: 2024-06-03 01:28:46.69358936 +0000 UTC m=+0.862450689 container died 659834fab216079600d3fc5dae7bf7b120e448bcef2e43a60fc2d09e4984cb03 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1) 2024-06-03T01:28:47.221 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93431]: 2024-06-03 01:28:47.082948473 +0000 UTC m=+1.251809795 container cleanup 659834fab216079600d3fc5dae7bf7b120e448bcef2e43a60fc2d09e4984cb03 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=HEAD) 2024-06-03T01:28:47.222 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 bash[93431]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3 2024-06-03T01:28:47.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:47 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:28:47.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:47 smithi032 ceph-mon[111919]: osdmap e77: 6 total, 5 up, 6 in 2024-06-03T01:28:47.676 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:47 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:28:47.676 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:47 smithi044 ceph-mon[84214]: osdmap e77: 6 total, 5 up, 6 in 2024-06-03T01:28:47.676 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93457]: 2024-06-03 01:28:47.334014301 +0000 UTC m=+0.634658187 container remove 659834fab216079600d3fc5dae7bf7b120e448bcef2e43a60fc2d09e4984cb03 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-03T01:28:47.970 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93546]: 2024-06-03 01:28:47.676860943 +0000 UTC m=+0.111300331 container create c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-03T01:28:47.970 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93546]: 2024-06-03 01:28:47.583592117 +0000 UTC m=+0.018031525 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:47.970 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93546]: 2024-06-03 01:28:47.896110641 +0000 UTC m=+0.330550051 container init c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-06-03T01:28:47.970 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93546]: 2024-06-03 01:28:47.900009788 +0000 UTC m=+0.334449181 container start c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-06-03T01:28:47.970 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:47 smithi044 podman[93546]: 2024-06-03 01:28:47.970833131 +0000 UTC m=+0.405272528 container attach c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T01:28:48.221 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:48 smithi044 podman[93546]: 2024-06-03 01:28:48.216117343 +0000 UTC m=+0.650556740 container died c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T01:28:48.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:48 smithi032 ceph-mon[111919]: pgmap v126: 65 pgs: 3 peering, 13 stale+active+clean, 49 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:48.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:48 smithi032 ceph-mon[111919]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:28:48.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:48 smithi032 ceph-mon[111919]: osdmap e78: 6 total, 5 up, 6 in 2024-06-03T01:28:48.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:48 smithi044 ceph-mon[84214]: pgmap v126: 65 pgs: 3 peering, 13 stale+active+clean, 49 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:28:48.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:48 smithi044 ceph-mon[84214]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:28:48.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:48 smithi044 ceph-mon[84214]: osdmap e78: 6 total, 5 up, 6 in 2024-06-03T01:28:49.084 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:48 smithi044 podman[93572]: 2024-06-03 01:28:48.833239964 +0000 UTC m=+0.611035656 container remove c8a94e1e6fc6a326dc9a0af72dd33b1e64a9ff597cf3e64c64888e9a8500ebe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:28:49.084 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:48 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-03T01:28:49.085 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:48 smithi044 systemd[1]: Stopped Ceph osd.3 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:28:49.085 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:48 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.3.service: Consumed 1min 14.583s CPU time. 2024-06-03T01:28:49.085 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 systemd[1]: Starting Ceph osd.3 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:28:49.387 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.29261362 +0000 UTC m=+0.102117883 container create 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T01:28:49.387 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.208204462 +0000 UTC m=+0.017708681 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:49.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 38/366 objects degraded (10.383%), 8 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:49.722 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.504146877 +0000 UTC m=+0.313651102 container init 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:28:49.722 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.507997479 +0000 UTC m=+0.317501706 container start 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-03T01:28:49.722 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.578338606 +0000 UTC m=+0.387842825 container attach 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-03T01:28:49.788 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:49 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 38/366 objects degraded (10.383%), 8 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:50.290 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-03T01:28:50.290 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-03T01:28:50.290 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-03T01:28:50.290 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 bash[93670]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate[93686]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-03T01:28:50.291 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:49 smithi044 podman[93670]: 2024-06-03 01:28:49.997928167 +0000 UTC m=+0.807432378 container died 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T01:28:50.581 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:50 smithi044 ceph-mon[84214]: pgmap v128: 65 pgs: 6 active+undersized, 8 active+undersized+degraded, 3 peering, 5 stale+active+clean, 43 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38/366 objects degraded (10.383%) 2024-06-03T01:28:50.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:50 smithi032 ceph-mon[111919]: pgmap v128: 65 pgs: 6 active+undersized, 8 active+undersized+degraded, 3 peering, 5 stale+active+clean, 43 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38/366 objects degraded (10.383%) 2024-06-03T01:28:50.901 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:50 smithi044 podman[93806]: 2024-06-03 01:28:50.582433048 +0000 UTC m=+0.573844841 container remove 100f1290caf942c762d1deb2932dd098e79e7512f6ba73028ef721bc05c0d759 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:28:50.901 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:50 smithi044 podman[93867]: 2024-06-03 01:28:50.799940886 +0000 UTC m=+0.106361115 container create 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-03T01:28:50.901 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:50 smithi044 podman[93867]: 2024-06-03 01:28:50.71207846 +0000 UTC m=+0.018498696 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:28:51.471 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 podman[93867]: 2024-06-03 01:28:51.010589077 +0000 UTC m=+0.317009300 container init 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T01:28:51.472 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 podman[93867]: 2024-06-03 01:28:51.014410554 +0000 UTC m=+0.320830786 container start 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-03T01:28:51.472 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 ceph-osd[93886]: -- 172.21.15.44:0/2455692699 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae12825680 con 0x55ae127eec00 2024-06-03T01:28:51.472 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 bash[93867]: 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba 2024-06-03T01:28:51.472 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 systemd[1]: Started Ceph osd.3 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:28:51.971 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:51 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:28:51.553+0000 7f3bab87e7c0 -1 Falling back to public interface 2024-06-03T01:28:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: pgmap v129: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 3 peering, 2 stale+active+clean, 40 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43/366 objects degraded (11.749%) 2024-06-03T01:28:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: pgmap v129: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 3 peering, 2 stale+active+clean, 40 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43/366 objects degraded (11.749%) 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:28:53.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:53 smithi044 ceph-mon[84214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:28:53.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:53 smithi032 ceph-mon[111919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:28:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:54 smithi044 ceph-mon[84214]: pgmap v130: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:54 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:54 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:54.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:54 smithi032 ceph-mon[111919]: pgmap v130: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:54.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:54 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:54.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:54 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:56.249 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:56 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:28:56.145+0000 7f3bab87e7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-06-03T01:28:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:56 smithi032 ceph-mon[111919]: pgmap v131: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:56 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:56.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:56 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:56.673 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:56 smithi044 ceph-mon[84214]: pgmap v131: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:56.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:56 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:56.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:56 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:28:56.957 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:56 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:28:56.673+0000 7f3bab87e7c0 -1 osd.3 76 log_to_monitors true 2024-06-03T01:28:57.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:57 smithi044 ceph-mon[84214]: from='osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:28:57.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:57 smithi044 ceph-mon[84214]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:28:57.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:57 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 67/366 objects degraded (18.306%), 17 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:57.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:57 smithi032 ceph-mon[111919]: from='osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:28:57.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:57 smithi032 ceph-mon[111919]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T01:28:57.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:57 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 67/366 objects degraded (18.306%), 17 pgs degraded (PG_DEGRADED) 2024-06-03T01:28:58.577 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:28:58.317+0000 7f3b98ccd700 -1 osd.3 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: pgmap v132: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: from='osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: osdmap e79: 6 total, 5 up, 6 in 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:28:58.578 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:58 smithi044 ceph-mon[84214]: from='osd.3 ' entity='osd.3' 2024-06-03T01:28:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: pgmap v132: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:28:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T01:28:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: from='osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:28:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: osdmap e79: 6 total, 5 up, 6 in 2024-06-03T01:28:58.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:28:58.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:58 smithi032 ceph-mon[111919]: from='osd.3 ' entity='osd.3' 2024-06-03T01:29:00.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: pgmap v134: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044] boot 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: osdmap e80: 6 total, 6 up, 6 in 2024-06-03T01:29:00.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:28:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:29:00.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: pgmap v134: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: osd.3 [v2:172.21.15.44:6800/2551114044,v1:172.21.15.44:6801/2551114044] boot 2024-06-03T01:29:00.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: osdmap e80: 6 total, 6 up, 6 in 2024-06-03T01:29:00.337 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:28:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:29:01.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:01 smithi032 ceph-mon[111919]: osdmap e81: 6 total, 6 up, 6 in 2024-06-03T01:29:01.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:01 smithi044 ceph-mon[84214]: osdmap e81: 6 total, 6 up, 6 in 2024-06-03T01:29:03.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:02 smithi032 ceph-mon[111919]: pgmap v137: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:29:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:02 smithi044 ceph-mon[84214]: pgmap v137: 65 pgs: 16 active+undersized, 17 active+undersized+degraded, 32 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/366 objects degraded (18.306%) 2024-06-03T01:29:04.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:03 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 67/366 objects degraded (18.306%), 17 pgs degraded) 2024-06-03T01:29:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:03 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 67/366 objects degraded (18.306%), 17 pgs degraded) 2024-06-03T01:29:05.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:04 smithi032 ceph-mon[111919]: pgmap v138: 65 pgs: 3 active+undersized, 62 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:04 smithi044 ceph-mon[84214]: pgmap v138: 65 pgs: 3 active+undersized, 62 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:07.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:06 smithi032 ceph-mon[111919]: pgmap v139: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:07.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:06 smithi044 ceph-mon[84214]: pgmap v139: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:08 smithi032 ceph-mon[111919]: pgmap v140: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:08.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:08 smithi044 ceph-mon[84214]: pgmap v140: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:08.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 -- 172.21.15.32:0/1223214220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c101850 msgr2=0x7f8d9c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1223214220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c101850 0x7f8d9c1058a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009910 tx=0x7f8d90009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 -- 172.21.15.32:0/1223214220 shutdown_connections 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1223214220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c101850 0x7f8d9c1058a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1223214220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 -- 172.21.15.32:0/1223214220 >> 172.21.15.32:0/1223214220 conn(0x7f8d9c0fc750 msgr2=0x7f8d9c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 -- 172.21.15.32:0/1223214220 shutdown_connections 2024-06-03T01:29:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.625+0000 7f8da0f24700 1 -- 172.21.15.32:0/1223214220 wait complete. 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.626+0000 7f8da0f24700 1 Processor -- start 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.626+0000 7f8da0f24700 1 -- start start 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.626+0000 7f8da0f24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8da0f24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c101850 0x7f8d9c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8da0f24700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c19f4a0 con 0x7f8d9c100ea0 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8da0f24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c198950 con 0x7f8d9c101850 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34064/0 (socket says 172.21.15.32:34064) 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 -- 172.21.15.32:0/1634606221 learned_addr learned my addr 172.21.15.32:0/1634606221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d99d9b700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c101850 0x7f8d9c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 -- 172.21.15.32:0/1634606221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c101850 msgr2=0x7f8d9c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c101850 0x7f8d9c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 -- 172.21.15.32:0/1634606221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900095f0 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.627+0000 7f8d9a59c700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c8a0 tx=0x7f8d8c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.628+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8c0078e0 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.628+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c198c30 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.628+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d8c00f450 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.628+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8c00e5e0 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.628+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c199180 con 0x7f8d9c100ea0 2024-06-03T01:29:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.629+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c105020 con 0x7f8d9c100ea0 2024-06-03T01:29:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.629+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d8c00e740 con 0x7f8d9c100ea0 2024-06-03T01:29:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.630+0000 7f8d8b7fe700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 0x7f8d840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.630+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f8d8c08e190 con 0x7f8d9c100ea0 2024-06-03T01:29:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.630+0000 7f8d99d9b700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 0x7f8d840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.631+0000 7f8d99d9b700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 0x7f8d840706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0fcee0 tx=0x7f8d9001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.635+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8d8c05afb0 con 0x7f8d9c100ea0 2024-06-03T01:29:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.864+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d9c1993a0 con 0x7f8d8406e200 2024-06-03T01:29:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.865+0000 7f8d8b7fe700 1 -- 172.21.15.32:0/1634606221 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8d9c1993a0 con 0x7f8d8406e200 2024-06-03T01:29:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 msgr2=0x7f8d840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 0x7f8d840706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0fcee0 tx=0x7f8d9001a040 comp rx=0 tx=0).stop 2024-06-03T01:29:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 msgr2=0x7f8d9c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c8a0 tx=0x7f8d8c00cbb0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 shutdown_connections 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8d8406e200 0x7f8d840706c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d9c100ea0 0x7f8d9c19e8d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 --2- 172.21.15.32:0/1634606221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d9c101850 0x7f8d9c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 >> 172.21.15.32:0/1634606221 conn(0x7f8d9c0fc750 msgr2=0x7f8d9c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 shutdown_connections 2024-06-03T01:29:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.867+0000 7f8da0f24700 1 -- 172.21.15.32:0/1634606221 wait complete. 2024-06-03T01:29:08.878 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.977+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/4033064649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc0ffb40 msgr2=0x7fd6cc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.977+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/4033064649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc0fffc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009910 tx=0x7fd6c0009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.977+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/4033064649 shutdown_connections 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/4033064649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc0fffc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/4033064649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/4033064649 >> 172.21.15.32:0/4033064649 conn(0x7fd6cc0fb650 msgr2=0x7fd6cc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/4033064649 shutdown_connections 2024-06-03T01:29:08.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/4033064649 wait complete. 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 Processor -- start 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.978+0000 7fd6d1cf8700 1 -- start start 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6d1cf8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6d1cf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6d1cf8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19f500 con 0x7fd6cc105bf0 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6d1cf8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc1989b0 con 0x7fd6cc0ffb40 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 -- 172.21.15.32:0/2622809790 learned_addr learned my addr 172.21.15.32:0/2622809790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:08.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6cb7fe700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 -- 172.21.15.32:0/2622809790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc0ffb40 msgr2=0x7fd6cc19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 -- 172.21.15.32:0/2622809790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00095f0 con 0x7fd6cc105bf0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6caffd700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006010 tx=0x7fd6c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c001d070 con 0x7fd6cc105bf0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c0004fb0 con 0x7fd6cc105bf0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc198c30 con 0x7fd6cc105bf0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.979+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c0022ea0 con 0x7fd6cc105bf0 2024-06-03T01:29:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.980+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc199120 con 0x7fd6cc105bf0 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.980+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc04f2e0 con 0x7fd6cc105bf0 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.981+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd6c0022470 con 0x7fd6cc105bf0 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.981+0000 7fd6c8ff9700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 0x7fd6b40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.982+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fd6c008f680 con 0x7fd6cc105bf0 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.982+0000 7fd6cb7fe700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 0x7fd6b40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:08.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.982+0000 7fd6cb7fe700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 0x7fd6b40706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc006fd0 tx=0x7fd6bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:08.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:08.986+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd6c005c560 con 0x7fd6cc105bf0 2024-06-03T01:29:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.213+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6cc199450 con 0x7fd6b406e200 2024-06-03T01:29:09.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.215+0000 7fd6c8ff9700 1 -- 172.21.15.32:0/2622809790 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd6cc199450 con 0x7fd6b406e200 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 msgr2=0x7fd6b40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 0x7fd6b40706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc006fd0 tx=0x7fd6bc008040 comp rx=0 tx=0).stop 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 msgr2=0x7fd6cc19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006010 tx=0x7fd6c000be20 comp rx=0 tx=0).stop 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 shutdown_connections 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd6b406e200 0x7fd6b40706c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc0ffb40 0x7fd6cc19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 --2- 172.21.15.32:0/2622809790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6cc105bf0 0x7fd6cc19ee70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 >> 172.21.15.32:0/2622809790 conn(0x7fd6cc0fb650 msgr2=0x7fd6cc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 shutdown_connections 2024-06-03T01:29:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.217+0000 7fd6d1cf8700 1 -- 172.21.15.32:0/2622809790 wait complete. 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 -- 172.21.15.32:0/3067639241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef90101bb0 msgr2=0x7fef90101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 --2- 172.21.15.32:0/3067639241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef90101bb0 0x7fef90101f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef84009910 tx=0x7fef84009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 -- 172.21.15.32:0/3067639241 shutdown_connections 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 --2- 172.21.15.32:0/3067639241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef900ff340 0x7fef900ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 --2- 172.21.15.32:0/3067639241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef90101bb0 0x7fef90101f90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.325+0000 7fef97203700 1 -- 172.21.15.32:0/3067639241 >> 172.21.15.32:0/3067639241 conn(0x7fef90074e30 msgr2=0x7fef90075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:09.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.326+0000 7fef97203700 1 -- 172.21.15.32:0/3067639241 shutdown_connections 2024-06-03T01:29:09.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.326+0000 7fef97203700 1 -- 172.21.15.32:0/3067639241 wait complete. 2024-06-03T01:29:09.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.326+0000 7fef97203700 1 Processor -- start 2024-06-03T01:29:09.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.326+0000 7fef97203700 1 -- start start 2024-06-03T01:29:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef97203700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef97203700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 0x7fef9019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef97203700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9019f410 con 0x7fef900ff340 2024-06-03T01:29:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef97203700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901988c0 con 0x7fef90101bb0 2024-06-03T01:29:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34110/0 (socket says 172.21.15.32:34110) 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 -- 172.21.15.32:0/3307769602 learned_addr learned my addr 172.21.15.32:0/3307769602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef8ffff700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 0x7fef9019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 -- 172.21.15.32:0/3307769602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 msgr2=0x7fef9019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 0x7fef9019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 -- 172.21.15.32:0/3307769602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef8ffff700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 0x7fef9019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef94f9f700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8401d070 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef84005e00 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef90198b40 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.327+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef84017400 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.328+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef90199030 con 0x7fef900ff340 2024-06-03T01:29:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.328+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9010b230 con 0x7fef900ff340 2024-06-03T01:29:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.329+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fef8400f460 con 0x7fef900ff340 2024-06-03T01:29:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.329+0000 7fef8dffb700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 0x7fef800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.330+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fef8408f730 con 0x7fef900ff340 2024-06-03T01:29:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.330+0000 7fef8ffff700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 0x7fef800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.330+0000 7fef8ffff700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 0x7fef800706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005950 tx=0x7fef7c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.334+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fef8405c4e0 con 0x7fef900ff340 2024-06-03T01:29:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.550+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef90199250 con 0x7fef8006e200 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.562+0000 7fef8dffb700 1 -- 172.21.15.32:0/3307769602 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fef90199250 con 0x7fef8006e200 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 48s ago 16m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (16m) 48s ago 16m 8656k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (15m) 15s ago 15m 8166k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (2m) 48s ago 16m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (2m) 15s ago 15m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:29:09.562 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 48s ago 16m 81.4M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (10m) 48s ago 10m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (10m) 48s ago 10m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (10m) 15s ago 10m 16.9M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (10m) 15s ago 10m 18.6M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (7m) 48s ago 17m 505M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (6m) 15s ago 15m 451M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (3m) 48s ago 17m 49.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (3m) 15s ago 15m 39.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (6m) 48s ago 16m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 15s ago 15m 15.8M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (2m) 48s ago 14m 138M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (90s) 48s ago 14m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:29:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (54s) 48s ago 13m 12.1M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:29:09.564 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (18s) 15s ago 13m 15.1M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:29:09.564 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (12m) 15s ago 12m 247M 2241M 18.2.1 5be31c24972a 6333fa08fe0a 2024-06-03T01:29:09.564 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (12m) 15s ago 12m 264M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:29:09.564 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 48s ago 15m 78.3M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:29:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.564+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 msgr2=0x7fef800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.564+0000 7fef97203700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 0x7fef800706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005950 tx=0x7fef7c0058e0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 msgr2=0x7fef9019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef8400be20 comp rx=0 tx=0).stop 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 shutdown_connections 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fef8006e200 0x7fef800706c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef900ff340 0x7fef9019e840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 --2- 172.21.15.32:0/3307769602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef90101bb0 0x7fef9019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 >> 172.21.15.32:0/3307769602 conn(0x7fef90074e30 msgr2=0x7fef900fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 shutdown_connections 2024-06-03T01:29:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.565+0000 7fef97203700 1 -- 172.21.15.32:0/3307769602 wait complete. 2024-06-03T01:29:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 -- 172.21.15.32:0/2572258026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a14100020 msgr2=0x7f8a14103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/2572258026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a14100020 0x7f8a14103f10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800aa20 tx=0x7f8a0800ad30 comp rx=0 tx=0).stop 2024-06-03T01:29:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 -- 172.21.15.32:0/2572258026 shutdown_connections 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/2572258026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a14100020 0x7f8a14103f10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/2572258026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a140ff670 0x7f8a140ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 -- 172.21.15.32:0/2572258026 >> 172.21.15.32:0/2572258026 conn(0x7f8a14074e30 msgr2=0x7f8a14075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 -- 172.21.15.32:0/2572258026 shutdown_connections 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 -- 172.21.15.32:0/2572258026 wait complete. 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.672+0000 7f8a1b15a700 1 Processor -- start 2024-06-03T01:29:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a1b15a700 1 -- start start 2024-06-03T01:29:09.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a1b15a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a1b15a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a14100020 0x7f8a141014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a1b15a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14101ac0 con 0x7f8a140ff670 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a1b15a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14101c30 con 0x7f8a14100020 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34128/0 (socket says 172.21.15.32:34128) 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 -- 172.21.15.32:0/365953384 learned_addr learned my addr 172.21.15.32:0/365953384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a13fff700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a14100020 0x7f8a141014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 -- 172.21.15.32:0/365953384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a14100020 msgr2=0x7f8a141014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a14100020 0x7f8a141014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 -- 172.21.15.32:0/365953384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0800a700 con 0x7f8a140ff670 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.673+0000 7f8a18ef6700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400b4b0 tx=0x7f8a0400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.674+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a04011800 con 0x7f8a140ff670 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.674+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a04011e20 con 0x7f8a140ff670 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.674+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141a2af0 con 0x7f8a140ff670 2024-06-03T01:29:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.674+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0400f2e0 con 0x7f8a140ff670 2024-06-03T01:29:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.674+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141a2f00 con 0x7f8a140ff670 2024-06-03T01:29:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.675+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14109870 con 0x7f8a140ff670 2024-06-03T01:29:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.675+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8a04010380 con 0x7f8a140ff670 2024-06-03T01:29:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.676+0000 7f8a11ffb700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 0x7f89fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.676+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f8a0408ddb0 con 0x7f8a140ff670 2024-06-03T01:29:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.676+0000 7f8a13fff700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 0x7f89fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.677+0000 7f8a13fff700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 0x7f89fc0706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800afd0 tx=0x7f8a08005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.681+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8a04057030 con 0x7f8a140ff670 2024-06-03T01:29:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.982+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8a1404f2e0 con 0x7f8a140ff670 2024-06-03T01:29:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.982+0000 7f8a11ffb700 1 -- 172.21.15.32:0/365953384 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+787 (secure 0 0 0) 0x7f8a0405a6f0 con 0x7f8a140ff670 2024-06-03T01:29:09.982 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:29:09.982 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:29:09.982 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:29:09.983 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 8 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 msgr2=0x7f89fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 0x7f89fc0706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800afd0 tx=0x7f8a08005dc0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 msgr2=0x7f8a14100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:09.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400b4b0 tx=0x7f8a0400b870 comp rx=0 tx=0).stop 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 shutdown_connections 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f89fc06e200 0x7f89fc0706c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a140ff670 0x7f8a14100fb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 --2- 172.21.15.32:0/365953384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a14100020 0x7f8a141014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 >> 172.21.15.32:0/365953384 conn(0x7f8a14074e30 msgr2=0x7f8a140fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 shutdown_connections 2024-06-03T01:29:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:09.984+0000 7f8a1b15a700 1 -- 172.21.15.32:0/365953384 wait complete. 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 -- 172.21.15.32:0/3834901006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 msgr2=0x7fb664100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 --2- 172.21.15.32:0/3834901006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb664100180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb654009910 tx=0x7fb654009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 -- 172.21.15.32:0/3834901006 shutdown_connections 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 --2- 172.21.15.32:0/3834901006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb664100750 0x7fb6641047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 --2- 172.21.15.32:0/3834901006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb664100180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 -- 172.21.15.32:0/3834901006 >> 172.21.15.32:0/3834901006 conn(0x7fb6640fb650 msgr2=0x7fb6640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:10.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 -- 172.21.15.32:0/3834901006 shutdown_connections 2024-06-03T01:29:10.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.094+0000 7fb668c17700 1 -- 172.21.15.32:0/3834901006 wait complete. 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 Processor -- start 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 -- start start 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb664100750 0x7fb664197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664198230 con 0x7fb6640ffda0 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb668c17700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641983a0 con 0x7fb664100750 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34152/0 (socket says 172.21.15.32:34152) 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 -- 172.21.15.32:0/1960713801 learned_addr learned my addr 172.21.15.32:0/1960713801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 -- 172.21.15.32:0/1960713801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb664100750 msgr2=0x7fb664197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb664100750 0x7fb664197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.095+0000 7fb66259c700 1 -- 172.21.15.32:0/1960713801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540095f0 con 0x7fb6640ffda0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb66259c700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb654006010 tx=0x7fb65400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65401d070 con 0x7fb6640ffda0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6641a0740 con 0x7fb6640ffda0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb65400f460 con 0x7fb6640ffda0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb654017630 con 0x7fb6640ffda0 2024-06-03T01:29:10.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.096+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a0bb0 con 0x7fb6640ffda0 2024-06-03T01:29:10.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.097+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664108140 con 0x7fb6640ffda0 2024-06-03T01:29:10.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.097+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb65400f5d0 con 0x7fb6640ffda0 2024-06-03T01:29:10.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.098+0000 7fb6537fe700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 0x7fb64c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.098+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fb65408f660 con 0x7fb6640ffda0 2024-06-03T01:29:10.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.098+0000 7fb661d9b700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 0x7fb64c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.098+0000 7fb661d9b700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 0x7fb64c0706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb658005fd0 tx=0x7fb658005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.103+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb6540588e0 con 0x7fb6640ffda0 2024-06-03T01:29:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:10 smithi032 ceph-mon[111919]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:10 smithi032 ceph-mon[111919]: pgmap v141: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:10 smithi032 ceph-mon[111919]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:10 smithi032 ceph-mon[111919]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:10 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/365953384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.361+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb66404f2e0 con 0x7fb6640ffda0 2024-06-03T01:29:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.361+0000 7fb6537fe700 1 -- 172.21.15.32:0/1960713801 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7fb65405bfa0 con 0x7fb6640ffda0 2024-06-03T01:29:10.361 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:29:10.361 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:29:10.362 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:29:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 msgr2=0x7fb64c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 0x7fb64c0706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb658005fd0 tx=0x7fb658005ee0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 msgr2=0x7fb6641976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb654006010 tx=0x7fb65400be20 comp rx=0 tx=0).stop 2024-06-03T01:29:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 shutdown_connections 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb64c06e200 0x7fb64c0706c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6640ffda0 0x7fb6641976d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 --2- 172.21.15.32:0/1960713801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb664100750 0x7fb664197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.363+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 >> 172.21.15.32:0/1960713801 conn(0x7fb6640fb650 msgr2=0x7fb6640fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.364+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 shutdown_connections 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.364+0000 7fb668c17700 1 -- 172.21.15.32:0/1960713801 wait complete. 2024-06-03T01:29:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:29:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:10 smithi044 ceph-mon[84214]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:10 smithi044 ceph-mon[84214]: pgmap v141: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:10 smithi044 ceph-mon[84214]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:10 smithi044 ceph-mon[84214]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:10.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:10 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/365953384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.472+0000 7f66d5800700 1 -- 172.21.15.32:0/142474933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 msgr2=0x7f66d01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.472+0000 7f66d5800700 1 --2- 172.21.15.32:0/142474933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d01058a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009910 tx=0x7f66c4009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 -- 172.21.15.32:0/142474933 shutdown_connections 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 --2- 172.21.15.32:0/142474933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d01058a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 --2- 172.21.15.32:0/142474933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 0x7f66d0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 -- 172.21.15.32:0/142474933 >> 172.21.15.32:0/142474933 conn(0x7f66d00fc750 msgr2=0x7f66d00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 -- 172.21.15.32:0/142474933 shutdown_connections 2024-06-03T01:29:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 -- 172.21.15.32:0/142474933 wait complete. 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.473+0000 7f66d5800700 1 Processor -- start 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66d5800700 1 -- start start 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66d5800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 0x7f66d019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66d5800700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66d5800700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d019d2d0 con 0x7f66d0101850 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66d5800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0196780 con 0x7f66d0100ea0 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34172/0 (socket says 172.21.15.32:34172) 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 -- 172.21.15.32:0/4029010390 learned_addr learned my addr 172.21.15.32:0/4029010390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ceffd700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 0x7f66d019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 -- 172.21.15.32:0/4029010390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 msgr2=0x7f66d019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 0x7f66d019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.474+0000 7f66ce7fc700 1 -- 172.21.15.32:0/4029010390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40095f0 con 0x7f66d0101850 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66ce7fc700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66c40092e0 tx=0x7f66c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c401d070 con 0x7f66d0101850 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66c400f460 con 0x7f66d0101850 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d0196a00 con 0x7f66d0101850 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c40175d0 con 0x7f66d0101850 2024-06-03T01:29:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.475+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0196ef0 con 0x7f66d0101850 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.476+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d01092a0 con 0x7f66d0101850 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.476+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f66c4017840 con 0x7f66d0101850 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.477+0000 7f66b7fff700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 0x7f66b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.477+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f66c408f610 con 0x7f66d0101850 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.477+0000 7f66ceffd700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 0x7f66b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.477+0000 7f66ceffd700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 0x7f66b80706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005950 tx=0x7f66c000a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.481+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f66c405c4b0 con 0x7f66d0101850 2024-06-03T01:29:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.707+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66d0197160 con 0x7f66b806e200 2024-06-03T01:29:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.708+0000 7f66b7fff700 1 -- 172.21.15.32:0/4029010390 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f66d0197160 con 0x7f66b806e200 2024-06-03T01:29:10.708 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "10/23 daemons upgraded", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:29:10.709 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:29:10.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 msgr2=0x7f66b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 0x7f66b80706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005950 tx=0x7f66c000a300 comp rx=0 tx=0).stop 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 msgr2=0x7f66d019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66c40092e0 tx=0x7f66c400bd70 comp rx=0 tx=0).stop 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 shutdown_connections 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f66b806e200 0x7f66b80706c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d0100ea0 0x7f66d019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 --2- 172.21.15.32:0/4029010390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d0101850 0x7f66d019cc40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 >> 172.21.15.32:0/4029010390 conn(0x7f66d00fc750 msgr2=0x7f66d00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 shutdown_connections 2024-06-03T01:29:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.711+0000 7f66d5800700 1 -- 172.21.15.32:0/4029010390 wait complete. 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.819+0000 7faca54a1700 1 -- 172.21.15.32:0/3955112414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 msgr2=0x7faca010d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.819+0000 7faca54a1700 1 --2- 172.21.15.32:0/3955112414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca010d290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac88009910 tx=0x7fac88009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 -- 172.21.15.32:0/3955112414 shutdown_connections 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 --2- 172.21.15.32:0/3955112414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca010d290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 --2- 172.21.15.32:0/3955112414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca0068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 -- 172.21.15.32:0/3955112414 >> 172.21.15.32:0/3955112414 conn(0x7faca0075670 msgr2=0x7faca0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 -- 172.21.15.32:0/3955112414 shutdown_connections 2024-06-03T01:29:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 -- 172.21.15.32:0/3955112414 wait complete. 2024-06-03T01:29:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 Processor -- start 2024-06-03T01:29:10.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.820+0000 7faca54a1700 1 -- start start 2024-06-03T01:29:10.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7faca54a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7faca54a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7faca54a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca019f580 con 0x7faca0068ea0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7faca54a1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0198a30 con 0x7faca0068580 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34178/0 (socket says 172.21.15.32:34178) 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 -- 172.21.15.32:0/1259256714 learned_addr learned my addr 172.21.15.32:0/1259256714 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9effd700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 -- 172.21.15.32:0/1259256714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 msgr2=0x7faca019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 -- 172.21.15.32:0/1259256714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880095f0 con 0x7faca0068ea0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9effd700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac9e7fc700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac88006010 tx=0x7fac8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.821+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac8801d070 con 0x7faca0068ea0 2024-06-03T01:29:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.822+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac88004fb0 con 0x7faca0068ea0 2024-06-03T01:29:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.822+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca0198cb0 con 0x7faca0068ea0 2024-06-03T01:29:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.822+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac88022ea0 con 0x7faca0068ea0 2024-06-03T01:29:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.822+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca01991a0 con 0x7faca0068ea0 2024-06-03T01:29:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.823+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca010aa00 con 0x7faca0068ea0 2024-06-03T01:29:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.823+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fac88022470 con 0x7faca0068ea0 2024-06-03T01:29:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.824+0000 7fac97fff700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 0x7fac8c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.824+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fac8808f770 con 0x7faca0068ea0 2024-06-03T01:29:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.824+0000 7fac9effd700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 0x7fac8c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.824+0000 7fac9effd700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 0x7fac8c0706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac90009fa0 tx=0x7fac90009570 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:10.829+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fac8805c5d0 con 0x7faca0068ea0 2024-06-03T01:29:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.122+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faca004f2e0 con 0x7faca0068ea0 2024-06-03T01:29:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.122+0000 7fac97fff700 1 -- 172.21.15.32:0/1259256714 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fac88022720 con 0x7faca0068ea0 2024-06-03T01:29:11.123 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:29:11.123 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:29:11.123 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 msgr2=0x7fac8c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 0x7fac8c0706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac90009fa0 tx=0x7fac90009570 comp rx=0 tx=0).stop 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 msgr2=0x7faca019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac88006010 tx=0x7fac8800be20 comp rx=0 tx=0).stop 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 shutdown_connections 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fac8c06e200 0x7fac8c0706c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faca0068580 0x7faca019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 --2- 172.21.15.32:0/1259256714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faca0068ea0 0x7faca019eef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 >> 172.21.15.32:0/1259256714 conn(0x7faca0075670 msgr2=0x7faca00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 shutdown_connections 2024-06-03T01:29:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:11.125+0000 7faca54a1700 1 -- 172.21.15.32:0/1259256714 wait complete. 2024-06-03T01:29:11.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:11 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1960713801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:29:11.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:11 smithi044 ceph-mon[84214]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:11 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1960713801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:29:11.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:11 smithi032 ceph-mon[111919]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:12.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:12 smithi032 ceph-mon[111919]: pgmap v142: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:12.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:12 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1259256714' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:29:12.329 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:12 smithi044 ceph-mon[84214]: pgmap v142: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:12.329 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:12 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1259256714' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:29:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:13 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:13.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:13 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:14.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:14 smithi032 ceph-mon[111919]: pgmap v143: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:14 smithi044 ceph-mon[84214]: pgmap v143: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:15 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:15 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:15.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:15 smithi032 ceph-mon[111919]: Upgrade: osd.4 is safe to restart 2024-06-03T01:29:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:15 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:15 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-03T01:29:15.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:15 smithi044 ceph-mon[84214]: Upgrade: osd.4 is safe to restart 2024-06-03T01:29:16.919 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: pgmap v144: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:16.920 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: Upgrade: Updating osd.4 2024-06-03T01:29:16.920 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:16.920 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:29:16.920 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:16.920 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:16 smithi044 ceph-mon[84214]: Deploying daemon osd.4 on smithi044 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: pgmap v144: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: Upgrade: Updating osd.4 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:17.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:16 smithi032 ceph-mon[111919]: Deploying daemon osd.4 on smithi044 2024-06-03T01:29:18.915 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:18 smithi044 ceph-mon[84214]: pgmap v145: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:18 smithi032 ceph-mon[111919]: pgmap v145: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:19.191 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:18 smithi044 systemd[1]: Stopping Ceph osd.4 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:29:19.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:19 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:29:19.190+0000 7fecec407700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:29:19.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:19 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:29:19.190+0000 7fecec407700 -1 osd.4 81 *** Got signal Terminated *** 2024-06-03T01:29:19.472 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:19 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[49544]: 2024-06-03T01:29:19.190+0000 7fecec407700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:29:19.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:19 smithi044 ceph-mon[84214]: osd.4 marked itself down and dead 2024-06-03T01:29:20.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:19 smithi032 ceph-mon[111919]: osd.4 marked itself down and dead 2024-06-03T01:29:20.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:20 smithi044 podman[98321]: 2024-06-03 01:29:20.219602919 +0000 UTC m=+1.129463186 container died 6333fa08fe0aeada0b43e2c75d50243b1d7fb627408d7f88ccc91a5bbf5c0909 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T01:29:20.808 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:20 smithi044 ceph-mon[84214]: pgmap v146: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:20.808 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:20 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:29:20.808 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:20 smithi044 ceph-mon[84214]: osdmap e82: 6 total, 5 up, 6 in 2024-06-03T01:29:20.808 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:20 smithi044 podman[98321]: 2024-06-03 01:29:20.565716713 +0000 UTC m=+1.475576978 container cleanup 6333fa08fe0aeada0b43e2c75d50243b1d7fb627408d7f88ccc91a5bbf5c0909 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T01:29:20.809 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:20 smithi044 bash[98321]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4 2024-06-03T01:29:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:20 smithi032 ceph-mon[111919]: pgmap v146: 65 pgs: 65 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:20 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:29:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:20 smithi032 ceph-mon[111919]: osdmap e82: 6 total, 5 up, 6 in 2024-06-03T01:29:21.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:20 smithi044 podman[98347]: 2024-06-03 01:29:20.808941412 +0000 UTC m=+0.583776673 container remove 6333fa08fe0aeada0b43e2c75d50243b1d7fb627408d7f88ccc91a5bbf5c0909 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=-18.2.1) 2024-06-03T01:29:21.537 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.131714061 +0000 UTC m=+0.018013679 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:21.537 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.260046453 +0000 UTC m=+0.146346060 container create 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-03T01:29:21.537 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.463562293 +0000 UTC m=+0.349861900 container init 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-03T01:29:21.537 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.467104411 +0000 UTC m=+0.353404018 container start 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:29:21.537 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.53731912 +0000 UTC m=+0.423618737 container attach 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-03T01:29:21.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:21 smithi044 ceph-mon[84214]: osdmap e83: 6 total, 5 up, 6 in 2024-06-03T01:29:21.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:21 smithi044 podman[98437]: 2024-06-03 01:29:21.774495228 +0000 UTC m=+0.660794843 container died 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:29:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:21 smithi032 ceph-mon[111919]: osdmap e83: 6 total, 5 up, 6 in 2024-06-03T01:29:22.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:22 smithi044 ceph-mon[84214]: pgmap v149: 65 pgs: 5 stale+active+clean, 60 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:22.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:22.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:22 smithi044 podman[98464]: 2024-06-03 01:29:22.766704476 +0000 UTC m=+0.981388192 container remove 6a69f0ab9f1435e07ed225cc88e5ff8cc53ed202b8f22c1b04cfd8344a285db6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:29:22.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:22 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-03T01:29:22.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:22 smithi044 systemd[1]: Stopped Ceph osd.4 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:29:22.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:22 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.4.service: Consumed 1min 5.440s CPU time. 2024-06-03T01:29:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:22 smithi032 ceph-mon[111919]: pgmap v149: 65 pgs: 5 stale+active+clean, 60 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:23.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:23.341 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 systemd[1]: Starting Ceph osd.4 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:29:23.341 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.131292904 +0000 UTC m=+0.018367967 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:23.342 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.234545953 +0000 UTC m=+0.121620995 container create 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:29:23.721 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.462502595 +0000 UTC m=+0.349577650 container init 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-03T01:29:23.722 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.466294757 +0000 UTC m=+0.353369810 container start 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-03T01:29:23.722 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.537280133 +0000 UTC m=+0.424355201 container attach 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-03T01:29:24.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:23 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 51/366 objects degraded (13.934%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 51/366 objects degraded (13.934%), 13 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:24.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 bash[98566]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-06-03T01:29:24.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-03T01:29:24.223 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate[98581]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-03T01:29:24.223 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:23 smithi044 podman[98566]: 2024-06-03 01:29:23.958719099 +0000 UTC m=+0.845794152 container died 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-03T01:29:24.918 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:24 smithi044 ceph-mon[84214]: pgmap v150: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 1 stale+active+clean, 38 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-06-03T01:29:24.918 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:24 smithi044 podman[98716]: 2024-06-03 01:29:24.591692206 +0000 UTC m=+0.627128178 container remove 09128d7127b35f78af12868afc9803de98ac8b2052cb20394653f05753d55baa (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:29:24.918 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:24 smithi044 podman[98774]: 2024-06-03 01:29:24.712536521 +0000 UTC m=+0.017713550 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:24.918 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:24 smithi044 podman[98774]: 2024-06-03 01:29:24.817504849 +0000 UTC m=+0.122681864 container create d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-03T01:29:25.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:24 smithi032 ceph-mon[111919]: pgmap v150: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 1 stale+active+clean, 38 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-06-03T01:29:25.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:25 smithi044 podman[98774]: 2024-06-03 01:29:25.036844723 +0000 UTC m=+0.342021748 container init d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-03T01:29:25.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:25 smithi044 podman[98774]: 2024-06-03 01:29:25.040055454 +0000 UTC m=+0.345232479 container start d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-03T01:29:25.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:25 smithi044 bash[98774]: d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 2024-06-03T01:29:25.222 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:25 smithi044 systemd[1]: Started Ceph osd.4 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:29:25.972 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:25 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:29:25.583+0000 7f2de99a77c0 -1 Falling back to public interface 2024-06-03T01:29:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:26 smithi044 ceph-mon[84214]: pgmap v151: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:26.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:26 smithi032 ceph-mon[111919]: pgmap v151: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:26.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:28 smithi032 ceph-mon[111919]: pgmap v152: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:28 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:28 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:28 smithi044 ceph-mon[84214]: pgmap v152: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:28 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:28 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:30.267 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:30 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:29:30.202+0000 7f2de99a77c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-06-03T01:29:30.563 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:30 smithi044 ceph-mon[84214]: pgmap v153: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:30.563 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:30.563 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:30 smithi032 ceph-mon[111919]: pgmap v153: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:30.971 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:30 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:29:30.737+0000 7f2de99a77c0 -1 osd.4 81 log_to_monitors true 2024-06-03T01:29:31.567 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:31 smithi044 ceph-mon[84214]: from='osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:29:31.567 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:31 smithi044 ceph-mon[84214]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:29:31.568 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:29:31 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:29:31.359+0000 7f2ddadfe700 -1 osd.4 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:29:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:31 smithi032 ceph-mon[111919]: from='osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:29:31.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:31 smithi032 ceph-mon[111919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:29:32.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: pgmap v154: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:32.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:29:32.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: osdmap e84: 6 total, 5 up, 6 in 2024-06-03T01:29:32.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: from='osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:29:32.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:29:32.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:32 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: pgmap v154: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: osdmap e84: 6 total, 5 up, 6 in 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: from='osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:29:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:32 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:33.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383] boot 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: osdmap e85: 6 total, 6 up, 6 in 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.723 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: osd.4 [v2:172.21.15.44:6808/4083215383,v1:172.21.15.44:6809/4083215383] boot 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: osdmap e85: 6 total, 6 up, 6 in 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:33.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:34.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:34 smithi044 ceph-mon[84214]: pgmap v157: 65 pgs: 13 peering, 9 active+undersized, 10 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/366 objects degraded (9.290%) 2024-06-03T01:29:34.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:34 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:34.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:34 smithi044 ceph-mon[84214]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-06-03T01:29:34.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:34 smithi044 ceph-mon[84214]: osdmap e86: 6 total, 6 up, 6 in 2024-06-03T01:29:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:34 smithi032 ceph-mon[111919]: pgmap v157: 65 pgs: 13 peering, 9 active+undersized, 10 active+undersized+degraded, 33 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/366 objects degraded (9.290%) 2024-06-03T01:29:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:34 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:34 smithi032 ceph-mon[111919]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-06-03T01:29:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:34 smithi032 ceph-mon[111919]: osdmap e86: 6 total, 6 up, 6 in 2024-06-03T01:29:36.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:36 smithi044 ceph-mon[84214]: pgmap v159: 65 pgs: 13 peering, 6 active+undersized, 8 active+undersized+degraded, 38 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/366 objects degraded (6.557%) 2024-06-03T01:29:36.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:36 smithi032 ceph-mon[111919]: pgmap v159: 65 pgs: 13 peering, 6 active+undersized, 8 active+undersized+degraded, 38 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/366 objects degraded (6.557%) 2024-06-03T01:29:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:37 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:37.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:37 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:37.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:38.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:38 smithi044 ceph-mon[84214]: pgmap v160: 65 pgs: 13 peering, 52 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:38.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:38 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded) 2024-06-03T01:29:38.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:38 smithi032 ceph-mon[111919]: pgmap v160: 65 pgs: 13 peering, 52 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:38.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:38 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded) 2024-06-03T01:29:40.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:40 smithi044 ceph-mon[84214]: pgmap v161: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:40 smithi032 ceph-mon[111919]: pgmap v161: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 -- 172.21.15.32:0/20789959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3840105bd0 msgr2=0x7f3840105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 --2- 172.21.15.32:0/20789959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3840105bd0 0x7f3840105fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3830009910 tx=0x7f3830009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 -- 172.21.15.32:0/20789959 shutdown_connections 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 --2- 172.21.15.32:0/20789959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38400ffc20 0x7f3840100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 --2- 172.21.15.32:0/20789959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3840105bd0 0x7f3840105fb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.247+0000 7f3846a4f700 1 -- 172.21.15.32:0/20789959 >> 172.21.15.32:0/20789959 conn(0x7f3840075670 msgr2=0x7f3840075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.248+0000 7f3846a4f700 1 -- 172.21.15.32:0/20789959 shutdown_connections 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.248+0000 7f3846a4f700 1 -- 172.21.15.32:0/20789959 wait complete. 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.248+0000 7f3846a4f700 1 Processor -- start 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.248+0000 7f3846a4f700 1 -- start start 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f3846a4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f3846a4f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3840105bd0 0x7f3840196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f3846a4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3840197550 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f3846a4f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019b360 con 0x7f3840105bd0 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56656/0 (socket says 172.21.15.32:56656) 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 -- 172.21.15.32:0/2389211523 learned_addr learned my addr 172.21.15.32:0/2389211523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383f7fe700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3840105bd0 0x7f3840196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 -- 172.21.15.32:0/2389211523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3840105bd0 msgr2=0x7f3840196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3840105bd0 0x7f3840196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 -- 172.21.15.32:0/2389211523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300095f0 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383ffff700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3830006010 tx=0x7f383000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.249+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383001d070 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.250+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3830004fb0 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.250+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384019b5e0 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.250+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383000f690 con 0x7f38400ffc20 2024-06-03T01:29:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.250+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384019bad0 con 0x7f38400ffc20 2024-06-03T01:29:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.251+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384004f2e0 con 0x7f38400ffc20 2024-06-03T01:29:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.252+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f38300228a0 con 0x7f38400ffc20 2024-06-03T01:29:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.252+0000 7f383d7fa700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 0x7f38280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.253+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f3830090270 con 0x7f38400ffc20 2024-06-03T01:29:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.253+0000 7f383f7fe700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 0x7f38280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.253+0000 7f383f7fe700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 0x7f38280706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.257+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f383005d000 con 0x7f38400ffc20 2024-06-03T01:29:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.488+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f384019bd60 con 0x7f382806e200 2024-06-03T01:29:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.489+0000 7f383d7fa700 1 -- 172.21.15.32:0/2389211523 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f384019bd60 con 0x7f382806e200 2024-06-03T01:29:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 msgr2=0x7f38280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 0x7f38280706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834008040 comp rx=0 tx=0).stop 2024-06-03T01:29:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 msgr2=0x7f3840196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3830006010 tx=0x7f383000be20 comp rx=0 tx=0).stop 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 shutdown_connections 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f382806e200 0x7f38280706c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38400ffc20 0x7f3840196980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 --2- 172.21.15.32:0/2389211523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3840105bd0 0x7f3840196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 >> 172.21.15.32:0/2389211523 conn(0x7f3840075670 msgr2=0x7f38400686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 shutdown_connections 2024-06-03T01:29:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.491+0000 7f3846a4f700 1 -- 172.21.15.32:0/2389211523 wait complete. 2024-06-03T01:29:41.503 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 -- 172.21.15.32:0/1033365720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0101850 msgr2=0x7f64a01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/1033365720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0101850 0x7f64a01058a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6494009910 tx=0x7f6494009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 -- 172.21.15.32:0/1033365720 shutdown_connections 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/1033365720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0101850 0x7f64a01058a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/1033365720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0100ea0 0x7f64a0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.601+0000 7f64a5b3a700 1 -- 172.21.15.32:0/1033365720 >> 172.21.15.32:0/1033365720 conn(0x7f64a00fc750 msgr2=0x7f64a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.602+0000 7f64a5b3a700 1 -- 172.21.15.32:0/1033365720 shutdown_connections 2024-06-03T01:29:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.602+0000 7f64a5b3a700 1 -- 172.21.15.32:0/1033365720 wait complete. 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.602+0000 7f64a5b3a700 1 Processor -- start 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.602+0000 7f64a5b3a700 1 -- start start 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f64a5b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f64a5b3a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0101850 0x7f64a019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f64a5b3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a019d2d0 con 0x7f64a0100ea0 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f64a5b3a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0196780 con 0x7f64a0101850 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56684/0 (socket says 172.21.15.32:56684) 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 -- 172.21.15.32:0/2635881519 learned_addr learned my addr 172.21.15.32:0/2635881519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649effd700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0101850 0x7f64a019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 -- 172.21.15.32:0/2635881519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0101850 msgr2=0x7f64a019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0101850 0x7f64a019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 -- 172.21.15.32:0/2635881519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940095f0 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.603+0000 7f649f7fe700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f649000c8a0 tx=0x7f649000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.604+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64900078e0 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.604+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f649000f450 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.604+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a0196a60 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.604+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649000e5e0 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.604+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a0196fb0 con 0x7f64a0100ea0 2024-06-03T01:29:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.605+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a01092a0 con 0x7f64a0100ea0 2024-06-03T01:29:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.605+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f649000f5c0 con 0x7f64a0100ea0 2024-06-03T01:29:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.606+0000 7f649cff9700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 0x7f64880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.606+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f649008e1e0 con 0x7f64a0100ea0 2024-06-03T01:29:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.606+0000 7f649effd700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 0x7f64880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.606+0000 7f649effd700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 0x7f64880706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64a0197bf0 tx=0x7f649401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.610+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f649005af90 con 0x7f64a0100ea0 2024-06-03T01:29:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.839+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64a0197d30 con 0x7f648806e200 2024-06-03T01:29:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.840+0000 7f649cff9700 1 -- 172.21.15.32:0/2635881519 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f64a0197d30 con 0x7f648806e200 2024-06-03T01:29:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 msgr2=0x7f64880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 0x7f64880706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64a0197bf0 tx=0x7f649401a040 comp rx=0 tx=0).stop 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 msgr2=0x7f64a019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f649000c8a0 tx=0x7f649000cbb0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 shutdown_connections 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f648806e200 0x7f64880706c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64a0100ea0 0x7f64a019c700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 --2- 172.21.15.32:0/2635881519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64a0101850 0x7f64a019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.842+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 >> 172.21.15.32:0/2635881519 conn(0x7f64a00fc750 msgr2=0x7f64a00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.843+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 shutdown_connections 2024-06-03T01:29:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.843+0000 7f64a5b3a700 1 -- 172.21.15.32:0/2635881519 wait complete. 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 -- 172.21.15.32:0/4205141383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 msgr2=0x7f49f410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 --2- 172.21.15.32:0/4205141383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f410abf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009910 tx=0x7f49e8009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 -- 172.21.15.32:0/4205141383 shutdown_connections 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 --2- 172.21.15.32:0/4205141383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f410abf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 --2- 172.21.15.32:0/4205141383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 0x7f49f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.950+0000 7f49f97fc700 1 -- 172.21.15.32:0/4205141383 >> 172.21.15.32:0/4205141383 conn(0x7f49f40fb650 msgr2=0x7f49f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.951+0000 7f49f97fc700 1 -- 172.21.15.32:0/4205141383 shutdown_connections 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.951+0000 7f49f97fc700 1 -- 172.21.15.32:0/4205141383 wait complete. 2024-06-03T01:29:41.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.951+0000 7f49f97fc700 1 Processor -- start 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.951+0000 7f49f97fc700 1 -- start start 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 0x7f49f419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f419f540 con 0x7f49f4102700 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41989f0 con 0x7f49f4101de0 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56688/0 (socket says 172.21.15.32:56688) 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 -- 172.21.15.32:0/3194805982 learned_addr learned my addr 172.21.15.32:0/3194805982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f2ffd700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 0x7f49f419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 -- 172.21.15.32:0/3194805982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 msgr2=0x7f49f419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 0x7f49f419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 -- 172.21.15.32:0/3194805982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80095f0 con 0x7f49f4102700 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f27fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49e80092e0 tx=0x7f49e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e801d070 con 0x7f49f4102700 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e800f460 con 0x7f49f4102700 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f4198c70 con 0x7f49f4102700 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e8017610 con 0x7f49f4102700 2024-06-03T01:29:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.952+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f4199160 con 0x7f49f4102700 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.953+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f410a750 con 0x7f49f4102700 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.954+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f49e800f5d0 con 0x7f49f4102700 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.955+0000 7f49dbfff700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 0x7f49dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.955+0000 7f49f2ffd700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 0x7f49dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.955+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f49e808f6b0 con 0x7f49f4102700 2024-06-03T01:29:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.955+0000 7f49f2ffd700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 0x7f49dc0706c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:41.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:41.960+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f49e805c460 con 0x7f49f4102700 2024-06-03T01:29:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.177+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f4199f70 con 0x7f49dc06e200 2024-06-03T01:29:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.188+0000 7f49dbfff700 1 -- 172.21.15.32:0/3194805982 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f49f4199f70 con 0x7f49dc06e200 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 81s ago 16m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (17m) 81s ago 17m 8656k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (16m) 14s ago 16m 8250k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (3m) 81s ago 17m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (2m) 14s ago 15m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 81s ago 16m 81.4M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (11m) 81s ago 11m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (11m) 81s ago 11m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (11m) 14s ago 11m 17.1M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (11m) 14s ago 11m 18.8M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:29:42.189 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (8m) 81s ago 18m 505M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (6m) 14s ago 15m 452M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (4m) 81s ago 18m 49.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (3m) 14s ago 15m 40.6M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (6m) 81s ago 17m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 14s ago 15m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (2m) 81s ago 15m 138M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (2m) 81s ago 14m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (86s) 81s ago 14m 12.1M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (51s) 14s ago 13m 101M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (17s) 14s ago 13m 11.7M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (12m) 14s ago 12m 265M 2241M 18.2.1 5be31c24972a 454878053f65 2024-06-03T01:29:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 81s ago 16m 78.3M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 msgr2=0x7f49dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 0x7f49dc0706c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).stop 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 msgr2=0x7f49f419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49e80092e0 tx=0x7f49e800be20 comp rx=0 tx=0).stop 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 shutdown_connections 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f49dc06e200 0x7f49dc0706c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49f4101de0 0x7f49f419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 --2- 172.21.15.32:0/3194805982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49f4102700 0x7f49f419eeb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 >> 172.21.15.32:0/3194805982 conn(0x7f49f40fb650 msgr2=0x7f49f4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 shutdown_connections 2024-06-03T01:29:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.190+0000 7f49f97fc700 1 -- 172.21.15.32:0/3194805982 wait complete. 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- 172.21.15.32:0/871756354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d00ffb40 msgr2=0x7f34d00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 --2- 172.21.15.32:0/871756354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d00ffb40 0x7f34d00fffc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009910 tx=0x7f34c4009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- 172.21.15.32:0/871756354 shutdown_connections 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 --2- 172.21.15.32:0/871756354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d00ffb40 0x7f34d00fffc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 --2- 172.21.15.32:0/871756354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d0105bf0 0x7f34d0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- 172.21.15.32:0/871756354 >> 172.21.15.32:0/871756354 conn(0x7f34d00fb650 msgr2=0x7f34d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- 172.21.15.32:0/871756354 shutdown_connections 2024-06-03T01:29:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- 172.21.15.32:0/871756354 wait complete. 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 Processor -- start 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.297+0000 7f34d563b700 1 -- start start 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00ffb40 0x7f34d01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0102230 con 0x7f34d0105bf0 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0102370 con 0x7f34d00ffb40 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56698/0 (socket says 172.21.15.32:56698) 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 -- 172.21.15.32:0/871143087 learned_addr learned my addr 172.21.15.32:0/871143087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ceffd700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00ffb40 0x7f34d01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 -- 172.21.15.32:0/871143087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00ffb40 msgr2=0x7f34d01017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00ffb40 0x7f34d01017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 -- 172.21.15.32:0/871143087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40095f0 con 0x7f34d0105bf0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34ce7fc700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c401d070 con 0x7f34d0105bf0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34c4004fb0 con 0x7f34d0105bf0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d01025f0 con 0x7f34d0105bf0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c4022ea0 con 0x7f34d0105bf0 2024-06-03T01:29:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.298+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d01a2e70 con 0x7f34d0105bf0 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.299+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d004f2e0 con 0x7f34d0105bf0 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.300+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f34c4022470 con 0x7f34d0105bf0 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.300+0000 7f34b7fff700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 0x7f34b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.301+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f34c408f750 con 0x7f34d0105bf0 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.301+0000 7f34ceffd700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 0x7f34b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.301+0000 7f34ceffd700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 0x7f34b80706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34c0006fd0 tx=0x7f34c0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.305+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f34c4058a80 con 0x7f34d0105bf0 2024-06-03T01:29:42.459 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:42 smithi032 ceph-mon[111919]: pgmap v162: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:42.459 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:42 smithi032 ceph-mon[111919]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.607+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f34d0102ce0 con 0x7f34d0105bf0 2024-06-03T01:29:42.609 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:42 smithi044 ceph-mon[84214]: pgmap v162: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:42.609 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:42 smithi044 ceph-mon[84214]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:42.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.608+0000 7f34b7fff700 1 -- 172.21.15.32:0/871143087 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+787 (secure 0 0 0) 0x7f34c4022720 con 0x7f34d0105bf0 2024-06-03T01:29:42.609 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:29:42.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:29:42.609 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:29:42.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 5 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-06-03T01:29:42.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 9 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 msgr2=0x7f34b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 0x7f34b80706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34c0006fd0 tx=0x7f34c0008040 comp rx=0 tx=0).stop 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 msgr2=0x7f34d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c400be20 comp rx=0 tx=0).stop 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 shutdown_connections 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f34b806e200 0x7f34b80706c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d00ffb40 0x7f34d01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 --2- 172.21.15.32:0/871143087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d0105bf0 0x7f34d0101cf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 >> 172.21.15.32:0/871143087 conn(0x7f34d00fb650 msgr2=0x7f34d0068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 shutdown_connections 2024-06-03T01:29:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.610+0000 7f34d563b700 1 -- 172.21.15.32:0/871143087 wait complete. 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 -- 172.21.15.32:0/3252800251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 msgr2=0x7f966c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 --2- 172.21.15.32:0/3252800251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c101280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9668009910 tx=0x7f9668009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 -- 172.21.15.32:0/3252800251 shutdown_connections 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 --2- 172.21.15.32:0/3252800251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 0x7f966c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 --2- 172.21.15.32:0/3252800251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c101280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 -- 172.21.15.32:0/3252800251 >> 172.21.15.32:0/3252800251 conn(0x7f966c0fc750 msgr2=0x7f966c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 -- 172.21.15.32:0/3252800251 shutdown_connections 2024-06-03T01:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.718+0000 7f9673393700 1 -- 172.21.15.32:0/3252800251 wait complete. 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 Processor -- start 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 -- start start 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 0x7f966c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19f540 con 0x7f966c100ea0 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f9673393700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c1989f0 con 0x7f966c101850 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56716/0 (socket says 172.21.15.32:56716) 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 -- 172.21.15.32:0/2918129896 learned_addr learned my addr 172.21.15.32:0/2918129896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967092e700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 0x7f966c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 -- 172.21.15.32:0/2918129896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 msgr2=0x7f966c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 0x7f966c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.719+0000 7f967112f700 1 -- 172.21.15.32:0/2918129896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680095f0 con 0x7f966c100ea0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f967112f700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9668006010 tx=0x7f966800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f966801d070 con 0x7f966c100ea0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9668004fb0 con 0x7f966c100ea0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9668022ea0 con 0x7f966c100ea0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c198c70 con 0x7f966c100ea0 2024-06-03T01:29:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.720+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c199160 con 0x7f966c100ea0 2024-06-03T01:29:42.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.721+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c105020 con 0x7f966c100ea0 2024-06-03T01:29:42.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.722+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9668022470 con 0x7f966c100ea0 2024-06-03T01:29:42.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.722+0000 7f96627fc700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:42.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.722+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f966808f770 con 0x7f966c100ea0 2024-06-03T01:29:42.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.722+0000 7f967092e700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:42.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.723+0000 7f967092e700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 0x7f96580706c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f965c006fd0 tx=0x7f965c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.727+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f966805c5d0 con 0x7f966c100ea0 2024-06-03T01:29:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.978+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f966c04f2e0 con 0x7f966c100ea0 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.979+0000 7f96627fc700 1 -- 172.21.15.32:0/2918129896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f9668022720 con 0x7f966c100ea0 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:29:42.980 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:24:42.122740+0000 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:29:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14478,1=14498} 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:14478} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6826/1879817357,v1:172.21.15.32:6827/1879817357] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:42.982 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:14498} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.32:6828/3721991653,v1:172.21.15.32:6829/3721991653] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:29:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 msgr2=0x7f96580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 0x7f96580706c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f965c006fd0 tx=0x7f965c008040 comp rx=0 tx=0).stop 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 msgr2=0x7f966c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9668006010 tx=0x7f966800be20 comp rx=0 tx=0).stop 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 shutdown_connections 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f966c100ea0 0x7f966c19e970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 --2- 172.21.15.32:0/2918129896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f966c101850 0x7f966c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 >> 172.21.15.32:0/2918129896 conn(0x7f966c0fc750 msgr2=0x7f966c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 shutdown_connections 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:42.981+0000 7f9673393700 1 -- 172.21.15.32:0/2918129896 wait complete. 2024-06-03T01:29:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 -- 172.21.15.32:0/1207121355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 msgr2=0x7f5960103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 --2- 172.21.15.32:0/1207121355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f5960103f10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5954009910 tx=0x7f5954009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 -- 172.21.15.32:0/1207121355 shutdown_connections 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 --2- 172.21.15.32:0/1207121355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f5960103f10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 --2- 172.21.15.32:0/1207121355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 0x7f59600ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 -- 172.21.15.32:0/1207121355 >> 172.21.15.32:0/1207121355 conn(0x7f5960074e30 msgr2=0x7f5960075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 -- 172.21.15.32:0/1207121355 shutdown_connections 2024-06-03T01:29:43.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 -- 172.21.15.32:0/1207121355 wait complete. 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.091+0000 7f59665d3700 1 Processor -- start 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f59665d3700 1 -- start start 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f59665d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 0x7f596019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f59665d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f59665d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596019d1f0 con 0x7f5960100020 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f59665d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59601966a0 con 0x7f59600ff670 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56728/0 (socket says 172.21.15.32:56728) 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 -- 172.21.15.32:0/3854315628 learned_addr learned my addr 172.21.15.32:0/3854315628 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595ffff700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 0x7f596019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 -- 172.21.15.32:0/3854315628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 msgr2=0x7f596019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 0x7f596019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 -- 172.21.15.32:0/3854315628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540095f0 con 0x7f5960100020 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.092+0000 7f595f7fe700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f595400bda0 tx=0x7f595400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.093+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f595401d070 con 0x7f5960100020 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.093+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f595400f460 con 0x7f5960100020 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.093+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5960196920 con 0x7f5960100020 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.093+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5954017610 con 0x7f5960100020 2024-06-03T01:29:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.093+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5960196e10 con 0x7f5960100020 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.094+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5960109920 con 0x7f5960100020 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.094+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f595400fa40 con 0x7f5960100020 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.095+0000 7f595d7fa700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 0x7f59480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.095+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f595408f650 con 0x7f5960100020 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.095+0000 7f595ffff700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 0x7f59480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.095+0000 7f595ffff700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 0x7f59480706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5950005950 tx=0x7f59500058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.099+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f595405c3e0 con 0x7f5960100020 2024-06-03T01:29:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.327+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5960197080 con 0x7f594806e200 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.328+0000 7f595d7fa700 1 -- 172.21.15.32:0/3854315628 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5960197080 con 0x7f594806e200 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:29:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "11/23 daemons upgraded", 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:29:43.330 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:29:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.330+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 msgr2=0x7f59480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.330+0000 7f59665d3700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 0x7f59480706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5950005950 tx=0x7f59500058e0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.330+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 msgr2=0x7f596019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.330+0000 7f59665d3700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f595400bda0 tx=0x7f595400be80 comp rx=0 tx=0).stop 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 shutdown_connections 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f594806e200 0x7f59480706c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59600ff670 0x7f596019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 --2- 172.21.15.32:0/3854315628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5960100020 0x7f596019cb60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 >> 172.21.15.32:0/3854315628 conn(0x7f5960074e30 msgr2=0x7f59600fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 shutdown_connections 2024-06-03T01:29:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.331+0000 7f59665d3700 1 -- 172.21.15.32:0/3854315628 wait complete. 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 -- 172.21.15.32:0/2536514590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4100ea0 msgr2=0x7f04f4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 --2- 172.21.15.32:0/2536514590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4100ea0 0x7f04f4101280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009910 tx=0x7f04e4009c20 comp rx=0 tx=0).stop 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 -- 172.21.15.32:0/2536514590 shutdown_connections 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 --2- 172.21.15.32:0/2536514590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4101850 0x7f04f41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 --2- 172.21.15.32:0/2536514590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4100ea0 0x7f04f4101280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 -- 172.21.15.32:0/2536514590 >> 172.21.15.32:0/2536514590 conn(0x7f04f40fc750 msgr2=0x7f04f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 -- 172.21.15.32:0/2536514590 shutdown_connections 2024-06-03T01:29:43.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.440+0000 7f04f8df4700 1 -- 172.21.15.32:0/2536514590 wait complete. 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 Processor -- start 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 -- start start 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4100ea0 0x7f04f419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f419d230 con 0x7f04f4101850 2024-06-03T01:29:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f8df4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4196770 con 0x7f04f4100ea0 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56754/0 (socket says 172.21.15.32:56754) 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.441+0000 7f04f1d9b700 1 -- 172.21.15.32:0/1064552045 learned_addr learned my addr 172.21.15.32:0/1064552045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f259c700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4100ea0 0x7f04f419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f1d9b700 1 -- 172.21.15.32:0/1064552045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4100ea0 msgr2=0x7f04f419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f1d9b700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4100ea0 0x7f04f419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f1d9b700 1 -- 172.21.15.32:0/1064552045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e40095f0 con 0x7f04f4101850 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f1d9b700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f04e800c960 tx=0x7f04e800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e8007a70 con 0x7f04f4101850 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04e8007bd0 con 0x7f04f4101850 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f4196a50 con 0x7f04f4101850 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e80186c0 con 0x7f04f4101850 2024-06-03T01:29:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.442+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f4196fa0 con 0x7f04f4101850 2024-06-03T01:29:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.443+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4109240 con 0x7f04f4101850 2024-06-03T01:29:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.444+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f04e800faa0 con 0x7f04f4101850 2024-06-03T01:29:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.444+0000 7f04e37fe700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 0x7f04dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:29:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.444+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f04e808e130 con 0x7f04f4101850 2024-06-03T01:29:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.444+0000 7f04f259c700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 0x7f04dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:29:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.445+0000 7f04f259c700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 0x7f04dc0706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e4005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:29:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.449+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f04e805acd0 con 0x7f04f4101850 2024-06-03T01:29:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:43 smithi032 ceph-mon[111919]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:43 smithi032 ceph-mon[111919]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:43 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/871143087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:43.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:43 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2918129896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:29:43.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:43 smithi044 ceph-mon[84214]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:43.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:43 smithi044 ceph-mon[84214]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:43.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:43 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/871143087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:29:43.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:43 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2918129896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:29:43.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.743+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f04f404f2e0 con 0x7f04f4101850 2024-06-03T01:29:43.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.743+0000 7f04e37fe700 1 -- 172.21.15.32:0/1064552045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f04e805aaf0 con 0x7f04f4101850 2024-06-03T01:29:43.744 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:29:43.745 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:29:43.745 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 msgr2=0x7f04dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 0x7f04dc0706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e4005fd0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 msgr2=0x7f04f419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f04e800c960 tx=0x7f04e800cd20 comp rx=0 tx=0).stop 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 shutdown_connections 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f04dc06e200 0x7f04dc0706c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f4100ea0 0x7f04f419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 --2- 172.21.15.32:0/1064552045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f4101850 0x7f04f419cc30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:29:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 >> 172.21.15.32:0/1064552045 conn(0x7f04f40fc750 msgr2=0x7f04f40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:29:43.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 shutdown_connections 2024-06-03T01:29:43.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:29:43.746+0000 7f04f8df4700 1 -- 172.21.15.32:0/1064552045 wait complete. 2024-06-03T01:29:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:44 smithi044 ceph-mon[84214]: pgmap v163: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:44 smithi044 ceph-mon[84214]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:44.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:44 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1064552045' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:29:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:44 smithi032 ceph-mon[111919]: pgmap v163: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:44 smithi032 ceph-mon[111919]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:29:44.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:44 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1064552045' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:29:46.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:46 smithi044 ceph-mon[84214]: pgmap v164: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:46.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:46 smithi032 ceph-mon[111919]: pgmap v164: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:48.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:48 smithi032 ceph-mon[111919]: pgmap v165: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:48.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:48 smithi044 ceph-mon[84214]: pgmap v165: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:49 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:49 smithi032 ceph-mon[111919]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:49.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:49 smithi032 ceph-mon[111919]: Upgrade: osd.5 is safe to restart 2024-06-03T01:29:49.370 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:49 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:49.394 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:49 smithi044 ceph-mon[84214]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-03T01:29:49.394 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:49 smithi044 ceph-mon[84214]: Upgrade: osd.5 is safe to restart 2024-06-03T01:29:50.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: pgmap v166: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:50.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: Upgrade: Updating osd.5 2024-06-03T01:29:50.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:50.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:29:50.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:50.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:50 smithi044 ceph-mon[84214]: Deploying daemon osd.5 on smithi044 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: pgmap v166: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: Upgrade: Updating osd.5 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:29:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:50 smithi032 ceph-mon[111919]: Deploying daemon osd.5 on smithi044 2024-06-03T01:29:52.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:52 smithi032 ceph-mon[111919]: pgmap v167: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:52.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:52.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:52.943 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:52 smithi044 ceph-mon[84214]: pgmap v167: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:52.943 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:52.943 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:29:53.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:52 smithi044 systemd[1]: Stopping Ceph osd.5 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:29:53.721 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:53 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:29:53.349+0000 7fc430607700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:29:53.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:53 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:29:53.349+0000 7fc430607700 -1 osd.5 86 *** Got signal Terminated *** 2024-06-03T01:29:53.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:53 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[56312]: 2024-06-03T01:29:53.349+0000 7fc430607700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:29:54.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:53 smithi032 ceph-mon[111919]: osd.5 marked itself down and dead 2024-06-03T01:29:54.189 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:53 smithi044 ceph-mon[84214]: osd.5 marked itself down and dead 2024-06-03T01:29:54.472 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:54 smithi044 podman[103214]: 2024-06-03 01:29:54.19035984 +0000 UTC m=+1.088763706 container died 454878053f65b5a7eb6d2ff90acd5a3d2e1d6f821a9fa90579b626f803ded68e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-06-03T01:29:54.773 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:54 smithi044 podman[103214]: 2024-06-03 01:29:54.507502978 +0000 UTC m=+1.405906832 container cleanup 454878053f65b5a7eb6d2ff90acd5a3d2e1d6f821a9fa90579b626f803ded68e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.29.1) 2024-06-03T01:29:54.773 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:54 smithi044 bash[103214]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5 2024-06-03T01:29:54.774 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:54 smithi044 podman[103233]: 2024-06-03 01:29:54.700538525 +0000 UTC m=+0.506556303 container remove 454878053f65b5a7eb6d2ff90acd5a3d2e1d6f821a9fa90579b626f803ded68e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:29:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:54 smithi032 ceph-mon[111919]: pgmap v168: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:54 smithi032 ceph-mon[111919]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:29:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:54 smithi032 ceph-mon[111919]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-06-03T01:29:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:54 smithi032 ceph-mon[111919]: osdmap e87: 6 total, 5 up, 6 in 2024-06-03T01:29:55.166 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:54 smithi044 ceph-mon[84214]: pgmap v168: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:55.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:54 smithi044 ceph-mon[84214]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-03T01:29:55.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:54 smithi044 ceph-mon[84214]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-06-03T01:29:55.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:54 smithi044 ceph-mon[84214]: osdmap e87: 6 total, 5 up, 6 in 2024-06-03T01:29:55.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.070372126 +0000 UTC m=+0.018151623 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:55.472 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.168005722 +0000 UTC m=+0.115785202 container create cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:29:55.472 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.404468395 +0000 UTC m=+0.352247884 container init cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-03T01:29:55.472 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.407864187 +0000 UTC m=+0.355643665 container start cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:29:55.782 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.487895104 +0000 UTC m=+0.435674590 container attach cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:29:55.782 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:55 smithi044 podman[103320]: 2024-06-03 01:29:55.716720354 +0000 UTC m=+0.664499836 container died cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T01:29:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:55 smithi032 ceph-mon[111919]: osdmap e88: 6 total, 5 up, 6 in 2024-06-03T01:29:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:55 smithi044 ceph-mon[84214]: osdmap e88: 6 total, 5 up, 6 in 2024-06-03T01:29:56.709 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 podman[103346]: 2024-06-03 01:29:56.258292398 +0000 UTC m=+0.534955963 container remove cb8d55823cab0c4c4085d5b49a382f143862afe0f0b7f9e1bef010d86e21fa7b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-03T01:29:56.709 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-06-03T01:29:56.709 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 systemd[1]: Stopped Ceph osd.5 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:29:56.710 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.5.service: Consumed 1min 5.732s CPU time. 2024-06-03T01:29:56.710 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 systemd[1]: Starting Ceph osd.5 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:29:56.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:56 smithi044 ceph-mon[84214]: pgmap v171: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:56.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:56 smithi044 ceph-mon[84214]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-06-03T01:29:56.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 podman[103447]: 2024-06-03 01:29:56.709252404 +0000 UTC m=+0.115690615 container create 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T01:29:56.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 podman[103447]: 2024-06-03 01:29:56.612022242 +0000 UTC m=+0.018460475 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:56.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 podman[103447]: 2024-06-03 01:29:56.953614529 +0000 UTC m=+0.360052748 container init 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T01:29:56.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:56 smithi044 podman[103447]: 2024-06-03 01:29:56.95701914 +0000 UTC m=+0.363457359 container start 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:29:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:56 smithi032 ceph-mon[111919]: pgmap v171: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:29:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:56 smithi032 ceph-mon[111919]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-06-03T01:29:57.426 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 podman[103447]: 2024-06-03 01:29:57.062155833 +0000 UTC m=+0.468594052 container attach 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-03T01:29:57.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 bash[103447]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate[103461]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-03T01:29:57.723 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 podman[103447]: 2024-06-03 01:29:57.451537144 +0000 UTC m=+0.857975356 container died 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:29:58.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:57 smithi032 ceph-mon[111919]: pgmap v172: 65 pgs: 8 active+undersized, 10 active+undersized+degraded, 3 peering, 3 stale+active+clean, 41 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/366 objects degraded (8.197%) 2024-06-03T01:29:58.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:57 smithi032 ceph-mon[111919]: Health check failed: Degraded data redundancy: 30/366 objects degraded (8.197%), 10 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:58.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-mon[84214]: pgmap v172: 65 pgs: 8 active+undersized, 10 active+undersized+degraded, 3 peering, 3 stale+active+clean, 41 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/366 objects degraded (8.197%) 2024-06-03T01:29:58.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:57 smithi044 ceph-mon[84214]: Health check failed: Degraded data redundancy: 30/366 objects degraded (8.197%), 10 pgs degraded (PG_DEGRADED) 2024-06-03T01:29:58.207 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:57 smithi044 podman[103596]: 2024-06-03 01:29:57.991419044 +0000 UTC m=+0.529499227 container remove 799335cebb54f7e34e23f1f996073bdbd3a27efb9f58a6d006c930067f422e06 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T01:29:58.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 podman[103659]: 2024-06-03 01:29:58.110652239 +0000 UTC m=+0.018064033 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:29:58.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 podman[103659]: 2024-06-03 01:29:58.267646916 +0000 UTC m=+0.175058696 container create a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-03T01:29:58.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 podman[103659]: 2024-06-03 01:29:58.511764804 +0000 UTC m=+0.419176585 container init a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-03T01:29:58.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 podman[103659]: 2024-06-03 01:29:58.515189553 +0000 UTC m=+0.422601338 container start a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:29:58.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 ceph-osd[103677]: -- 172.21.15.44:0/2904080703 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a95cac7680 con 0x55a95ca90c00 2024-06-03T01:29:58.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 bash[103659]: a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 2024-06-03T01:29:58.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:58 smithi044 systemd[1]: Started Ceph osd.5 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:29:59.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:29:59 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:29:59.071+0000 7efea2a5f7c0 -1 Falling back to public interface 2024-06-03T01:29:59.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:59.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:29:59.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:29:59 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:00.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:29:59 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:00.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pgmap v173: 65 pgs: 12 active+undersized, 15 active+undersized+degraded, 3 peering, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:00.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Reduced data availability: 2 pgs peering; Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded 2024-06-03T01:30:00.922 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: [WRN] OSD_DOWN: 1 osds down 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: osd.5 (root=default,host=smithi044) is down 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: [WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: all OSDs are running squid or later but require_osd_release < squid 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: [WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 3.c is stuck peering for 7m, current state peering, last acting [0,1] 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 3.10 is stuck peering for 7m, current state peering, last acting [0,1] 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: [WRN] PG_DEGRADED: Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-06-03T01:30:00.923 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.8 is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.b is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.d is active+undersized+degraded, acting [1,2] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.f is active+undersized+degraded, acting [4,0] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-06-03T01:30:00.924 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:00 smithi044 ceph-mon[84214]: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-06-03T01:30:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pgmap v173: 65 pgs: 12 active+undersized, 15 active+undersized+degraded, 3 peering, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Reduced data availability: 2 pgs peering; Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded 2024-06-03T01:30:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: [WRN] OSD_DOWN: 1 osds down 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: osd.5 (root=default,host=smithi044) is down 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: [WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: all OSDs are running squid or later but require_osd_release < squid 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: [WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 3.c is stuck peering for 7m, current state peering, last acting [0,1] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 3.10 is stuck peering for 7m, current state peering, last acting [0,1] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: [WRN] PG_DEGRADED: Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.8 is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.b is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:01.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.d is active+undersized+degraded, acting [1,2] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.f is active+undersized+degraded, acting [4,0] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-06-03T01:30:01.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:00 smithi032 ceph-mon[111919]: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-06-03T01:30:01.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:01 smithi044 ceph-mon[84214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-06-03T01:30:02.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:01 smithi032 ceph-mon[111919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-06-03T01:30:03.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:02 smithi032 ceph-mon[111919]: pgmap v174: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:03.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:02 smithi044 ceph-mon[84214]: pgmap v174: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:03.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:03.843 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:30:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:30:03.542+0000 7efea2a5f7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-06-03T01:30:04.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:30:04 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:30:04.000+0000 7efea2a5f7c0 -1 osd.5 86 log_to_monitors true 2024-06-03T01:30:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:03 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:04.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:03 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:04.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:03 smithi044 ceph-mon[84214]: pgmap v175: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:03 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:03 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:04.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:03 smithi032 ceph-mon[111919]: pgmap v175: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:04 smithi044 ceph-mon[84214]: from='osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:30:05.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:04 smithi044 ceph-mon[84214]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:30:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:04 smithi032 ceph-mon[111919]: from='osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:30:05.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:04 smithi032 ceph-mon[111919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:30:06.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:30:05.908+0000 7efe8feae700 -1 osd.5 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: osdmap e89: 6 total, 5 up, 6 in 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: from='osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: pgmap v177: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:06.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:05 smithi044 ceph-mon[84214]: from='osd.5 ' entity='osd.5' 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: osdmap e89: 6 total, 5 up, 6 in 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: from='osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: pgmap v177: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:06.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:05 smithi032 ceph-mon[111919]: from='osd.5 ' entity='osd.5' 2024-06-03T01:30:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:06 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[111915]: 2024-06-03T01:30:06.909+0000 7fd35226c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all osd 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-03T01:30:07.837 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Upgrade: Setting require_osd_release to 19 squid 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded (PG_DEGRADED) 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832] boot 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: osdmap e90: 6 total, 6 up, 6 in 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:30:07.838 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:30:07.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T01:30:07.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: osdmap e91: 6 total, 6 up, 6 in 2024-06-03T01:30:07.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-06-03T01:30:07.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.839 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.885 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all osd 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-03T01:30:07.886 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Upgrade: Setting require_osd_release to 19 squid 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 58/366 objects degraded (15.847%), 15 pgs degraded (PG_DEGRADED) 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: osd.5 [v2:172.21.15.44:6816/3634012832,v1:172.21.15.44:6817/3634012832] boot 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: osdmap e90: 6 total, 6 up, 6 in 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:30:07.887 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:30:07.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T01:30:07.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: osdmap e91: 6 total, 6 up, 6 in 2024-06-03T01:30:07.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-06-03T01:30:07.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:07.888 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:08.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-06-03T01:30:08.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: pgmap v180: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: osdmap e92: 6 total, 6 up, 6 in 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/1054828690,v1:172.21.15.32:6827/1054828690] up:boot 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/4125418426,v1:172.21.15.32:6829/4125418426] up:boot 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:30:08.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:08 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: pgmap v180: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: osdmap e92: 6 total, 6 up, 6 in 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/1054828690,v1:172.21.15.32:6827/1054828690] up:boot 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/4125418426,v1:172.21.15.32:6829/4125418426] up:boot 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:30:08.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:08 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:09 smithi044 ceph-mon[84214]: Upgrade: Updating mds.cephfs.smithi032.itqzrm 2024-06-03T01:30:09.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:09 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:09.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:09 smithi044 ceph-mon[84214]: Deploying daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:30:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:09 smithi032 ceph-mon[111919]: Upgrade: Updating mds.cephfs.smithi032.itqzrm 2024-06-03T01:30:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.itqzrm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:09 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:10.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:09 smithi032 ceph-mon[111919]: Deploying daemon mds.cephfs.smithi032.itqzrm on smithi032 2024-06-03T01:30:10.802 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:10 smithi032 ceph-mon[111919]: pgmap v182: 65 pgs: 13 active+undersized, 11 active+undersized+degraded, 41 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/366 objects degraded (9.290%) 2024-06-03T01:30:10.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:10 smithi044 ceph-mon[84214]: pgmap v182: 65 pgs: 13 active+undersized, 11 active+undersized+degraded, 41 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/366 objects degraded (9.290%) 2024-06-03T01:30:12.956 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:12 smithi044 ceph-mon[84214]: pgmap v183: 65 pgs: 4 active+undersized, 6 active+undersized+degraded, 55 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-06-03T01:30:12.956 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:12 smithi044 ceph-mon[84214]: osdmap e93: 6 total, 6 up, 6 in 2024-06-03T01:30:12.956 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:12 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:12.957 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:12 smithi044 ceph-mon[84214]: Health check update: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded (PG_DEGRADED) 2024-06-03T01:30:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:12 smithi032 ceph-mon[111919]: pgmap v183: 65 pgs: 4 active+undersized, 6 active+undersized+degraded, 55 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-06-03T01:30:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:12 smithi032 ceph-mon[111919]: osdmap e93: 6 total, 6 up, 6 in 2024-06-03T01:30:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:12 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:13.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:12 smithi032 ceph-mon[111919]: Health check update: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded (PG_DEGRADED) 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 -- 172.21.15.32:0/4263556675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 msgr2=0x7f606810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 --2- 172.21.15.32:0/4263556675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606810ac10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f605c009910 tx=0x7f605c009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 -- 172.21.15.32:0/4263556675 shutdown_connections 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 --2- 172.21.15.32:0/4263556675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606810ac10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 --2- 172.21.15.32:0/4263556675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 0x7f60681021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.871+0000 7f606cab7700 1 -- 172.21.15.32:0/4263556675 >> 172.21.15.32:0/4263556675 conn(0x7f60680fb650 msgr2=0x7f60680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.872+0000 7f606cab7700 1 -- 172.21.15.32:0/4263556675 shutdown_connections 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.872+0000 7f606cab7700 1 -- 172.21.15.32:0/4263556675 wait complete. 2024-06-03T01:30:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.872+0000 7f606cab7700 1 Processor -- start 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.872+0000 7f606cab7700 1 -- start start 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f606cab7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 0x7f606819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f606cab7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f606cab7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606819d260 con 0x7f6068102720 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f606cab7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068196710 con 0x7f6068101e00 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46022/0 (socket says 172.21.15.32:46022) 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 -- 172.21.15.32:0/2239601476 learned_addr learned my addr 172.21.15.32:0/2239601476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f606659c700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 0x7f606819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 -- 172.21.15.32:0/2239601476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 msgr2=0x7f606819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 0x7f606819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 -- 172.21.15.32:0/2239601476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0095f0 con 0x7f6068102720 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.873+0000 7f6065d9b700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f605c00bd70 tx=0x7f605c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.874+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605c01d070 con 0x7f6068102720 2024-06-03T01:30:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.874+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f605c005dc0 con 0x7f6068102720 2024-06-03T01:30:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.874+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6068196990 con 0x7f6068102720 2024-06-03T01:30:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.874+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605c017400 con 0x7f6068102720 2024-06-03T01:30:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.874+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6068196e80 con 0x7f6068102720 2024-06-03T01:30:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.875+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6068108310 con 0x7f6068102720 2024-06-03T01:30:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.875+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f605c00f7b0 con 0x7f6068102720 2024-06-03T01:30:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.876+0000 7f60577fe700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 0x7f60500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.876+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f605c08f780 con 0x7f6068102720 2024-06-03T01:30:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.876+0000 7f606659c700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 0x7f60500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.876+0000 7f606659c700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 0x7f60500706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6058005950 tx=0x7f60580058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:13.881+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f605c05c480 con 0x7f6068102720 2024-06-03T01:30:13.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:13 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:14.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:13 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.107+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60681970a0 con 0x7f605006e200 2024-06-03T01:30:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.108+0000 7f60577fe700 1 -- 172.21.15.32:0/2239601476 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f60681970a0 con 0x7f605006e200 2024-06-03T01:30:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 msgr2=0x7f60500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 0x7f60500706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6058005950 tx=0x7f60580058e0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 msgr2=0x7f606819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f605c00bd70 tx=0x7f605c00be50 comp rx=0 tx=0).stop 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 shutdown_connections 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f605006e200 0x7f60500706c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6068101e00 0x7f606819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 --2- 172.21.15.32:0/2239601476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6068102720 0x7f606819cbd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 >> 172.21.15.32:0/2239601476 conn(0x7f60680fb650 msgr2=0x7f6068105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 shutdown_connections 2024-06-03T01:30:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.110+0000 7f606cab7700 1 -- 172.21.15.32:0/2239601476 wait complete. 2024-06-03T01:30:14.121 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:30:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 -- 172.21.15.32:0/3912104581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc101850 msgr2=0x7f30fc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 --2- 172.21.15.32:0/3912104581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc101850 0x7f30fc1058a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009910 tx=0x7f30f0009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 -- 172.21.15.32:0/3912104581 shutdown_connections 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 --2- 172.21.15.32:0/3912104581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc101850 0x7f30fc1058a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 --2- 172.21.15.32:0/3912104581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc100ea0 0x7f30fc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 -- 172.21.15.32:0/3912104581 >> 172.21.15.32:0/3912104581 conn(0x7f30fc0fc750 msgr2=0x7f30fc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 -- 172.21.15.32:0/3912104581 shutdown_connections 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 -- 172.21.15.32:0/3912104581 wait complete. 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.220+0000 7f3102477700 1 Processor -- start 2024-06-03T01:30:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f3102477700 1 -- start start 2024-06-03T01:30:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f3102477700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f3102477700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc101850 0x7f30fc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f3102477700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19d2d0 con 0x7f30fc100ea0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f3102477700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc196780 con 0x7f30fc101850 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46034/0 (socket says 172.21.15.32:46034) 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 -- 172.21.15.32:0/2834477840 learned_addr learned my addr 172.21.15.32:0/2834477840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fb7fe700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc101850 0x7f30fc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 -- 172.21.15.32:0/2834477840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc101850 msgr2=0x7f30fc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc101850 0x7f30fc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.221+0000 7f30fbfff700 1 -- 172.21.15.32:0/2834477840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00095f0 con 0x7f30fc100ea0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f30fbfff700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d8d0 tx=0x7f30ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec009940 con 0x7f30fc100ea0 2024-06-03T01:30:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc196a60 con 0x7f30fc100ea0 2024-06-03T01:30:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30ec010460 con 0x7f30fc100ea0 2024-06-03T01:30:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec00f5f0 con 0x7f30fc100ea0 2024-06-03T01:30:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.222+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc196fb0 con 0x7f30fc100ea0 2024-06-03T01:30:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.223+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc1092a0 con 0x7f30fc100ea0 2024-06-03T01:30:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.223+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f30ec0105d0 con 0x7f30fc100ea0 2024-06-03T01:30:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.224+0000 7f30f97fa700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 0x7f30e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.224+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f30ec08e2b0 con 0x7f30fc100ea0 2024-06-03T01:30:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.224+0000 7f30fb7fe700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 0x7f30e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.224+0000 7f30fb7fe700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 0x7f30e40706c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30fc197bf0 tx=0x7f30f001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.229+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30ec05afb0 con 0x7f30fc100ea0 2024-06-03T01:30:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.454+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30fc197da0 con 0x7f30e406e200 2024-06-03T01:30:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.455+0000 7f30f97fa700 1 -- 172.21.15.32:0/2834477840 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f30fc197da0 con 0x7f30e406e200 2024-06-03T01:30:14.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.457+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 msgr2=0x7f30e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.457+0000 7f3102477700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 0x7f30e40706c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30fc197bf0 tx=0x7f30f001a040 comp rx=0 tx=0).stop 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.457+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 msgr2=0x7f30fc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.457+0000 7f3102477700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d8d0 tx=0x7f30ec00dc90 comp rx=0 tx=0).stop 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 shutdown_connections 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f30e406e200 0x7f30e40706c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc100ea0 0x7f30fc19c700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 --2- 172.21.15.32:0/2834477840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30fc101850 0x7f30fc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 >> 172.21.15.32:0/2834477840 conn(0x7f30fc0fc750 msgr2=0x7f30fc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 shutdown_connections 2024-06-03T01:30:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.458+0000 7f3102477700 1 -- 172.21.15.32:0/2834477840 wait complete. 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.565+0000 7f77bd76a700 1 -- 172.21.15.32:0/2262078312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 msgr2=0x7f77b80735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.565+0000 7f77bd76a700 1 --2- 172.21.15.32:0/2262078312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b80735a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009910 tx=0x7f77a8009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.565+0000 7f77bd76a700 1 -- 172.21.15.32:0/2262078312 shutdown_connections 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.565+0000 7f77bd76a700 1 --2- 172.21.15.32:0/2262078312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 0x7f77b810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.565+0000 7f77bd76a700 1 --2- 172.21.15.32:0/2262078312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b80735a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.566+0000 7f77bd76a700 1 -- 172.21.15.32:0/2262078312 >> 172.21.15.32:0/2262078312 conn(0x7f77b80fc750 msgr2=0x7f77b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.566+0000 7f77bd76a700 1 -- 172.21.15.32:0/2262078312 shutdown_connections 2024-06-03T01:30:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.566+0000 7f77bd76a700 1 -- 172.21.15.32:0/2262078312 wait complete. 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.566+0000 7f77bd76a700 1 Processor -- start 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.566+0000 7f77bd76a700 1 -- start start 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77bd76a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77bd76a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 0x7f77b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77bd76a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b819f400 con 0x7f77b80731c0 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77bd76a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8198900 con 0x7f77b8073ae0 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46054/0 (socket says 172.21.15.32:46054) 2024-06-03T01:30:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 -- 172.21.15.32:0/553935370 learned_addr learned my addr 172.21.15.32:0/553935370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b67fc700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 0x7f77b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 -- 172.21.15.32:0/553935370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 msgr2=0x7f77b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 0x7f77b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 -- 172.21.15.32:0/553935370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80095f0 con 0x7f77b80731c0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f77b6ffd700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77a800bd70 tx=0x7f77a800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.567+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a801d070 con 0x7f77b80731c0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.568+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77a8005dc0 con 0x7f77b80731c0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.568+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b8198b80 con 0x7f77b80731c0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.568+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a8017400 con 0x7f77b80731c0 2024-06-03T01:30:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.568+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b8199070 con 0x7f77b80731c0 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.569+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b810a640 con 0x7f77b80731c0 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.569+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f77a800f460 con 0x7f77b80731c0 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.570+0000 7f779ffff700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 0x7f77a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.570+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f77a808f710 con 0x7f77b80731c0 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.570+0000 7f77b67fc700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 0x7f77a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.570+0000 7f77b67fc700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 0x7f77a00706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77b80fdee0 tx=0x7f77ac00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.574+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f77a805c4d0 con 0x7f77b80731c0 2024-06-03T01:30:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.791+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f77b8199290 con 0x7f77a006e200 2024-06-03T01:30:14.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:14 smithi032 ceph-mon[111919]: pgmap v185: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:14.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:14 smithi032 ceph-mon[111919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded) 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.802+0000 7f779ffff700 1 -- 172.21.15.32:0/553935370 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f77b8199290 con 0x7f77a006e200 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 114s ago 17m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (17m) 114s ago 17m 8656k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (16m) 13s ago 16m 8446k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:30:14.802 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (3m) 114s ago 17m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (3m) 13s ago 16m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 114s ago 17m 81.4M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (11m) 114s ago 11m 124M - 18.2.1 5be31c24972a 183890c2658d 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (11m) 114s ago 11m 122M - 18.2.1 5be31c24972a 5fa1834f2899 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (11m) 13s ago 11m 17.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (11m) 13s ago 11m 19.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (8m) 114s ago 18m 505M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (7m) 13s ago 16m 452M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (4m) 114s ago 19m 49.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (4m) 13s ago 16m 41.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 114s ago 17m 15.3M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:30:14.803 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 13s ago 16m 15.8M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (3m) 114s ago 15m 138M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (2m) 114s ago 15m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (119s) 114s ago 14m 12.1M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (84s) 13s ago 14m 102M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (50s) 13s ago 13m 90.8M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (16s) 13s ago 13m 12.0M 2241M 19.0.0-2946-g3d89accf b045ac545f82 a11e0341a922 2024-06-03T01:30:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 114s ago 16m 78.3M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 msgr2=0x7f77a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 0x7f77a00706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77b80fdee0 tx=0x7f77ac00b410 comp rx=0 tx=0).stop 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 msgr2=0x7f77b819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77a800bd70 tx=0x7f77a800bda0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 shutdown_connections 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f77a006e200 0x7f77a00706c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77b80731c0 0x7f77b819e830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 --2- 172.21.15.32:0/553935370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b8073ae0 0x7f77b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 >> 172.21.15.32:0/553935370 conn(0x7f77b80fc750 msgr2=0x7f77b8107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 shutdown_connections 2024-06-03T01:30:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.804+0000 7f77bd76a700 1 -- 172.21.15.32:0/553935370 wait complete. 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.913+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/1497781218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 msgr2=0x7f2bb4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.913+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/1497781218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb4100180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009910 tx=0x7f2ba4009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/1497781218 shutdown_connections 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/1497781218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/1497781218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb4100180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/1497781218 >> 172.21.15.32:0/1497781218 conn(0x7f2bb40fb650 msgr2=0x7f2bb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/1497781218 shutdown_connections 2024-06-03T01:30:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/1497781218 wait complete. 2024-06-03T01:30:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 Processor -- start 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.914+0000 7f2bb8b1d700 1 -- start start 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb8b1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb8b1d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb8b1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb419d2c0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb8b1d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4196770 con 0x7f2bb4100750 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46076/0 (socket says 172.21.15.32:46076) 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 -- 172.21.15.32:0/2777445104 learned_addr learned my addr 172.21.15.32:0/2777445104 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2babfff700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 -- 172.21.15.32:0/2777445104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 msgr2=0x7f2bb419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 -- 172.21.15.32:0/2777445104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40095f0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2babfff700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.915+0000 7f2bb259c700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4006010 tx=0x7f2ba400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.916+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba401d070 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.916+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ba4004fb0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.916+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb41969f0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.916+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb4196ee0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.916+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba4022ea0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.917+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb404f2e0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.917+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ba4022470 con 0x7f2bb40ffda0 2024-06-03T01:30:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.918+0000 7f2bab7fe700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 0x7f2ba00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.918+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f2ba408f820 con 0x7f2bb40ffda0 2024-06-03T01:30:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.918+0000 7f2babfff700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 0x7f2ba00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.918+0000 7f2babfff700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 0x7f2ba00706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4197be0 tx=0x7f2b9c005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:14.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:14.923+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ba405c5d0 con 0x7f2bb40ffda0 2024-06-03T01:30:14.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:14 smithi044 ceph-mon[84214]: pgmap v185: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:14.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:14 smithi044 ceph-mon[84214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded) 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.218+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2bb4197210 con 0x7f2bb40ffda0 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.219+0000 7f2bab7fe700 1 -- 172.21.15.32:0/2777445104 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+697 (secure 0 0 0) 0x7f2ba4022720 con 0x7f2bb40ffda0 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:30:15.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:30:15.220 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-06-03T01:30:15.221 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 10 2024-06-03T01:30:15.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:30:15.221 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:30:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 msgr2=0x7f2ba00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 0x7f2ba00706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4197be0 tx=0x7f2b9c005f50 comp rx=0 tx=0).stop 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 msgr2=0x7f2bb419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4006010 tx=0x7f2ba400be20 comp rx=0 tx=0).stop 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 shutdown_connections 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2ba006e200 0x7f2ba00706c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bb40ffda0 0x7f2bb419c6f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 --2- 172.21.15.32:0/2777445104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb4100750 0x7f2bb419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 >> 172.21.15.32:0/2777445104 conn(0x7f2bb40fb650 msgr2=0x7f2bb4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 shutdown_connections 2024-06-03T01:30:15.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.221+0000 7f2bb8b1d700 1 -- 172.21.15.32:0/2777445104 wait complete. 2024-06-03T01:30:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3749245178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba84100750 msgr2=0x7fba841047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3749245178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba84100750 0x7fba841047a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba74009910 tx=0x7fba74009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3749245178 shutdown_connections 2024-06-03T01:30:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3749245178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba84100750 0x7fba841047a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3749245178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba840ffda0 0x7fba84100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3749245178 >> 172.21.15.32:0/3749245178 conn(0x7fba840fb650 msgr2=0x7fba840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3749245178 shutdown_connections 2024-06-03T01:30:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.335+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3749245178 wait complete. 2024-06-03T01:30:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 Processor -- start 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 -- start start 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 0x7fba8419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419d240 con 0x7fba840ffda0 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.336+0000 7fba8b8d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba841966f0 con 0x7fba84100750 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba8966c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba88e6b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 0x7fba8419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba8966c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46082/0 (socket says 172.21.15.32:46082) 2024-06-03T01:30:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba8966c700 1 -- 172.21.15.32:0/3004772347 learned_addr learned my addr 172.21.15.32:0/3004772347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba88e6b700 1 -- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 msgr2=0x7fba8419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba88e6b700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba88e6b700 1 -- 172.21.15.32:0/3004772347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740095f0 con 0x7fba84100750 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.337+0000 7fba8966c700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba88e6b700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 0x7fba8419cbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba74006010 tx=0x7fba7400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7401d070 con 0x7fba84100750 2024-06-03T01:30:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba7400b610 con 0x7fba84100750 2024-06-03T01:30:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba84196970 con 0x7fba84100750 2024-06-03T01:30:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba84196e30 con 0x7fba84100750 2024-06-03T01:30:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.338+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba74022470 con 0x7fba84100750 2024-06-03T01:30:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.339+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba68005320 con 0x7fba84100750 2024-06-03T01:30:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.340+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fba740050d0 con 0x7fba84100750 2024-06-03T01:30:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.341+0000 7fba7a7fc700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 0x7fba700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.341+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fba7408f700 con 0x7fba84100750 2024-06-03T01:30:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.341+0000 7fba8966c700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 0x7fba700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.341+0000 7fba8966c700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 0x7fba700706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba80011550 tx=0x7fba80009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.345+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fba7405c520 con 0x7fba84100750 2024-06-03T01:30:15.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.604+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba68005cc0 con 0x7fba84100750 2024-06-03T01:30:15.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.604+0000 7fba7a7fc700 1 -- 172.21.15.32:0/3004772347 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1692 (secure 0 0 0) 0x7fba7405c0b0 con 0x7fba84100750 2024-06-03T01:30:15.604 INFO:teuthology.orchestra.run.smithi032.stdout:e24 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 22 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:30:06.909705+0000 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:30:15.605 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 91 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:up {} 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:failed 0,1 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:30:15.606 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:24295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/3478864978,v1:172.21.15.44:6825/3478864978] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:30:15.607 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/1054828690,v1:172.21.15.32:6827/1054828690] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:30:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 msgr2=0x7fba700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 0x7fba700706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba80011550 tx=0x7fba80009260 comp rx=0 tx=0).stop 2024-06-03T01:30:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 msgr2=0x7fba8419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 0x7fba8419cbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba74006010 tx=0x7fba7400f690 comp rx=0 tx=0).stop 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 shutdown_connections 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fba7006e200 0x7fba700706c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba840ffda0 0x7fba8419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 --2- 172.21.15.32:0/3004772347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba84100750 0x7fba8419cbb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 >> 172.21.15.32:0/3004772347 conn(0x7fba840fb650 msgr2=0x7fba84069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 shutdown_connections 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.607+0000 7fba8b8d0700 1 -- 172.21.15.32:0/3004772347 wait complete. 2024-06-03T01:30:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.722+0000 7fa162d07700 1 -- 172.21.15.32:0/1341551771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c100320 msgr2=0x7fa15c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.722+0000 7fa162d07700 1 --2- 172.21.15.32:0/1341551771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c100320 0x7fa15c10cfe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa148009910 tx=0x7fa148009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 -- 172.21.15.32:0/1341551771 shutdown_connections 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 --2- 172.21.15.32:0/1341551771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c100320 0x7fa15c10cfe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 --2- 172.21.15.32:0/1341551771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0ff970 0x7fa15c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 -- 172.21.15.32:0/1341551771 >> 172.21.15.32:0/1341551771 conn(0x7fa15c0fb650 msgr2=0x7fa15c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 -- 172.21.15.32:0/1341551771 shutdown_connections 2024-06-03T01:30:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 -- 172.21.15.32:0/1341551771 wait complete. 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 Processor -- start 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.723+0000 7fa162d07700 1 -- start start 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa162d07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa162d07700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c100320 0x7fa15c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa162d07700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c19f3f0 con 0x7fa15c0ff970 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa162d07700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c1988a0 con 0x7fa15c100320 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46110/0 (socket says 172.21.15.32:46110) 2024-06-03T01:30:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 -- 172.21.15.32:0/2332527896 learned_addr learned my addr 172.21.15.32:0/2332527896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa15bfff700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c100320 0x7fa15c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 -- 172.21.15.32:0/2332527896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c100320 msgr2=0x7fa15c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c100320 0x7fa15c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 -- 172.21.15.32:0/2332527896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480095f0 con 0x7fa15c0ff970 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa160aa3700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d900 tx=0x7fa15000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1500041f0 con 0x7fa15c0ff970 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa150004cf0 con 0x7fa15c0ff970 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.724+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15000b750 con 0x7fa15c0ff970 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.725+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c198b80 con 0x7fa15c0ff970 2024-06-03T01:30:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.725+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c199080 con 0x7fa15c0ff970 2024-06-03T01:30:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.726+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c10a750 con 0x7fa15c0ff970 2024-06-03T01:30:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.726+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa150004750 con 0x7fa15c0ff970 2024-06-03T01:30:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.727+0000 7fa159ffb700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 0x7fa14c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.727+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fa15008e180 con 0x7fa15c0ff970 2024-06-03T01:30:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.727+0000 7fa15bfff700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 0x7fa14c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:15.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.728+0000 7fa15bfff700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 0x7fa14c0706c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0fbdb0 tx=0x7fa14800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:15.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.732+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa15005ae80 con 0x7fa15c0ff970 2024-06-03T01:30:15.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:15 smithi032 ceph-mon[111919]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:15 smithi032 ceph-mon[111919]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:15 smithi032 ceph-mon[111919]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:15 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2777445104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:15.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:15 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3004772347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:30:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.969+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa15c1992f0 con 0x7fa14c06e200 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.971+0000 7fa159ffb700 1 -- 172.21.15.32:0/2332527896 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa15c1992f0 con 0x7fa14c06e200 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:30:15.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "12/23 daemons upgraded", 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mds daemons", 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:30:15.972 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:30:15.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:15 smithi044 ceph-mon[84214]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:15 smithi044 ceph-mon[84214]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:15 smithi044 ceph-mon[84214]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:15 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2777445104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:15.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:15 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3004772347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:30:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 msgr2=0x7fa14c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 0x7fa14c0706c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0fbdb0 tx=0x7fa14800b540 comp rx=0 tx=0).stop 2024-06-03T01:30:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 msgr2=0x7fa15c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15000d900 tx=0x7fa15000dc10 comp rx=0 tx=0).stop 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 shutdown_connections 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa14c06e200 0x7fa14c0706c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15c0ff970 0x7fa15c19e820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 --2- 172.21.15.32:0/2332527896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c100320 0x7fa15c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.973+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 >> 172.21.15.32:0/2332527896 conn(0x7fa15c0fb650 msgr2=0x7fa15c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.974+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 shutdown_connections 2024-06-03T01:30:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:15.974+0000 7fa162d07700 1 -- 172.21.15.32:0/2332527896 wait complete. 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.093+0000 7fd043c63700 1 -- 172.21.15.32:0/2257093095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c106cd0 msgr2=0x7fd03c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.093+0000 7fd043c63700 1 --2- 172.21.15.32:0/2257093095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c106cd0 0x7fd03c1070b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd038009910 tx=0x7fd038009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 -- 172.21.15.32:0/2257093095 shutdown_connections 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 --2- 172.21.15.32:0/2257093095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c068580 0x7fd03c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 --2- 172.21.15.32:0/2257093095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c106cd0 0x7fd03c1070b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 -- 172.21.15.32:0/2257093095 >> 172.21.15.32:0/2257093095 conn(0x7fd03c075670 msgr2=0x7fd03c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 -- 172.21.15.32:0/2257093095 shutdown_connections 2024-06-03T01:30:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 -- 172.21.15.32:0/2257093095 wait complete. 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 Processor -- start 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.094+0000 7fd043c63700 1 -- start start 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd043c63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd043c63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c106cd0 0x7fd03c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd043c63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c197550 con 0x7fd03c068580 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd043c63700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c19b360 con 0x7fd03c106cd0 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46128/0 (socket says 172.21.15.32:46128) 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 -- 172.21.15.32:0/3636087603 learned_addr learned my addr 172.21.15.32:0/3636087603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0411fe700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c106cd0 0x7fd03c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 -- 172.21.15.32:0/3636087603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c106cd0 msgr2=0x7fd03c196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c106cd0 0x7fd03c196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 -- 172.21.15.32:0/3636087603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380095f0 con 0x7fd03c068580 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.095+0000 7fd0419ff700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd038006010 tx=0x7fd03800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.096+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03801d070 con 0x7fd03c068580 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.096+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd03800f460 con 0x7fd03c068580 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.096+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd038017610 con 0x7fd03c068580 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.096+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c19b5e0 con 0x7fd03c068580 2024-06-03T01:30:16.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.096+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c19bad0 con 0x7fd03c068580 2024-06-03T01:30:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.097+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c1091f0 con 0x7fd03c068580 2024-06-03T01:30:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.097+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd03800f5d0 con 0x7fd03c068580 2024-06-03T01:30:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.098+0000 7fd032ffd700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 0x7fd0280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.098+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fd03808f840 con 0x7fd03c068580 2024-06-03T01:30:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.098+0000 7fd0411fe700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 0x7fd0280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:16.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.098+0000 7fd0411fe700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 0x7fd0280706c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd03c197fa0 tx=0x7fd02c009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:16.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.103+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd03805c520 con 0x7fd03c068580 2024-06-03T01:30:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.402+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd03c197ce0 con 0x7fd03c068580 2024-06-03T01:30:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.402+0000 7fd032ffd700 1 -- 172.21.15.32:0/3636087603 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd03805c0b0 con 0x7fd03c068580 2024-06-03T01:30:16.402 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is degraded 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-03T01:30:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-03T01:30:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 msgr2=0x7fd0280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 0x7fd0280706c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd03c197fa0 tx=0x7fd02c009190 comp rx=0 tx=0).stop 2024-06-03T01:30:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 msgr2=0x7fd03c196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd038006010 tx=0x7fd03800bd70 comp rx=0 tx=0).stop 2024-06-03T01:30:16.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 shutdown_connections 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd02806e200 0x7fd0280706c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd03c068580 0x7fd03c196980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 --2- 172.21.15.32:0/3636087603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd03c106cd0 0x7fd03c196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 >> 172.21.15.32:0/3636087603 conn(0x7fd03c075670 msgr2=0x7fd03c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 shutdown_connections 2024-06-03T01:30:16.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:16.404+0000 7fd043c63700 1 -- 172.21.15.32:0/3636087603 wait complete. 2024-06-03T01:30:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:16 smithi032 ceph-mon[111919]: pgmap v186: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:16 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3636087603' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:30:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:16 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:16.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:16 smithi044 ceph-mon[84214]: pgmap v186: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:16.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:16 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3636087603' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:30:16.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:16.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:16.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:16 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:17.963 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:17 smithi032 ceph-mon[111919]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:17.964 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:17 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:boot 2024-06-03T01:30:17.964 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:17 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:17.964 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:17.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:17 smithi044 ceph-mon[84214]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:17.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:17 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:boot 2024-06-03T01:30:17.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:17 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:17.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:30:18.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:18 smithi044 ceph-mon[84214]: pgmap v187: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:19.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:18 smithi032 ceph-mon[111919]: pgmap v187: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:20.860 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:20 smithi032 ceph-mon[111919]: pgmap v188: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:20.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:20 smithi044 ceph-mon[84214]: pgmap v188: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: pgmap v189: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: pgmap v189: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:22.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:23.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:23 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:23.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:23 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:23.775 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:23 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:23.776 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:23 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:24.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:24 smithi032 ceph-mon[111919]: pgmap v190: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:24.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:24 smithi044 ceph-mon[84214]: pgmap v190: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: pgmap v191: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:26.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:26 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: pgmap v191: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:26.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:26.750 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:26 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: pgmap v192: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: Upgrade: Updating mds.cephfs.smithi032.tegslf 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:28 smithi032 ceph-mon[111919]: Deploying daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:30:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: pgmap v192: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: Upgrade: Updating mds.cephfs.smithi032.tegslf 2024-06-03T01:30:28.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.tegslf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:28.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:28 smithi044 ceph-mon[84214]: Deploying daemon mds.cephfs.smithi032.tegslf on smithi032 2024-06-03T01:30:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:30 smithi032 ceph-mon[111919]: pgmap v193: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:30 smithi032 ceph-mon[111919]: osdmap e94: 6 total, 6 up, 6 in 2024-06-03T01:30:30.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:30 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:30.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:30 smithi044 ceph-mon[84214]: pgmap v193: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:30.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:30 smithi044 ceph-mon[84214]: osdmap e94: 6 total, 6 up, 6 in 2024-06-03T01:30:30.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:30 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:32 smithi032 ceph-mon[111919]: pgmap v195: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:32.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:32 smithi044 ceph-mon[84214]: pgmap v195: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:boot 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:33.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:33.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:boot 2024-06-03T01:30:33.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:33.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:30:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:33.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:34.933 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:34 smithi032 ceph-mon[111919]: pgmap v196: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:34.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:34 smithi044 ceph-mon[84214]: pgmap v196: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:36.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:36 smithi032 ceph-mon[111919]: pgmap v197: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:36.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:36 smithi044 ceph-mon[84214]: pgmap v197: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:37.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:37.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:37.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:38.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:38.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:38.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:38.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:38 smithi044 ceph-mon[84214]: pgmap v198: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:39.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:38 smithi032 ceph-mon[111919]: pgmap v198: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:39.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:39 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:39.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:39 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:39 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:39 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:40.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:40 smithi032 ceph-mon[111919]: pgmap v199: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:40 smithi044 ceph-mon[84214]: pgmap v199: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:43.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: pgmap v200: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:43.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:42 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: pgmap v200: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:43.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:43.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:42 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:44.929 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: pgmap v201: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:44.929 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: Upgrade: Updating mds.cephfs.smithi044.gkcyfr 2024-06-03T01:30:44.929 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:44.929 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:44.930 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:44.930 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:44 smithi044 ceph-mon[84214]: Deploying daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: pgmap v201: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: Upgrade: Updating mds.cephfs.smithi044.gkcyfr 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.gkcyfr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:30:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:44 smithi032 ceph-mon[111919]: Deploying daemon mds.cephfs.smithi044.gkcyfr on smithi044 2024-06-03T01:30:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 -- 172.21.15.32:0/1948504111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0ff5a0 msgr2=0x7f68bc0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 --2- 172.21.15.32:0/1948504111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc0ff980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009910 tx=0x7f68ac009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 -- 172.21.15.32:0/1948504111 shutdown_connections 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 --2- 172.21.15.32:0/1948504111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0fff50 0x7f68bc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 --2- 172.21.15.32:0/1948504111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc0ff980 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 -- 172.21.15.32:0/1948504111 >> 172.21.15.32:0/1948504111 conn(0x7f68bc074e30 msgr2=0x7f68bc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 -- 172.21.15.32:0/1948504111 shutdown_connections 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.528+0000 7f68c2033700 1 -- 172.21.15.32:0/1948504111 wait complete. 2024-06-03T01:30:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 Processor -- start 2024-06-03T01:30:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 -- start start 2024-06-03T01:30:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc19f440 con 0x7f68bc0fff50 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68c2033700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc1988f0 con 0x7f68bc0ff5a0 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46662/0 (socket says 172.21.15.32:46662) 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.529+0000 7f68baffd700 1 -- 172.21.15.32:0/2746294954 learned_addr learned my addr 172.21.15.32:0/2746294954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68bb7fe700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68baffd700 1 -- 172.21.15.32:0/2746294954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0ff5a0 msgr2=0x7f68bc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68baffd700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68baffd700 1 -- 172.21.15.32:0/2746294954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0095f0 con 0x7f68bc0fff50 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68baffd700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68b000d900 tx=0x7f68b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b0009900 con 0x7f68bc0fff50 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b0010460 con 0x7f68bc0fff50 2024-06-03T01:30:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc198bd0 con 0x7f68bc0fff50 2024-06-03T01:30:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b000f5f0 con 0x7f68bc0fff50 2024-06-03T01:30:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.530+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc199120 con 0x7f68bc0fff50 2024-06-03T01:30:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.531+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc104fc0 con 0x7f68bc0fff50 2024-06-03T01:30:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.532+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f68b00105d0 con 0x7f68bc0fff50 2024-06-03T01:30:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.532+0000 7f68b8ff9700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 0x7f68a40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.532+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f68b008e070 con 0x7f68bc0fff50 2024-06-03T01:30:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.532+0000 7f68bb7fe700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 0x7f68a40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.533+0000 7f68bb7fe700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 0x7f68a40706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68ac006010 tx=0x7f68ac01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:46.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.537+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f68b005adf0 con 0x7f68bc0fff50 2024-06-03T01:30:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.767+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68bc199390 con 0x7f68a406e200 2024-06-03T01:30:46.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.768+0000 7f68b8ff9700 1 -- 172.21.15.32:0/2746294954 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f68bc199390 con 0x7f68a406e200 2024-06-03T01:30:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 msgr2=0x7f68a40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 0x7f68a40706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68ac006010 tx=0x7f68ac01a040 comp rx=0 tx=0).stop 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 msgr2=0x7f68bc19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68b000d900 tx=0x7f68b000dcc0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 shutdown_connections 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f68a406e200 0x7f68a40706c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68bc0ff5a0 0x7f68bc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 --2- 172.21.15.32:0/2746294954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68bc0fff50 0x7f68bc19edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.770+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 >> 172.21.15.32:0/2746294954 conn(0x7f68bc074e30 msgr2=0x7f68bc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.771+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 shutdown_connections 2024-06-03T01:30:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.771+0000 7f68c2033700 1 -- 172.21.15.32:0/2746294954 wait complete. 2024-06-03T01:30:46.781 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:30:46.782 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:46 smithi032 ceph-mon[111919]: pgmap v202: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 -- 172.21.15.32:0/3601643261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 msgr2=0x7f98f01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 --2- 172.21.15.32:0/3601643261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f01058a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009910 tx=0x7f98e4009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 -- 172.21.15.32:0/3601643261 shutdown_connections 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 --2- 172.21.15.32:0/3601643261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f01058a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 --2- 172.21.15.32:0/3601643261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 0x7f98f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 -- 172.21.15.32:0/3601643261 >> 172.21.15.32:0/3601643261 conn(0x7f98f00fc750 msgr2=0x7f98f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 -- 172.21.15.32:0/3601643261 shutdown_connections 2024-06-03T01:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.881+0000 7f98f6451700 1 -- 172.21.15.32:0/3601643261 wait complete. 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 Processor -- start 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 -- start start 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 0x7f98f0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0073b10 con 0x7f98f0101850 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98f6451700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0073c50 con 0x7f98f0100ea0 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48668/0 (socket says 172.21.15.32:48668) 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98ef7fe700 1 -- 172.21.15.32:0/173947563 learned_addr learned my addr 172.21.15.32:0/173947563 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.882+0000 7f98effff700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 0x7f98f0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ef7fe700 1 -- 172.21.15.32:0/173947563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 msgr2=0x7f98f0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ef7fe700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 0x7f98f0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ef7fe700 1 -- 172.21.15.32:0/173947563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40095f0 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ef7fe700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98e40092e0 tx=0x7f98e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e401d070 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f0073ed0 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98e400f460 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e40175d0 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.883+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f01a2ec0 con 0x7f98f0101850 2024-06-03T01:30:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.884+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f01091f0 con 0x7f98f0101850 2024-06-03T01:30:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.884+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98e400f5d0 con 0x7f98f0101850 2024-06-03T01:30:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.885+0000 7f98ed7fa700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 0x7f98d80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.885+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f98e408f700 con 0x7f98f0101850 2024-06-03T01:30:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.885+0000 7f98effff700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 0x7f98d80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.885+0000 7f98effff700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 0x7f98d80706c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009a10 tx=0x7f98e0009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:46.890+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f98e40588d0 con 0x7f98f0101850 2024-06-03T01:30:46.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:46 smithi044 ceph-mon[84214]: pgmap v202: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:47.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.123+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f00746a0 con 0x7f98d806e200 2024-06-03T01:30:47.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.124+0000 7f98ed7fa700 1 -- 172.21.15.32:0/173947563 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f98f00746a0 con 0x7f98d806e200 2024-06-03T01:30:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 msgr2=0x7f98d80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 0x7f98d80706c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009a10 tx=0x7f98e0009190 comp rx=0 tx=0).stop 2024-06-03T01:30:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 msgr2=0x7f98f00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98e40092e0 tx=0x7f98e400bd70 comp rx=0 tx=0).stop 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 shutdown_connections 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f98d806e200 0x7f98d80706c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f0100ea0 0x7f98f0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 --2- 172.21.15.32:0/173947563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f0101850 0x7f98f00735d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 >> 172.21.15.32:0/173947563 conn(0x7f98f00fc750 msgr2=0x7f98f00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 shutdown_connections 2024-06-03T01:30:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.126+0000 7f98f6451700 1 -- 172.21.15.32:0/173947563 wait complete. 2024-06-03T01:30:47.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 -- 172.21.15.32:0/4060823357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 msgr2=0x7f825c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 --2- 172.21.15.32:0/4060823357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c100080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8244009910 tx=0x7f8244009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:47.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 -- 172.21.15.32:0/4060823357 shutdown_connections 2024-06-03T01:30:47.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 --2- 172.21.15.32:0/4060823357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c100080 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 --2- 172.21.15.32:0/4060823357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 0x7f825c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 -- 172.21.15.32:0/4060823357 >> 172.21.15.32:0/4060823357 conn(0x7f825c075670 msgr2=0x7f825c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:47.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 -- 172.21.15.32:0/4060823357 shutdown_connections 2024-06-03T01:30:47.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.236+0000 7f8260f7f700 1 -- 172.21.15.32:0/4060823357 wait complete. 2024-06-03T01:30:47.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 Processor -- start 2024-06-03T01:30:47.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 -- start start 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 0x7f825c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c19d200 con 0x7f825c0ffc20 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f8260f7f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1966b0 con 0x7f825c105bd0 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f825a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f825a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48674/0 (socket says 172.21.15.32:48674) 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.237+0000 7f825a59c700 1 -- 172.21.15.32:0/3814544018 learned_addr learned my addr 172.21.15.32:0/3814544018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8251d9b700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 0x7f825c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f825a59c700 1 -- 172.21.15.32:0/3814544018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 msgr2=0x7f825c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f825a59c700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 0x7f825c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f825a59c700 1 -- 172.21.15.32:0/3814544018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440095f0 con 0x7f825c0ffc20 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f825a59c700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d8d0 tx=0x7f824c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:47.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824c0098a0 con 0x7f825c0ffc20 2024-06-03T01:30:47.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c196990 con 0x7f825c0ffc20 2024-06-03T01:30:47.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f824c010460 con 0x7f825c0ffc20 2024-06-03T01:30:47.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824c00f5f0 con 0x7f825c0ffc20 2024-06-03T01:30:47.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.238+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c196ee0 con 0x7f825c0ffc20 2024-06-03T01:30:47.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.239+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c04f2e0 con 0x7f825c0ffc20 2024-06-03T01:30:47.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.240+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f824c00f820 con 0x7f825c0ffc20 2024-06-03T01:30:47.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.240+0000 7f8253fff700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 0x7f82480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.240+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f824c08e180 con 0x7f825c0ffc20 2024-06-03T01:30:47.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.240+0000 7f8251d9b700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 0x7f82480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:47.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.241+0000 7f8251d9b700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 0x7f82480706c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ffa60 tx=0x7f824401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:47.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.245+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f824c05ae80 con 0x7f825c0ffc20 2024-06-03T01:30:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.463+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f825c1971c0 con 0x7f824806e200 2024-06-03T01:30:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.475+0000 7f8253fff700 1 -- 172.21.15.32:0/3814544018 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f825c1971c0 con 0x7f824806e200 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 9s ago 17m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (18m) 9s ago 18m 9084k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (17m) 45s ago 17m 8446k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (4m) 9s ago 18m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (4m) 45s ago 17m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 9s ago 17m 82.0M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (31s) 9s ago 12m 13.5M - 19.0.0-2946-g3d89accf b045ac545f82 aa391c5be983 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (15s) 9s ago 12m 13.1M - 19.0.0-2946-g3d89accf b045ac545f82 bda9b08ca07e 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (12m) 45s ago 12m 17.3M - 18.2.1 5be31c24972a 7f4a25b58a16 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (12m) 45s ago 12m 19.0M - 18.2.1 5be31c24972a 8980be55392a 2024-06-03T01:30:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (9m) 9s ago 19m 510M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (7m) 45s ago 16m 452M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (5m) 9s ago 19m 55.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (4m) 45s ago 16m 41.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 9s ago 18m 15.4M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 45s ago 16m 15.8M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (3m) 9s ago 16m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (3m) 9s ago 15m 147M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (2m) 9s ago 15m 89.8M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (116s) 45s ago 14m 102M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (82s) 45s ago 14m 90.8M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (49s) 45s ago 13m 12.0M 2241M 19.0.0-2946-g3d89accf b045ac545f82 a11e0341a922 2024-06-03T01:30:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (7m) 9s ago 17m 78.4M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:30:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 msgr2=0x7f82480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 0x7f82480706c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ffa60 tx=0x7f824401a040 comp rx=0 tx=0).stop 2024-06-03T01:30:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 msgr2=0x7f825c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d8d0 tx=0x7f824c00dbe0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 shutdown_connections 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f824806e200 0x7f82480706c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c0ffc20 0x7f825c19c630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 --2- 172.21.15.32:0/3814544018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c105bd0 0x7f825c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 >> 172.21.15.32:0/3814544018 conn(0x7f825c075670 msgr2=0x7f825c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 shutdown_connections 2024-06-03T01:30:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.477+0000 7f8260f7f700 1 -- 172.21.15.32:0/3814544018 wait complete. 2024-06-03T01:30:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 -- 172.21.15.32:0/2007317425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c105bd0 msgr2=0x7fbc2c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 --2- 172.21.15.32:0/2007317425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c105bd0 0x7fbc2c105fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009910 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 -- 172.21.15.32:0/2007317425 shutdown_connections 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 --2- 172.21.15.32:0/2007317425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 --2- 172.21.15.32:0/2007317425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c105bd0 0x7fbc2c105fb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 -- 172.21.15.32:0/2007317425 >> 172.21.15.32:0/2007317425 conn(0x7fbc2c075670 msgr2=0x7fbc2c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 -- 172.21.15.32:0/2007317425 shutdown_connections 2024-06-03T01:30:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.584+0000 7fbc3373e700 1 -- 172.21.15.32:0/2007317425 wait complete. 2024-06-03T01:30:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 Processor -- start 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 -- start start 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc2c105bd0 0x7fbc2c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1022c0 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc3373e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c102400 con 0x7fbc2c105bd0 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48692/0 (socket says 172.21.15.32:48692) 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 -- 172.21.15.32:0/3002211096 learned_addr learned my addr 172.21.15.32:0/3002211096 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 -- 172.21.15.32:0/3002211096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc2c105bd0 msgr2=0x7fbc2c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc2c105bd0 0x7fbc2c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.585+0000 7fbc314da700 1 -- 172.21.15.32:0/3002211096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280095f0 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc314da700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006010 tx=0x7fbc2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2801d070 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc28004fb0 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c102680 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc28022ea0 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.586+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1a2e80 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.587+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c04f2e0 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.588+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbc28022470 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.588+0000 7fbc227fc700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 0x7fbc180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.588+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fbc2808f720 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.588+0000 7fbc30cd9700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 0x7fbc180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.589+0000 7fbc30cd9700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 0x7fbc180706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c103030 tx=0x7fbc1c006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:47.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.593+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbc28058b40 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:47 smithi032 ceph-mon[111919]: osdmap e95: 6 total, 6 up, 6 in 2024-06-03T01:30:47.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:47 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:47.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:47 smithi032 ceph-mon[111919]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.889+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc2c1a3270 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.890+0000 7fbc227fc700 1 -- 172.21.15.32:0/3002211096 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+800 (secure 0 0 0) 0x7fbc28022720 con 0x7fbc2c0ffc20 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:30:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 12 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:30:47.891 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:30:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 msgr2=0x7fbc180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 0x7fbc180706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c103030 tx=0x7fbc1c006d20 comp rx=0 tx=0).stop 2024-06-03T01:30:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 msgr2=0x7fbc2c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006010 tx=0x7fbc2800be20 comp rx=0 tx=0).stop 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 shutdown_connections 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc1806e200 0x7fbc180706c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2c0ffc20 0x7fbc2c101840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 --2- 172.21.15.32:0/3002211096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc2c105bd0 0x7fbc2c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 >> 172.21.15.32:0/3002211096 conn(0x7fbc2c075670 msgr2=0x7fbc2c0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 shutdown_connections 2024-06-03T01:30:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:47.892+0000 7fbc3373e700 1 -- 172.21.15.32:0/3002211096 wait complete. 2024-06-03T01:30:47.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:47 smithi044 ceph-mon[84214]: osdmap e95: 6 total, 6 up, 6 in 2024-06-03T01:30:47.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:47 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:30:47.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:47 smithi044 ceph-mon[84214]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3343922190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80fff50 msgr2=0x7f2ea8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3343922190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80fff50 0x7f2ea8105840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009910 tx=0x7f2e9c009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3343922190 shutdown_connections 2024-06-03T01:30:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3343922190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80fff50 0x7f2ea8105840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3343922190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3343922190 >> 172.21.15.32:0/3343922190 conn(0x7f2ea8074e30 msgr2=0x7f2ea8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3343922190 shutdown_connections 2024-06-03T01:30:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.000+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3343922190 wait complete. 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 Processor -- start 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 -- start start 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80fff50 0x7f2ea8100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8100fa0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2eae7d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81010e0 con 0x7f2ea80fff50 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.001+0000 7f2ea7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48708/0 (socket says 172.21.15.32:48708) 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 -- 172.21.15.32:0/3704900086 learned_addr learned my addr 172.21.15.32:0/3704900086 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea77fe700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80fff50 0x7f2ea8100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 -- 172.21.15.32:0/3704900086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80fff50 msgr2=0x7f2ea8100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80fff50 0x7f2ea8100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 -- 172.21.15.32:0/3704900086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0095f0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea7fff700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c8a0 tx=0x7f2e9800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e980078e0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e9800f450 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e9800e5e0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81a2a90 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.002+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea81a2ea0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.003+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8109200 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.004+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2e9800f5c0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.004+0000 7f2ea57fa700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 0x7f2e900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.004+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f2e9808e1c0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.004+0000 7f2ea77fe700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 0x7f2e900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.005+0000 7f2ea77fe700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 0x7f2e900706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8101d10 tx=0x7f2e9c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.009+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2e98057370 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.275+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ea8108be0 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.275+0000 7f2ea57fa700 1 -- 172.21.15.32:0/3704900086 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1694 (secure 0 0 0) 0x7f2e9805aa30 con 0x7f2ea80ff5a0 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:e28 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 22 2024-06-03T01:30:48.276 INFO:teuthology.orchestra.run.smithi032.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:30:06.909705+0000 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 91 2024-06-03T01:30:48.277 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:up {} 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:failed 0,1 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:30:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:24287} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/1615260462,v1:172.21.15.44:6827/1615260462] compat {c=[1],r=[1],i=[7ff]}] 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:30:48.279 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{-1:44287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 msgr2=0x7f2e900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 0x7f2e900706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8101d10 tx=0x7f2e9c01a040 comp rx=0 tx=0).stop 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 msgr2=0x7f2ea8100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c8a0 tx=0x7f2e9800cbb0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 shutdown_connections 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f2e9006e200 0x7f2e900706c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ea80ff5a0 0x7f2ea8100520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 --2- 172.21.15.32:0/3704900086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea80fff50 0x7f2ea8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 >> 172.21.15.32:0/3704900086 conn(0x7f2ea8074e30 msgr2=0x7f2ea80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:48.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 shutdown_connections 2024-06-03T01:30:48.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.278+0000 7f2eae7d7700 1 -- 172.21.15.32:0/3704900086 wait complete. 2024-06-03T01:30:48.281 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 -- 172.21.15.32:0/4014222357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54102720 msgr2=0x7f5a5410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/4014222357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54102720 0x7f5a5410ac10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009910 tx=0x7f5a3c009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 -- 172.21.15.32:0/4014222357 shutdown_connections 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/4014222357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54102720 0x7f5a5410ac10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/4014222357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a541021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 -- 172.21.15.32:0/4014222357 >> 172.21.15.32:0/4014222357 conn(0x7f5a540fb650 msgr2=0x7f5a540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 -- 172.21.15.32:0/4014222357 shutdown_connections 2024-06-03T01:30:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.387+0000 7f5a58c0a700 1 -- 172.21.15.32:0/4014222357 wait complete. 2024-06-03T01:30:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 Processor -- start 2024-06-03T01:30:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 -- start start 2024-06-03T01:30:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54102720 0x7f5a5419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5419d400 con 0x7f5a54101e00 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a58c0a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a541968b0 con 0x7f5a54102720 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48728/0 (socket says 172.21.15.32:48728) 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 -- 172.21.15.32:0/3514608035 learned_addr learned my addr 172.21.15.32:0/3514608035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 -- 172.21.15.32:0/3514608035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54102720 msgr2=0x7f5a5419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a51d9b700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54102720 0x7f5a5419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54102720 0x7f5a5419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.388+0000 7f5a5259c700 1 -- 172.21.15.32:0/3514608035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0095f0 con 0x7f5a54101e00 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a5259c700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b4b0 tx=0x7f5a4400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a44011800 con 0x7f5a54101e00 2024-06-03T01:30:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a44011e20 con 0x7f5a54101e00 2024-06-03T01:30:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a54196b90 con 0x7f5a54101e00 2024-06-03T01:30:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4400f2e0 con 0x7f5a54101e00 2024-06-03T01:30:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.389+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a541970e0 con 0x7f5a54101e00 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.390+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a54108310 con 0x7f5a54101e00 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.390+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a44010380 con 0x7f5a54101e00 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.391+0000 7f5a4b7fe700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 0x7f5a400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.391+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f5a4408de60 con 0x7f5a54101e00 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.391+0000 7f5a51d9b700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 0x7f5a400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.391+0000 7f5a51d9b700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 0x7f5a400706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.396+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a4405ab60 con 0x7f5a54101e00 2024-06-03T01:30:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.625+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a54197e80 con 0x7f5a4006e200 2024-06-03T01:30:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.627+0000 7f5a4b7fe700 1 -- 172.21.15.32:0/3514608035 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5a54197e80 con 0x7f5a4006e200 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-06-03T01:30:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-06-03T01:30:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:30:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "14/23 daemons upgraded", 2024-06-03T01:30:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mds daemons", 2024-06-03T01:30:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:30:48.628 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:30:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 msgr2=0x7f5a400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 0x7f5a400706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c005960 comp rx=0 tx=0).stop 2024-06-03T01:30:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 msgr2=0x7f5a5419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b4b0 tx=0x7f5a4400b870 comp rx=0 tx=0).stop 2024-06-03T01:30:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 shutdown_connections 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5a4006e200 0x7f5a400706c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a54101e00 0x7f5a5419c830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 --2- 172.21.15.32:0/3514608035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54102720 0x7f5a5419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 >> 172.21.15.32:0/3514608035 conn(0x7f5a540fb650 msgr2=0x7f5a54105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 shutdown_connections 2024-06-03T01:30:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.629+0000 7f5a58c0a700 1 -- 172.21.15.32:0/3514608035 wait complete. 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- 172.21.15.32:0/2036503323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80106cd0 msgr2=0x7f7e801070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2036503323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80106cd0 0x7f7e801070b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009910 tx=0x7f7e74009c20 comp rx=0 tx=0).stop 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- 172.21.15.32:0/2036503323 shutdown_connections 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2036503323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80068580 0x7f7e800689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2036503323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80106cd0 0x7f7e801070b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- 172.21.15.32:0/2036503323 >> 172.21.15.32:0/2036503323 conn(0x7f7e80075670 msgr2=0x7f7e80075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- 172.21.15.32:0/2036503323 shutdown_connections 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- 172.21.15.32:0/2036503323 wait complete. 2024-06-03T01:30:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 Processor -- start 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.740+0000 7f7e86c67700 1 -- start start 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e86c67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e86c67700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 0x7f7e8019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e86c67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8019d230 con 0x7f7e80068580 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e86c67700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e801966e0 con 0x7f7e80106cd0 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48738/0 (socket says 172.21.15.32:48738) 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 -- 172.21.15.32:0/2657368353 learned_addr learned my addr 172.21.15.32:0/2657368353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e7ffff700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 0x7f7e8019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 -- 172.21.15.32:0/2657368353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 msgr2=0x7f7e8019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 0x7f7e8019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 -- 172.21.15.32:0/2657368353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740095f0 con 0x7f7e80068580 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e7ffff700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 0x7f7e8019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e84a03700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400f690 tx=0x7f7e7400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e7401d070 con 0x7f7e80068580 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80196960 con 0x7f7e80068580 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.741+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e740050d0 con 0x7f7e80068580 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.742+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e80196e50 con 0x7f7e80068580 2024-06-03T01:30:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.742+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e74022470 con 0x7f7e80068580 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.743+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e801091f0 con 0x7f7e80068580 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.743+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7e74003680 con 0x7f7e80068580 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.744+0000 7f7e7dffb700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 0x7f7e700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.744+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f7e7408f7c0 con 0x7f7e80068580 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.744+0000 7f7e7ffff700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 0x7f7e700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:30:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.744+0000 7f7e7ffff700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 0x7f7e700706c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e80197b50 tx=0x7f7e6c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:30:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:48.749+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7e7405c570 con 0x7f7e80068580 2024-06-03T01:30:48.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: pgmap v204: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:48.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:48.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3002211096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:30:48.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:48 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3704900086' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:30:49.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.048+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7e8004f2e0 con 0x7f7e80068580 2024-06-03T01:30:49.048 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: pgmap v204: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:49.049 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:49.049 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:49.049 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3002211096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:30:49.049 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:30:49.049 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:48 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3704900086' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:30:49.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.048+0000 7f7e7dffb700 1 -- 172.21.15.32:0/2657368353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f7e7405c100 con 0x7f7e80068580 2024-06-03T01:30:49.049 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-06-03T01:30:49.049 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-03T01:30:49.049 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is degraded 2024-06-03T01:30:49.050 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:30:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:30:49.050 INFO:teuthology.orchestra.run.smithi032.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-03T01:30:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 msgr2=0x7f7e700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 0x7f7e700706c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e80197b50 tx=0x7f7e6c009260 comp rx=0 tx=0).stop 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 msgr2=0x7f7e8019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400f690 tx=0x7f7e7400f770 comp rx=0 tx=0).stop 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 shutdown_connections 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f7e7006e200 0x7f7e700706c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e80068580 0x7f7e8019c660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 --2- 172.21.15.32:0/2657368353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e80106cd0 0x7f7e8019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:30:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 >> 172.21.15.32:0/2657368353 conn(0x7f7e80075670 msgr2=0x7f7e800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:30:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 shutdown_connections 2024-06-03T01:30:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:30:49.051+0000 7f7e86c67700 1 -- 172.21.15.32:0/2657368353 wait complete. 2024-06-03T01:30:49.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:49 smithi032 ceph-mon[111919]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:49.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:49 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2657368353' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:30:49.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:49 smithi044 ceph-mon[84214]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:30:49.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:49 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2657368353' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:30:50.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:50 smithi044 ceph-mon[84214]: pgmap v205: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:50 smithi032 ceph-mon[111919]: pgmap v205: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: pgmap v206: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.44:6824/2797390272,v1:172.21.15.44:6825/2797390272] up:boot 2024-06-03T01:30:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:52.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:30:52.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: pgmap v206: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.44:6824/2797390272,v1:172.21.15.44:6825/2797390272] up:boot 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:30:52.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:30:54.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:54 smithi044 ceph-mon[84214]: pgmap v207: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:54.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:54 smithi032 ceph-mon[111919]: pgmap v207: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:55.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:55 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:55.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:55.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:55 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:56 smithi044 ceph-mon[84214]: pgmap v208: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:56 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:56.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:56 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:56 smithi032 ceph-mon[111919]: pgmap v208: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:56 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:56 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:30:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:30:58 smithi044 ceph-mon[84214]: pgmap v209: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:30:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:30:58 smithi032 ceph-mon[111919]: pgmap v209: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:00.466 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: pgmap v210: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:00.466 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.467 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:00 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: pgmap v210: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:00.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:00.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:00.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:00 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:02.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: pgmap v211: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: Upgrade: Updating mds.cephfs.smithi044.mjxnhh 2024-06-03T01:31:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:31:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:02.586 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:02 smithi032 ceph-mon[111919]: Deploying daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:31:02.632 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: pgmap v211: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:02.632 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: Upgrade: Updating mds.cephfs.smithi044.mjxnhh 2024-06-03T01:31:02.632 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:02.632 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.mjxnhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:31:02.633 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:02.633 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:02 smithi044 ceph-mon[84214]: Deploying daemon mds.cephfs.smithi044.mjxnhh on smithi044 2024-06-03T01:31:04.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:04 smithi044 ceph-mon[84214]: pgmap v212: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:04.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:04 smithi032 ceph-mon[111919]: pgmap v212: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:05.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:05 smithi044 ceph-mon[84214]: osdmap e96: 6 total, 6 up, 6 in 2024-06-03T01:31:05.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:05 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:31:05.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:05 smithi032 ceph-mon[111919]: osdmap e96: 6 total, 6 up, 6 in 2024-06-03T01:31:05.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:05 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-03T01:31:06.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:06 smithi044 ceph-mon[84214]: pgmap v214: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:06.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:06 smithi032 ceph-mon[111919]: pgmap v214: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:07.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:07.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:08.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:08 smithi044 ceph-mon[84214]: pgmap v215: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:08.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:08 smithi032 ceph-mon[111919]: pgmap v215: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: pgmap v216: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.44:6826/4018885964,v1:172.21.15.44:6827/4018885964] up:boot 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:31:10.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:10 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: pgmap v216: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.44:6826/4018885964,v1:172.21.15.44:6827/4018885964] up:boot 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:31:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:10 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:31:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:12 smithi032 ceph-mon[111919]: pgmap v217: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:12.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:12 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:12 smithi044 ceph-mon[84214]: pgmap v217: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:12 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:14.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:14 smithi044 ceph-mon[84214]: pgmap v218: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:14.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:14 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:14 smithi032 ceph-mon[111919]: pgmap v218: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:14 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:16 smithi044 ceph-mon[84214]: pgmap v219: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:16.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:16 smithi032 ceph-mon[111919]: pgmap v219: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.060 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.itqzrm"}]': finished 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.tegslf"}]': finished 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.gkcyfr"}]': finished 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.mjxnhh"}]': finished 2024-06-03T01:31:18.061 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:17 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-03T01:31:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:18.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.itqzrm"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.itqzrm"}]': finished 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.tegslf"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.tegslf"}]': finished 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.gkcyfr"}]: dispatch 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.gkcyfr"}]': finished 2024-06-03T01:31:18.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.mjxnhh"}]: dispatch 2024-06-03T01:31:18.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.mjxnhh"}]': finished 2024-06-03T01:31:18.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:17 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-03T01:31:18.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all mds 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: Upgrade: Setting filesystem cephfs Joinable 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: pgmap v220: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: Standby daemon mds.cephfs.smithi032.itqzrm assigned to filesystem cephfs as rank 0 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: Standby daemon mds.cephfs.smithi032.tegslf assigned to filesystem cephfs as rank 1 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:replay,1=cephfs.smithi032.tegslf=up:replay} 2 up:standby 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:18.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:18.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:18 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all mds 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: Upgrade: Setting filesystem cephfs Joinable 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: pgmap v220: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: Standby daemon mds.cephfs.smithi032.itqzrm assigned to filesystem cephfs as rank 0 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: Standby daemon mds.cephfs.smithi032.tegslf assigned to filesystem cephfs as rank 1 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:31:19.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:replay,1=cephfs.smithi032.tegslf=up:replay} 2 up:standby 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:19.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:18 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 -- 172.21.15.32:0/1876489314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc0ffb40 msgr2=0x7f74cc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 --2- 172.21.15.32:0/1876489314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc0ffb40 0x7f74cc0fffc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009910 tx=0x7f74c0009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 -- 172.21.15.32:0/1876489314 shutdown_connections 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 --2- 172.21.15.32:0/1876489314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc0ffb40 0x7f74cc0fffc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 --2- 172.21.15.32:0/1876489314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc105bf0 0x7f74cc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.242+0000 7f74d1791700 1 -- 172.21.15.32:0/1876489314 >> 172.21.15.32:0/1876489314 conn(0x7f74cc0fb610 msgr2=0x7f74cc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.243+0000 7f74d1791700 1 -- 172.21.15.32:0/1876489314 shutdown_connections 2024-06-03T01:31:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.243+0000 7f74d1791700 1 -- 172.21.15.32:0/1876489314 wait complete. 2024-06-03T01:31:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.244+0000 7f74d1791700 1 Processor -- start 2024-06-03T01:31:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.244+0000 7f74d1791700 1 -- start start 2024-06-03T01:31:19.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.245+0000 7f74d1791700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 0x7f74cc195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.245+0000 7f74d1791700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.245+0000 7f74d1791700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1962e0 con 0x7f74cc105bf0 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.245+0000 7f74d1791700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc199010 con 0x7f74cc0ffb40 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74caffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 0x7f74cc195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47356/0 (socket says 172.21.15.32:47356) 2024-06-03T01:31:19.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 -- 172.21.15.32:0/406510569 learned_addr learned my addr 172.21.15.32:0/406510569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 -- 172.21.15.32:0/406510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 msgr2=0x7f74cc195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 0x7f74cc195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74ca7fc700 1 -- 172.21.15.32:0/406510569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00095f0 con 0x7f74cc105bf0 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.246+0000 7f74caffd700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 0x7f74cc195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74ca7fc700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74c001a850 tx=0x7f74c001a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:19.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c001f070 con 0x7f74cc105bf0 2024-06-03T01:31:19.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc199290 con 0x7f74cc105bf0 2024-06-03T01:31:19.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74c00050d0 con 0x7f74cc105bf0 2024-06-03T01:31:19.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc199780 con 0x7f74cc105bf0 2024-06-03T01:31:19.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.247+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c000f460 con 0x7f74cc105bf0 2024-06-03T01:31:19.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.250+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0679c0 con 0x7f74cc105bf0 2024-06-03T01:31:19.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.250+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f74c0023070 con 0x7f74cc105bf0 2024-06-03T01:31:19.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.251+0000 7f74b3fff700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 0x7f74b4070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.251+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f74c0092620 con 0x7f74cc105bf0 2024-06-03T01:31:19.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.251+0000 7f74caffd700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 0x7f74b4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.252+0000 7f74caffd700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 0x7f74b4070710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f74bc011490 tx=0x7f74bc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:19.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.261+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f74c005b750 con 0x7f74cc105bf0 2024-06-03T01:31:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.678+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74cc199aa0 con 0x7f74b406e250 2024-06-03T01:31:19.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.680+0000 7f74b3fff700 1 -- 172.21.15.32:0/406510569 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f74cc199aa0 con 0x7f74b406e250 2024-06-03T01:31:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 msgr2=0x7f74b4070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 0x7f74b4070710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f74bc011490 tx=0x7f74bc009260 comp rx=0 tx=0).stop 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 msgr2=0x7f74cc195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74c001a850 tx=0x7f74c001a880 comp rx=0 tx=0).stop 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 shutdown_connections 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f74b406e250 0x7f74b4070710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc0ffb40 0x7f74cc195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 --2- 172.21.15.32:0/406510569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74cc105bf0 0x7f74cc195c50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 >> 172.21.15.32:0/406510569 conn(0x7f74cc0fb610 msgr2=0x7f74cc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 shutdown_connections 2024-06-03T01:31:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.683+0000 7f74d1791700 1 -- 172.21.15.32:0/406510569 wait complete. 2024-06-03T01:31:19.695 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:31:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 -- 172.21.15.32:0/3329309299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5140105bd0 msgr2=0x7f5140105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 --2- 172.21.15.32:0/3329309299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5140105bd0 0x7f5140105fb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5130009910 tx=0x7f5130009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 -- 172.21.15.32:0/3329309299 shutdown_connections 2024-06-03T01:31:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 --2- 172.21.15.32:0/3329309299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51400ffc20 0x7f5140100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 --2- 172.21.15.32:0/3329309299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5140105bd0 0x7f5140105fb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.798+0000 7f5144c1f700 1 -- 172.21.15.32:0/3329309299 >> 172.21.15.32:0/3329309299 conn(0x7f5140075670 msgr2=0x7f5140075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.799+0000 7f5144c1f700 1 -- 172.21.15.32:0/3329309299 shutdown_connections 2024-06-03T01:31:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.801+0000 7f5144c1f700 1 -- 172.21.15.32:0/3329309299 wait complete. 2024-06-03T01:31:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 Processor -- start 2024-06-03T01:31:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 -- start start 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5140105bd0 0x7f514010b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514010c880 con 0x7f51400ffc20 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f5144c1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514010c9f0 con 0x7f5140105bd0 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f513e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f513e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47380/0 (socket says 172.21.15.32:47380) 2024-06-03T01:31:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.802+0000 7f513e59c700 1 -- 172.21.15.32:0/2151267512 learned_addr learned my addr 172.21.15.32:0/2151267512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f513dd9b700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5140105bd0 0x7f514010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f513e59c700 1 -- 172.21.15.32:0/2151267512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5140105bd0 msgr2=0x7f514010b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f513e59c700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5140105bd0 0x7f514010b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f513e59c700 1 -- 172.21.15.32:0/2151267512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300095f0 con 0x7f51400ffc20 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f513e59c700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5130005f50 tx=0x7f513000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513001d070 con 0x7f51400ffc20 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514010b5b0 con 0x7f51400ffc20 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.803+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f513000b610 con 0x7f51400ffc20 2024-06-03T01:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.804+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51401a7270 con 0x7f51400ffc20 2024-06-03T01:31:19.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.804+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5130022470 con 0x7f51400ffc20 2024-06-03T01:31:19.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.805+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51401080f0 con 0x7f51400ffc20 2024-06-03T01:31:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.806+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f513000fca0 con 0x7f51400ffc20 2024-06-03T01:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.807+0000 7f512f7fe700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 0x7f5128070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.807+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f513008f820 con 0x7f51400ffc20 2024-06-03T01:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.807+0000 7f513dd9b700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 0x7f5128070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.808+0000 7f513dd9b700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 0x7f5128070710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f514010c370 tx=0x7f5134009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:19.812+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5130058950 con 0x7f51400ffc20 2024-06-03T01:31:19.957 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:19 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all rgw 2024-06-03T01:31:19.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:19 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all rbd-mirror 2024-06-03T01:31:20.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:19 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all rgw 2024-06-03T01:31:20.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:19 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all rbd-mirror 2024-06-03T01:31:20.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.261+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f514010c070 con 0x7f512806e250 2024-06-03T01:31:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.263+0000 7f512f7fe700 1 -- 172.21.15.32:0/2151267512 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f514010c070 con 0x7f512806e250 2024-06-03T01:31:20.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 msgr2=0x7f5128070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 0x7f5128070710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f514010c370 tx=0x7f5134009260 comp rx=0 tx=0).stop 2024-06-03T01:31:20.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 msgr2=0x7f514010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5130005f50 tx=0x7f513000f740 comp rx=0 tx=0).stop 2024-06-03T01:31:20.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 shutdown_connections 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f512806e250 0x7f5128070710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51400ffc20 0x7f514010aad0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 --2- 172.21.15.32:0/2151267512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5140105bd0 0x7f514010b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.266+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 >> 172.21.15.32:0/2151267512 conn(0x7f5140075670 msgr2=0x7f51400fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.267+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 shutdown_connections 2024-06-03T01:31:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.267+0000 7f5144c1f700 1 -- 172.21.15.32:0/2151267512 wait complete. 2024-06-03T01:31:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 -- 172.21.15.32:0/1702391672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 msgr2=0x7fe69c0ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/1702391672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c0ff7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00aa20 tx=0x7fe68c00ad30 comp rx=0 tx=0).stop 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 -- 172.21.15.32:0/1702391672 shutdown_connections 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/1702391672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c0ff7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/1702391672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 -- 172.21.15.32:0/1702391672 >> 172.21.15.32:0/1702391672 conn(0x7fe69c074e30 msgr2=0x7fe69c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 -- 172.21.15.32:0/1702391672 shutdown_connections 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.417+0000 7fe6a3e18700 1 -- 172.21.15.32:0/1702391672 wait complete. 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.419+0000 7fe6a3e18700 1 Processor -- start 2024-06-03T01:31:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.419+0000 7fe6a3e18700 1 -- start start 2024-06-03T01:31:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.419+0000 7fe6a3e18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a3e18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c194bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a1bb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a1bb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47412/0 (socket says 172.21.15.32:47412) 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a1bb4700 1 -- 172.21.15.32:0/622919279 learned_addr learned my addr 172.21.15.32:0/622919279 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c195290 con 0x7fe69c0ff340 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c198e10 con 0x7fe69c105350 2024-06-03T01:31:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.420+0000 7fe6a13b3700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c194bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.421+0000 7fe6a1bb4700 1 -- 172.21.15.32:0/622919279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 msgr2=0x7fe69c194bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.421+0000 7fe6a1bb4700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c194bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.421+0000 7fe6a1bb4700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c00a700 con 0x7fe69c0ff340 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.421+0000 7fe6a13b3700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c194bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.421+0000 7fe6a1bb4700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe69800d900 tx=0x7fe69800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.422+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6980041f0 con 0x7fe69c0ff340 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.422+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c1990f0 con 0x7fe69c0ff340 2024-06-03T01:31:20.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.422+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c199640 con 0x7fe69c0ff340 2024-06-03T01:31:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.422+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe698004cf0 con 0x7fe69c0ff340 2024-06-03T01:31:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.423+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69800b750 con 0x7fe69c0ff340 2024-06-03T01:31:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.423+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c107850 con 0x7fe69c0ff340 2024-06-03T01:31:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.425+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe698004750 con 0x7fe69c0ff340 2024-06-03T01:31:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.426+0000 7fe692ffd700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 0x7fe6880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.426+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fe69808e720 con 0x7fe69c0ff340 2024-06-03T01:31:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.427+0000 7fe6a13b3700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 0x7fe6880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.428+0000 7fe6a13b3700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 0x7fe6880706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006010 tx=0x7fe68c00af70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:20.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.430+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe6980578d0 con 0x7fe69c0ff340 2024-06-03T01:31:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.871+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe69c199920 con 0x7fe68806e200 2024-06-03T01:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.883+0000 7fe692ffd700 1 -- 172.21.15.32:0/622919279 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe69c199920 con 0x7fe68806e200 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 43s ago 18m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (18m) 43s ago 18m 9084k - 18.2.1 5be31c24972a 029bc69b8fe2 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (17m) 8s ago 17m 8753k - 18.2.1 5be31c24972a 10ff7cd7fa11 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (4m) 43s ago 18m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (4m) 8s ago 17m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 43s ago 18m 82.0M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (64s) 43s ago 13m 13.5M - 19.0.0-2946-g3d89accf b045ac545f82 aa391c5be983 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (48s) 43s ago 12m 13.1M - 19.0.0-2946-g3d89accf b045ac545f82 bda9b08ca07e 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (29s) 8s ago 13m 14.9M - 19.0.0-2946-g3d89accf b045ac545f82 f5625d49b4b6 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (12s) 8s ago 12m 11.5M - 19.0.0-2946-g3d89accf b045ac545f82 b8eec5ca5f6b 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (9m) 43s ago 19m 510M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:31:20.884 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (8m) 8s ago 17m 452M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (5m) 43s ago 20m 55.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (5m) 8s ago 17m 45.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 43s ago 18m 15.4M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 8s ago 17m 15.7M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (4m) 43s ago 16m 142M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (3m) 43s ago 16m 147M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (3m) 43s ago 15m 89.8M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (2m) 8s ago 15m 103M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (116s) 8s ago 14m 91.6M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (82s) 8s ago 14m 93.8M 2241M 19.0.0-2946-g3d89accf b045ac545f82 a11e0341a922 2024-06-03T01:31:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (7m) 43s ago 18m 78.4M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 msgr2=0x7fe6880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 0x7fe6880706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006010 tx=0x7fe68c00af70 comp rx=0 tx=0).stop 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 msgr2=0x7fe69c194670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe69800d900 tx=0x7fe69800dc10 comp rx=0 tx=0).stop 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 shutdown_connections 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe68806e200 0x7fe6880706c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe69c0ff340 0x7fe69c194670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 --2- 172.21.15.32:0/622919279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69c105350 0x7fe69c194bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 >> 172.21.15.32:0/622919279 conn(0x7fe69c074e30 msgr2=0x7fe69c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 shutdown_connections 2024-06-03T01:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:20.885+0000 7fe6a3e18700 1 -- 172.21.15.32:0/622919279 wait complete. 2024-06-03T01:31:20.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:20 smithi032 ceph-mon[111919]: pgmap v221: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:20.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:20 smithi032 ceph-mon[111919]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:20.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:20.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:31:20.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:20 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 -- 172.21.15.32:0/4281473276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff828105bd0 msgr2=0x7ff828105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/4281473276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff828105bd0 0x7ff828105fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff818009910 tx=0x7ff818009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 -- 172.21.15.32:0/4281473276 shutdown_connections 2024-06-03T01:31:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/4281473276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8280ffc20 0x7ff828100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/4281473276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff828105bd0 0x7ff828105fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.016+0000 7ff82e5a8700 1 -- 172.21.15.32:0/4281473276 >> 172.21.15.32:0/4281473276 conn(0x7ff828075670 msgr2=0x7ff828075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.017+0000 7ff82e5a8700 1 -- 172.21.15.32:0/4281473276 shutdown_connections 2024-06-03T01:31:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.017+0000 7ff82e5a8700 1 -- 172.21.15.32:0/4281473276 wait complete. 2024-06-03T01:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.018+0000 7ff82e5a8700 1 Processor -- start 2024-06-03T01:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.018+0000 7ff82e5a8700 1 -- start start 2024-06-03T01:31:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.018+0000 7ff82e5a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.018+0000 7ff82e5a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff828105bd0 0x7ff8281959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47436/0 (socket says 172.21.15.32:47436) 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 -- 172.21.15.32:0/2392598631 learned_addr learned my addr 172.21.15.32:0/2392598631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff828195f00 con 0x7ff8280ffc20 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff828196070 con 0x7ff828105bd0 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff8277fe700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff828105bd0 0x7ff8281959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 -- 172.21.15.32:0/2392598631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff828105bd0 msgr2=0x7ff8281959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff828105bd0 0x7ff8281959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180095f0 con 0x7ff8280ffc20 2024-06-03T01:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.019+0000 7ff827fff700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff818005d10 tx=0x7ff81800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.020+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81801d070 con 0x7ff8280ffc20 2024-06-03T01:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.020+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8281962f0 con 0x7ff8280ffc20 2024-06-03T01:31:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.020+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff81800b610 con 0x7ff8280ffc20 2024-06-03T01:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.021+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff818003ea0 con 0x7ff8280ffc20 2024-06-03T01:31:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.021+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82819a7b0 con 0x7ff8280ffc20 2024-06-03T01:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.022+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82804f2e0 con 0x7ff8280ffc20 2024-06-03T01:31:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.022+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff81800faf0 con 0x7ff8280ffc20 2024-06-03T01:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.023+0000 7ff8257fa700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 0x7ff8100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.023+0000 7ff8277fe700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 0x7ff8100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.023+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7ff81805c140 con 0x7ff8280ffc20 2024-06-03T01:31:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.024+0000 7ff8277fe700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 0x7ff8100706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff828068c80 tx=0x7ff81c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.033+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff818057ce0 con 0x7ff8280ffc20 2024-06-03T01:31:21.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:20 smithi044 ceph-mon[84214]: pgmap v221: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:21.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:20 smithi044 ceph-mon[84214]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:21.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:21.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi032", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:31:21.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:20 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.551+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff828101280 con 0x7ff8280ffc20 2024-06-03T01:31:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.551+0000 7ff8257fa700 1 -- 172.21.15.32:0/2392598631 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff81805b3a0 con 0x7ff8280ffc20 2024-06-03T01:31:21.552 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:31:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:31:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 14 2024-06-03T01:31:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:31:21.554 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:31:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 msgr2=0x7ff8100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 0x7ff8100706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff828068c80 tx=0x7ff81c009260 comp rx=0 tx=0).stop 2024-06-03T01:31:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 msgr2=0x7ff828195480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff818005d10 tx=0x7ff81800f690 comp rx=0 tx=0).stop 2024-06-03T01:31:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 shutdown_connections 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff81006e200 0x7ff8100706c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8280ffc20 0x7ff828195480 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 --2- 172.21.15.32:0/2392598631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff828105bd0 0x7ff8281959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 >> 172.21.15.32:0/2392598631 conn(0x7ff828075670 msgr2=0x7ff8280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 shutdown_connections 2024-06-03T01:31:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.555+0000 7ff82e5a8700 1 -- 172.21.15.32:0/2392598631 wait complete. 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 -- 172.21.15.32:0/1876964790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348101850 msgr2=0x7ff3481058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 --2- 172.21.15.32:0/1876964790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348101850 0x7ff3481058a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009910 tx=0x7ff33c009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 -- 172.21.15.32:0/1876964790 shutdown_connections 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 --2- 172.21.15.32:0/1876964790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348101850 0x7ff3481058a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 --2- 172.21.15.32:0/1876964790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348100ea0 0x7ff348101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.727+0000 7ff34d122700 1 -- 172.21.15.32:0/1876964790 >> 172.21.15.32:0/1876964790 conn(0x7ff3480fc750 msgr2=0x7ff3480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.728+0000 7ff34d122700 1 -- 172.21.15.32:0/1876964790 shutdown_connections 2024-06-03T01:31:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.728+0000 7ff34d122700 1 -- 172.21.15.32:0/1876964790 wait complete. 2024-06-03T01:31:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 Processor -- start 2024-06-03T01:31:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 -- start start 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348101850 0x7ff3480735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348073b10 con 0x7ff348100ea0 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff34d122700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348073c50 con 0x7ff348101850 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.729+0000 7ff346d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff346d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47444/0 (socket says 172.21.15.32:47444) 2024-06-03T01:31:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff346d9d700 1 -- 172.21.15.32:0/4073140374 learned_addr learned my addr 172.21.15.32:0/4073140374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff34659c700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348101850 0x7ff3480735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff346d9d700 1 -- 172.21.15.32:0/4073140374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348101850 msgr2=0x7ff3480735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff346d9d700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348101850 0x7ff3480735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.730+0000 7ff346d9d700 1 -- 172.21.15.32:0/4073140374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0095f0 con 0x7ff348100ea0 2024-06-03T01:31:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.731+0000 7ff346d9d700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff338014300 tx=0x7ff338014610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.732+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff338017070 con 0x7ff348100ea0 2024-06-03T01:31:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.732+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff338012e00 con 0x7ff348100ea0 2024-06-03T01:31:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.732+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff348073ed0 con 0x7ff348100ea0 2024-06-03T01:31:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.732+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff348196940 con 0x7ff348100ea0 2024-06-03T01:31:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.733+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff338010c10 con 0x7ff348100ea0 2024-06-03T01:31:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.733+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3481092a0 con 0x7ff348100ea0 2024-06-03T01:31:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.735+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff3380120d0 con 0x7ff348100ea0 2024-06-03T01:31:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.736+0000 7ff32ffff700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 0x7ff3300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.736+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7ff3380963e0 con 0x7ff348100ea0 2024-06-03T01:31:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.736+0000 7ff34659c700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 0x7ff3300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.737+0000 7ff34659c700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 0x7ff3300706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff348074880 tx=0x7ff33c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:21.740+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff33805f630 con 0x7ff348100ea0 2024-06-03T01:31:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.038+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff348108c80 con 0x7ff348100ea0 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.039+0000 7ff32ffff700 1 -- 172.21.15.32:0/4073140374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1933 (secure 0 0 0) 0x7ff338062cf0 con 0x7ff348100ea0 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout:e33 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:31:22.040 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 33 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:31:17.977743+0000 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 91 2024-06-03T01:31:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=34414,1=44287} 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 0 members: 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:34414} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:31:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:44287} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/2797390272,v1:172.21.15.44:6825/2797390272] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6826/4018885964,v1:172.21.15.44:6827/4018885964] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.042+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 msgr2=0x7ff3300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.042+0000 7ff34d122700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 0x7ff3300706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff348074880 tx=0x7ff33c01a040 comp rx=0 tx=0).stop 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.042+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 msgr2=0x7ff348073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.042+0000 7ff34d122700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff338014300 tx=0x7ff338014610 comp rx=0 tx=0).stop 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 shutdown_connections 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff33006e200 0x7ff3300706c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff348100ea0 0x7ff348073090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 --2- 172.21.15.32:0/4073140374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348101850 0x7ff3480735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 >> 172.21.15.32:0/4073140374 conn(0x7ff3480fc750 msgr2=0x7ff3480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 shutdown_connections 2024-06-03T01:31:22.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.043+0000 7ff34d122700 1 -- 172.21.15.32:0/4073140374 wait complete. 2024-06-03T01:31:22.045 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-06-03T01:31:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:21 smithi032 ceph-mon[111919]: Upgrade: Updating ceph-exporter.smithi032 (1/2) 2024-06-03T01:31:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:21 smithi032 ceph-mon[111919]: Deploying daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:31:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:21 smithi032 ceph-mon[111919]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:22.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:21 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2392598631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 -- 172.21.15.32:0/1927240850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c100380 msgr2=0x7f269c10d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 --2- 172.21.15.32:0/1927240850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c100380 0x7f269c10d040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f268c009910 tx=0x7f268c009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 -- 172.21.15.32:0/1927240850 shutdown_connections 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 --2- 172.21.15.32:0/1927240850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c100380 0x7f269c10d040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 --2- 172.21.15.32:0/1927240850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c0ff9d0 0x7f269c0ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 -- 172.21.15.32:0/1927240850 >> 172.21.15.32:0/1927240850 conn(0x7f269c0fb610 msgr2=0x7f269c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 -- 172.21.15.32:0/1927240850 shutdown_connections 2024-06-03T01:31:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.158+0000 7f26a4617700 1 -- 172.21.15.32:0/1927240850 wait complete. 2024-06-03T01:31:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 Processor -- start 2024-06-03T01:31:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 -- start start 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 0x7f269c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c19f3f0 con 0x7f269c0ff9d0 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.159+0000 7f26a4617700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c1988a0 con 0x7f269c100380 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a23b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 0x7f269c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.32:57930/0 (socket says 172.21.15.32:57930) 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a23b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 0x7f269c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47464/0 (socket says 172.21.15.32:47464) 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 -- 172.21.15.32:0/1837443878 learned_addr learned my addr 172.21.15.32:0/1837443878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 -- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 msgr2=0x7f269c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 0x7f269c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 -- 172.21.15.32:0/1837443878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0095f0 con 0x7f269c100380 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.160+0000 7f26a1bb2700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f268c000c00 tx=0x7f268c0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.161+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268c01d070 con 0x7f269c100380 2024-06-03T01:31:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.161+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c198b80 con 0x7f269c100380 2024-06-03T01:31:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.161+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c1990d0 con 0x7f269c100380 2024-06-03T01:31:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.161+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f268c00f460 con 0x7f269c100380 2024-06-03T01:31:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.161+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268c017610 con 0x7f269c100380 2024-06-03T01:31:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.162+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2680005320 con 0x7f269c100380 2024-06-03T01:31:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.163+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f268c003a60 con 0x7f269c100380 2024-06-03T01:31:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.164+0000 7f26937fe700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 0x7f26880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.164+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f268c08f2d0 con 0x7f269c100380 2024-06-03T01:31:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.164+0000 7f26a23b3700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 0x7f26880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.164+0000 7f26a23b3700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 0x7f26880706c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2698005950 tx=0x7f269800a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.169+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f268c05c020 con 0x7f269c100380 2024-06-03T01:31:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:21 smithi044 ceph-mon[84214]: Upgrade: Updating ceph-exporter.smithi032 (1/2) 2024-06-03T01:31:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:21 smithi044 ceph-mon[84214]: Deploying daemon ceph-exporter.smithi032 on smithi032 2024-06-03T01:31:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:21 smithi044 ceph-mon[84214]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:22.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:21 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2392598631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.406+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2680000bf0 con 0x7f268806e200 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.408+0000 7f26937fe700 1 -- 172.21.15.32:0/1837443878 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f2680000bf0 con 0x7f268806e200 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9", 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-06-03T01:31:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "mds", 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "16/23 daemons upgraded", 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-06-03T01:31:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:31:22.411 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 msgr2=0x7f26880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 0x7f26880706c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2698005950 tx=0x7f269800a300 comp rx=0 tx=0).stop 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 msgr2=0x7f269c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f268c000c00 tx=0x7f268c0055f0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 shutdown_connections 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f268806e200 0x7f26880706c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f269c0ff9d0 0x7f269c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 --2- 172.21.15.32:0/1837443878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f269c100380 0x7f269c19ed60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.411+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 >> 172.21.15.32:0/1837443878 conn(0x7f269c0fb610 msgr2=0x7f269c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.412+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 shutdown_connections 2024-06-03T01:31:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.412+0000 7f26a4617700 1 -- 172.21.15.32:0/1837443878 wait complete. 2024-06-03T01:31:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.543+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/440579397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 msgr2=0x7f6b781070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.543+0000 7f6b7c9b5700 1 --2- 172.21.15.32:0/440579397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b781070b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009910 tx=0x7f6b68009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/440579397 shutdown_connections 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 --2- 172.21.15.32:0/440579397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 --2- 172.21.15.32:0/440579397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b781070b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/440579397 >> 172.21.15.32:0/440579397 conn(0x7f6b78075670 msgr2=0x7f6b78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/440579397 shutdown_connections 2024-06-03T01:31:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.544+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/440579397 wait complete. 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.545+0000 7f6b7c9b5700 1 Processor -- start 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.545+0000 7f6b7c9b5700 1 -- start start 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7c9b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7c9b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7c9b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78100770 con 0x7f6b78106cd0 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7c9b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781008b0 con 0x7f6b78068580 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47496/0 (socket says 172.21.15.32:47496) 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 -- 172.21.15.32:0/1908251018 learned_addr learned my addr 172.21.15.32:0/1908251018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7659c700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 -- 172.21.15.32:0/1908251018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 msgr2=0x7f6b780ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b75d9b700 1 -- 172.21.15.32:0/1908251018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680095f0 con 0x7f6b78106cd0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.546+0000 7f6b7659c700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b75d9b700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d8d0 tx=0x7f6b6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6c0098a0 con 0x7f6b78106cd0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b6c010460 con 0x7f6b78106cd0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6c00f5f0 con 0x7f6b78106cd0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/1908251018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b781a2a60 con 0x7f6b78106cd0 2024-06-03T01:31:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.547+0000 7f6b7c9b5700 1 -- 172.21.15.32:0/1908251018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b781a2e70 con 0x7f6b78106cd0 2024-06-03T01:31:22.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.549+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6b6c00f750 con 0x7f6b78106cd0 2024-06-03T01:31:22.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.549+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c0052f0 con 0x7f6b78106cd0 2024-06-03T01:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.550+0000 7f6b677fe700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 0x7f6b60070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.550+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f6b6c08d6c0 con 0x7f6b78106cd0 2024-06-03T01:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.550+0000 7f6b7659c700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 0x7f6b60070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.551+0000 7f6b7659c700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 0x7f6b60070710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b5c0 tx=0x7f6b6801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.560+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6b6c092060 con 0x7f6b78106cd0 2024-06-03T01:31:22.980 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:22 smithi032 ceph-mon[111919]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:22.981 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:22 smithi032 ceph-mon[111919]: pgmap v222: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 1 op/s 2024-06-03T01:31:22.981 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:22.981 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:22.981 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:22 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/4073140374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:31:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.979+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6b5c0059c0 con 0x7f6b78106cd0 2024-06-03T01:31:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.979+0000 7f6b677fe700 1 -- 172.21.15.32:0/1908251018 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f6b6c057840 con 0x7f6b78106cd0 2024-06-03T01:31:22.981 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-06-03T01:31:22.982 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-03T01:31:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is degraded 2024-06-03T01:31:22.982 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:31:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 msgr2=0x7f6b60070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 0x7f6b60070710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b5c0 tx=0x7f6b6801a040 comp rx=0 tx=0).stop 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 msgr2=0x7f6b78100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d8d0 tx=0x7f6b6c00dbe0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 shutdown_connections 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f6b6006e250 0x7f6b60070710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b78068580 0x7f6b780ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 --2- 172.21.15.32:0/1908251018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78106cd0 0x7f6b78100230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 >> 172.21.15.32:0/1908251018 conn(0x7f6b78075670 msgr2=0x7f6b780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 shutdown_connections 2024-06-03T01:31:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:22.982+0000 7f6b657fa700 1 -- 172.21.15.32:0/1908251018 wait complete. 2024-06-03T01:31:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:22 smithi044 ceph-mon[84214]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:22 smithi044 ceph-mon[84214]: pgmap v222: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 1 op/s 2024-06-03T01:31:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:23.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:22 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/4073140374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:31:23.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:23 smithi032 ceph-mon[111919]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:23.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:23 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1908251018' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:31:23.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:23 smithi044 ceph-mon[84214]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:23.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:23 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1908251018' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:31:25.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:24 smithi032 ceph-mon[111919]: pgmap v223: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-06-03T01:31:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:24 smithi044 ceph-mon[84214]: pgmap v223: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-06-03T01:31:26.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:25 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:26.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:25 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:26.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:25 smithi044 ceph-mon[84214]: pgmap v224: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-06-03T01:31:26.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:25 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:26.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:25 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:26.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:25 smithi032 ceph-mon[111919]: pgmap v224: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-06-03T01:31:27.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: Upgrade: Updating ceph-exporter.smithi044 (2/2) 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:resolve 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:resolve 2024-06-03T01:31:27.722 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:27 smithi044 ceph-mon[84214]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:resolve,1=cephfs.smithi032.tegslf=up:resolve} 2 up:standby 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: Upgrade: Updating ceph-exporter.smithi044 (2/2) 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:resolve 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:resolve 2024-06-03T01:31:27.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:27 smithi032 ceph-mon[111919]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:resolve,1=cephfs.smithi032.tegslf=up:resolve} 2 up:standby 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: pgmap v225: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: reconnect by client.24309 v1:172.21.15.32:0/1770476556 after 0 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: reconnect by client.24309 v1:172.21.15.32:0/1770476556 after 0 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:reconnect 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:reconnect 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:reconnect,1=cephfs.smithi032.tegslf=up:reconnect} 2 up:standby 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: reconnect by client.14530 v1:172.21.15.44:0/1127472913 after 0.00799984 2024-06-03T01:31:28.583 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:28 smithi044 ceph-mon[84214]: reconnect by client.14530 v1:172.21.15.44:0/1127472913 after 0.0149997 2024-06-03T01:31:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: pgmap v225: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-06-03T01:31:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: reconnect by client.24309 v1:172.21.15.32:0/1770476556 after 0 2024-06-03T01:31:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: reconnect by client.24309 v1:172.21.15.32:0/1770476556 after 0 2024-06-03T01:31:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:reconnect 2024-06-03T01:31:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:reconnect 2024-06-03T01:31:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:reconnect,1=cephfs.smithi032.tegslf=up:reconnect} 2 up:standby 2024-06-03T01:31:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: reconnect by client.14530 v1:172.21.15.44:0/1127472913 after 0.00799984 2024-06-03T01:31:28.836 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:28 smithi032 ceph-mon[111919]: reconnect by client.14530 v1:172.21.15.44:0/1127472913 after 0.0149997 2024-06-03T01:31:30.089 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:rejoin 2024-06-03T01:31:30.089 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:rejoin 2024-06-03T01:31:30.089 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:rejoin,1=cephfs.smithi032.tegslf=up:rejoin} 2 up:standby 2024-06-03T01:31:30.090 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: daemon mds.cephfs.smithi032.tegslf is now active in filesystem cephfs as rank 1 2024-06-03T01:31:30.090 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: daemon mds.cephfs.smithi032.itqzrm is now active in filesystem cephfs as rank 0 2024-06-03T01:31:30.090 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:29 smithi044 ceph-mon[84214]: pgmap v226: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8 op/s 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:rejoin 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:rejoin 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: fsmap cephfs:2/2 {0=cephfs.smithi032.itqzrm=up:rejoin,1=cephfs.smithi032.tegslf=up:rejoin} 2 up:standby 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: daemon mds.cephfs.smithi032.tegslf is now active in filesystem cephfs as rank 1 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: daemon mds.cephfs.smithi032.itqzrm is now active in filesystem cephfs as rank 0 2024-06-03T01:31:30.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:29 smithi032 ceph-mon[111919]: pgmap v226: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8 op/s 2024-06-03T01:31:31.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:31:31.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:active 2024-06-03T01:31:31.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:active 2024-06-03T01:31:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:31:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:30 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:31:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] up:active 2024-06-03T01:31:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: mds.? [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] up:active 2024-06-03T01:31:31.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:31:31.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:31.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:31.336 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:30 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:32.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:31 smithi044 ceph-mon[84214]: pgmap v227: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 170 B/s wr, 9 op/s 2024-06-03T01:31:32.313 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:31 smithi032 ceph-mon[111919]: pgmap v227: 65 pgs: 65 active+clean; 316 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 170 B/s wr, 9 op/s 2024-06-03T01:31:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:33 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:33 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.816 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:34 smithi044 ceph-mon[84214]: pgmap v228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-06-03T01:31:34.817 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:34 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.817 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:34 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:34 smithi032 ceph-mon[111919]: pgmap v228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-06-03T01:31:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:34 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:34.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:34 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:36 smithi032 ceph-mon[111919]: pgmap v229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.1 MiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-03T01:31:37.086 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:36 smithi044 ceph-mon[84214]: pgmap v229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.1 MiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: pgmap v230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 15 KiB/s wr, 13 op/s 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:38 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:38.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: pgmap v230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 15 KiB/s wr, 13 op/s 2024-06-03T01:31:38.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.474 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:38.475 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:38 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:39.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:39 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:39.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:39 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:40.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:40 smithi044 ceph-mon[84214]: pgmap v231: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 15 KiB/s wr, 12 op/s 2024-06-03T01:31:40.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:40.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:40 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:40 smithi032 ceph-mon[111919]: pgmap v231: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 15 KiB/s wr, 12 op/s 2024-06-03T01:31:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:40 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:42 smithi044 ceph-mon[84214]: pgmap v232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-06-03T01:31:42.581 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:42 smithi032 ceph-mon[111919]: pgmap v232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:43.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:43.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:43.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:43.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:43.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: Upgrade: Setting filesystem cephfs Joinable 2024-06-03T01:31:43.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:43 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-03T01:31:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:44.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: Upgrade: Setting filesystem cephfs Joinable 2024-06-03T01:31:44.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:43 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: pgmap v233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all ceph-exporter 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi032"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi032"}]': finished 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]': finished 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all iscsi 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all nfs 2024-06-03T01:31:44.973 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Setting container_image for all nvmeof 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Finalizing container_image settings 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-03T01:31:44.974 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-03T01:31:44.975 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: Upgrade: Complete! 2024-06-03T01:31:44.976 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:44.977 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: pgmap v233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: fsmap cephfs:2 {0=cephfs.smithi032.itqzrm=up:active,1=cephfs.smithi032.tegslf=up:active} 2 up:standby 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all ceph-exporter 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi032"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi032"}]': finished 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]: dispatch 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]': finished 2024-06-03T01:31:45.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all iscsi 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all nfs 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Setting container_image for all nvmeof 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:31:45.087 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Finalizing container_image settings 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.088 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: Upgrade: Complete! 2024-06-03T01:31:45.089 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:31:45.090 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:46.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:46 smithi044 ceph-mon[84214]: pgmap v234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 9.7 KiB/s wr, 3 op/s 2024-06-03T01:31:47.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:46 smithi032 ceph-mon[111919]: pgmap v234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 9.7 KiB/s wr, 3 op/s 2024-06-03T01:31:48.320 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:48 smithi032 ceph-mon[111919]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:48.320 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:48 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:48 smithi044 ceph-mon[84214]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:48.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:48 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:31:50.346 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:50 smithi032 ceph-mon[111919]: pgmap v236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:50.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:50 smithi044 ceph-mon[84214]: pgmap v236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:52 smithi044 ceph-mon[84214]: pgmap v237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:52.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:52 smithi032 ceph-mon[111919]: pgmap v237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:52.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:31:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/1766651974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 msgr2=0x7fa5c01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/1766651974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c01047a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009910 tx=0x7fa5b0009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/1766651974 shutdown_connections 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/1766651974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c01047a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/1766651974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 0x7fa5c0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/1766651974 >> 172.21.15.32:0/1766651974 conn(0x7fa5c00fb650 msgr2=0x7fa5c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/1766651974 shutdown_connections 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/1766651974 wait complete. 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 Processor -- start 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.110+0000 7fa5c6dd6700 1 -- start start 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5c6dd6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 0x7fa5c019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5c6dd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5c6dd6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c019d240 con 0x7fa5c0100750 2024-06-03T01:31:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5c6dd6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01966f0 con 0x7fa5c00ffda0 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55990/0 (socket says 172.21.15.32:55990) 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 -- 172.21.15.32:0/997953353 learned_addr learned my addr 172.21.15.32:0/997953353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5c4b72700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 0x7fa5c019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 -- 172.21.15.32:0/997953353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 msgr2=0x7fa5c019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 0x7fa5c019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 -- 172.21.15.32:0/997953353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00095f0 con 0x7fa5c0100750 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bffff700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000bd70 tx=0x7fa5b000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.111+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b001d070 con 0x7fa5c0100750 2024-06-03T01:31:53.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.112+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b0005dc0 con 0x7fa5c0100750 2024-06-03T01:31:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.112+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c0196970 con 0x7fa5c0100750 2024-06-03T01:31:53.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.112+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b0017400 con 0x7fa5c0100750 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.112+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c0196e60 con 0x7fa5c0100750 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.113+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c004f2e0 con 0x7fa5c0100750 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.113+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa5b0017670 con 0x7fa5c0100750 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.114+0000 7fa5bdffb700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 0x7fa5ac0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.114+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fa5b008f730 con 0x7fa5c0100750 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.114+0000 7fa5c4b72700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 0x7fa5ac0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:53.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.114+0000 7fa5c4b72700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 0x7fa5ac0706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8005950 tx=0x7fa5a800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.119+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa5b005c430 con 0x7fa5c0100750 2024-06-03T01:31:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.347+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c01970f0 con 0x7fa5ac06e200 2024-06-03T01:31:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.348+0000 7fa5bdffb700 1 -- 172.21.15.32:0/997953353 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa5c01970f0 con 0x7fa5ac06e200 2024-06-03T01:31:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 msgr2=0x7fa5ac0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 0x7fa5ac0706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8005950 tx=0x7fa5a800b410 comp rx=0 tx=0).stop 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 msgr2=0x7fa5c019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000bd70 tx=0x7fa5b000bda0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 shutdown_connections 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa5ac06e200 0x7fa5ac0706c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00ffda0 0x7fa5c019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 --2- 172.21.15.32:0/997953353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5c0100750 0x7fa5c019cbb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 >> 172.21.15.32:0/997953353 conn(0x7fa5c00fb650 msgr2=0x7fa5c0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 shutdown_connections 2024-06-03T01:31:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:53.350+0000 7fa5c6dd6700 1 -- 172.21.15.32:0/997953353 wait complete. 2024-06-03T01:31:53.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ps' 2024-06-03T01:31:54.214 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:31:54.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:54 smithi032 ceph-mon[111919]: pgmap v238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:54.329 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:54 smithi032 ceph-mon[111919]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:54 smithi044 ceph-mon[84214]: pgmap v238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:54.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:54 smithi044 ceph-mon[84214]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/2245847580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f40ffda0 msgr2=0x7fc5f4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/2245847580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f40ffda0 0x7fc5f4100180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009910 tx=0x7fc5e4009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/2245847580 shutdown_connections 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/2245847580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f4100750 0x7fc5f41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/2245847580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f40ffda0 0x7fc5f4100180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/2245847580 >> 172.21.15.32:0/2245847580 conn(0x7fc5f40fb650 msgr2=0x7fc5f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/2245847580 shutdown_connections 2024-06-03T01:31:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.669+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/2245847580 wait complete. 2024-06-03T01:31:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 Processor -- start 2024-06-03T01:31:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 -- start start 2024-06-03T01:31:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f40ffda0 0x7fc5f419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f419f440 con 0x7fc5f4100750 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.670+0000 7fc5f9c3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f41988f0 con 0x7fc5f40ffda0 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56008/0 (socket says 172.21.15.32:56008) 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 -- 172.21.15.32:0/530651797 learned_addr learned my addr 172.21.15.32:0/530651797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f37fe700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f40ffda0 0x7fc5f419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 -- 172.21.15.32:0/530651797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f40ffda0 msgr2=0x7fc5f419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f40ffda0 0x7fc5f419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 -- 172.21.15.32:0/530651797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40095f0 con 0x7fc5f4100750 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f2ffd700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800d900 tx=0x7fc5e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e8009900 con 0x7fc5f4100750 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5e8010460 con 0x7fc5f4100750 2024-06-03T01:31:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f4198bd0 con 0x7fc5f4100750 2024-06-03T01:31:55.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e800f5f0 con 0x7fc5f4100750 2024-06-03T01:31:55.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.671+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f4199120 con 0x7fc5f4100750 2024-06-03T01:31:55.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.672+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f404f2e0 con 0x7fc5f4100750 2024-06-03T01:31:55.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.673+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5e8009a60 con 0x7fc5f4100750 2024-06-03T01:31:55.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.673+0000 7fc5f0ff9700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:55.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.673+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fc5e808dda0 con 0x7fc5f4100750 2024-06-03T01:31:55.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.673+0000 7fc5f37fe700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:55.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.674+0000 7fc5f37fe700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 0x7fc5dc0706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006010 tx=0x7fc5e401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:55.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.678+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc5e805ab20 con 0x7fc5f4100750 2024-06-03T01:31:55.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.895+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f4199400 con 0x7fc5dc06e200 2024-06-03T01:31:55.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.906+0000 7fc5f0ff9700 1 -- 172.21.15.32:0/530651797 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc5f4199400 con 0x7fc5dc06e200 2024-06-03T01:31:55.906 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (8m) 17s ago 19m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (31s) 17s ago 19m 7344k - 19.0.0-2946-g3d89accf b045ac545f82 1afb2299e663 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (26s) 23s ago 18m 6937k - 19.0.0-2946-g3d89accf b045ac545f82 d96fb558e388 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (5m) 17s ago 19m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (5m) 23s ago 18m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 17s ago 18m 80.5M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (99s) 17s ago 13m 64.1M - 19.0.0-2946-g3d89accf b045ac545f82 aa391c5be983 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (83s) 17s ago 13m 71.9M - 19.0.0-2946-g3d89accf b045ac545f82 bda9b08ca07e 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (64s) 23s ago 13m 15.0M - 19.0.0-2946-g3d89accf b045ac545f82 f5625d49b4b6 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (47s) 23s ago 13m 12.5M - 19.0.0-2946-g3d89accf b045ac545f82 b8eec5ca5f6b 2024-06-03T01:31:55.907 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (10m) 17s ago 20m 511M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (9m) 23s ago 18m 453M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (6m) 17s ago 20m 59.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (6m) 23s ago 17m 46.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 17s ago 19m 15.4M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 23s ago 18m 16.0M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (4m) 17s ago 17m 144M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (4m) 17s ago 16m 132M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (3m) 17s ago 16m 93.7M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (3m) 23s ago 15m 113M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (2m) 23s ago 15m 97.0M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:31:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (117s) 23s ago 15m 100M 2241M 19.0.0-2946-g3d89accf b045ac545f82 a11e0341a922 2024-06-03T01:31:55.909 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (8m) 17s ago 18m 78.4M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:31:55.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 msgr2=0x7fc5dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:55.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 0x7fc5dc0706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006010 tx=0x7fc5e401a040 comp rx=0 tx=0).stop 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 msgr2=0x7fc5f419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800d900 tx=0x7fc5e800dc10 comp rx=0 tx=0).stop 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 shutdown_connections 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f40ffda0 0x7fc5f419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 --2- 172.21.15.32:0/530651797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f4100750 0x7fc5f419edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.908+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 >> 172.21.15.32:0/530651797 conn(0x7fc5f40fb650 msgr2=0x7fc5f4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.909+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 shutdown_connections 2024-06-03T01:31:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:55.909+0000 7fc5f9c3b700 1 -- 172.21.15.32:0/530651797 wait complete. 2024-06-03T01:31:56.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:56 smithi044 ceph-mon[84214]: pgmap v239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:56.567 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:56 smithi032 ceph-mon[111919]: pgmap v239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:56.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch upgrade status' 2024-06-03T01:31:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:31:57.213 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:57 smithi032 ceph-mon[111919]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:57.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:57 smithi044 ceph-mon[84214]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:58.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:58 smithi032 ceph-mon[111919]: pgmap v240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.376+0000 7ff7ec034700 1 -- 172.21.15.32:0/2304855696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 msgr2=0x7ff7e41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.376+0000 7ff7ec034700 1 --2- 172.21.15.32:0/2304855696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e41047b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009910 tx=0x7ff7d4009c20 comp rx=0 tx=0).stop 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 -- 172.21.15.32:0/2304855696 shutdown_connections 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 --2- 172.21.15.32:0/2304855696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e41047b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 --2- 172.21.15.32:0/2304855696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 0x7ff7e4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 -- 172.21.15.32:0/2304855696 >> 172.21.15.32:0/2304855696 conn(0x7ff7e4074e30 msgr2=0x7ff7e4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 -- 172.21.15.32:0/2304855696 shutdown_connections 2024-06-03T01:31:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 -- 172.21.15.32:0/2304855696 wait complete. 2024-06-03T01:31:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 Processor -- start 2024-06-03T01:31:58.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 -- start start 2024-06-03T01:31:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 0x7ff7e4101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.377+0000 7ff7ec034700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7ec034700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e41022d0 con 0x7ff7e4101130 2024-06-03T01:31:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7ec034700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4102410 con 0x7ff7e4100780 2024-06-03T01:31:58.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32860/0 (socket says 172.21.15.32:32860) 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 -- 172.21.15.32:0/452395448 learned_addr learned my addr 172.21.15.32:0/452395448 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e9dd0700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 0x7ff7e4101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 -- 172.21.15.32:0/452395448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 msgr2=0x7ff7e4101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 0x7ff7e4101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 -- 172.21.15.32:0/452395448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40095f0 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7e95cf700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4005f50 tx=0x7ff7d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d401d070 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7d400f460 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e4102690 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d4017630 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.378+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e41a2eb0 con 0x7ff7e4101130 2024-06-03T01:31:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.379+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e404f2e0 con 0x7ff7e4101130 2024-06-03T01:31:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.380+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff7d400f5d0 con 0x7ff7e4101130 2024-06-03T01:31:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.380+0000 7ff7daffd700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 0x7ff7d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:31:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.380+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7ff7d408f7e0 con 0x7ff7e4101130 2024-06-03T01:31:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.380+0000 7ff7e9dd0700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 0x7ff7d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:31:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.381+0000 7ff7e9dd0700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 0x7ff7d00706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:31:58.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.385+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff7d4058a30 con 0x7ff7e4101130 2024-06-03T01:31:58.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:58 smithi044 ceph-mon[84214]: pgmap v240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:31:58.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.626+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e4103970 con 0x7ff7d006e200 2024-06-03T01:31:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.627+0000 7ff7daffd700 1 -- 172.21.15.32:0/452395448 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff7e4103970 con 0x7ff7d006e200 2024-06-03T01:31:58.627 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:31:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": null, 2024-06-03T01:31:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": false, 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "", 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": null, 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-06-03T01:31:58.628 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:31:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 msgr2=0x7ff7d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 0x7ff7d00706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-06-03T01:31:58.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 msgr2=0x7ff7e4101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4005f50 tx=0x7ff7d400be20 comp rx=0 tx=0).stop 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 shutdown_connections 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff7d006e200 0x7ff7d00706c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7e4100780 0x7ff7e4101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 --2- 172.21.15.32:0/452395448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7e4101130 0x7ff7e4101d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.629+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 >> 172.21.15.32:0/452395448 conn(0x7ff7e4074e30 msgr2=0x7ff7e40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.630+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 shutdown_connections 2024-06-03T01:31:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:31:58.630+0000 7ff7ec034700 1 -- 172.21.15.32:0/452395448 wait complete. 2024-06-03T01:31:59.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:31:59 smithi044 ceph-mon[84214]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:59.566 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:31:59 smithi032 ceph-mon[111919]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:31:59.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph health detail' 2024-06-03T01:31:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:32:00.219 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:00 smithi032 ceph-mon[111919]: pgmap v241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:00.373 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:00 smithi044 ceph-mon[84214]: pgmap v241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 -- 172.21.15.32:0/1961331170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 msgr2=0x7f58681058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 --2- 172.21.15.32:0/1961331170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f58681058a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f585c009910 tx=0x7f585c009c20 comp rx=0 tx=0).stop 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 -- 172.21.15.32:0/1961331170 shutdown_connections 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 --2- 172.21.15.32:0/1961331170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f58681058a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 --2- 172.21.15.32:0/1961331170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 0x7f5868101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 -- 172.21.15.32:0/1961331170 >> 172.21.15.32:0/1961331170 conn(0x7f58680fc750 msgr2=0x7f58680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 -- 172.21.15.32:0/1961331170 shutdown_connections 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.492+0000 7f5870651700 1 -- 172.21.15.32:0/1961331170 wait complete. 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 Processor -- start 2024-06-03T01:32:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 -- start start 2024-06-03T01:32:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 0x7f586819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586819f430 con 0x7f5868101850 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f5870651700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681988e0 con 0x7f5868100ea0 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32884/0 (socket says 172.21.15.32:32884) 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 -- 172.21.15.32:0/654835756 learned_addr learned my addr 172.21.15.32:0/654835756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586e3ed700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 0x7f586819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 -- 172.21.15.32:0/654835756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 msgr2=0x7f586819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 0x7f586819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.493+0000 7f586dbec700 1 -- 172.21.15.32:0/654835756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0095f0 con 0x7f5868101850 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f586dbec700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f585c0092e0 tx=0x7f585c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c01d070 con 0x7f5868101850 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585c005dc0 con 0x7f5868101850 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c017400 con 0x7f5868101850 2024-06-03T01:32:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5868198b60 con 0x7f5868101850 2024-06-03T01:32:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.494+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5868199050 con 0x7f5868101850 2024-06-03T01:32:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.495+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5868105020 con 0x7f5868101850 2024-06-03T01:32:01.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.496+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f585c00f6f0 con 0x7f5868101850 2024-06-03T01:32:01.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.496+0000 7f585b7fe700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 0x7f58540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:01.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.496+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f585c08f5b0 con 0x7f5868101850 2024-06-03T01:32:01.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.496+0000 7f586e3ed700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 0x7f58540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:01.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.497+0000 7f586e3ed700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 0x7f58540706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5864005950 tx=0x7f586400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:32:01.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.501+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f585c05c330 con 0x7f5868101850 2024-06-03T01:32:01.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.795+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5868104a00 con 0x7f5868101850 2024-06-03T01:32:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.796+0000 7f585b7fe700 1 -- 172.21.15.32:0/654835756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f585c05bec0 con 0x7f5868101850 2024-06-03T01:32:01.796 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-06-03T01:32:01.796 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-06-03T01:32:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 msgr2=0x7f58540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 0x7f58540706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5864005950 tx=0x7f586400b410 comp rx=0 tx=0).stop 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 msgr2=0x7f586819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f585c0092e0 tx=0x7f585c00bd70 comp rx=0 tx=0).stop 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 shutdown_connections 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f585406e200 0x7f58540706c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868100ea0 0x7f586819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 --2- 172.21.15.32:0/654835756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5868101850 0x7f586819eda0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 >> 172.21.15.32:0/654835756 conn(0x7f58680fc750 msgr2=0x7f58680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:32:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 shutdown_connections 2024-06-03T01:32:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:01.798+0000 7f5870651700 1 -- 172.21.15.32:0/654835756 wait complete. 2024-06-03T01:32:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:02 smithi044 ceph-mon[84214]: pgmap v242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:02.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:02 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/654835756' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:32:02.489 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:02 smithi032 ceph-mon[111919]: pgmap v242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:02.489 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:02 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/654835756' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-03T01:32:02.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions' 2024-06-03T01:32:02.728 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:32:04.324 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:04 smithi032 ceph-mon[111919]: pgmap v243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 -- 172.21.15.32:0/3713909669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 msgr2=0x7fb2dc1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 --2- 172.21.15.32:0/3713909669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc1047b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009910 tx=0x7fb2d0009c20 comp rx=0 tx=0).stop 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 -- 172.21.15.32:0/3713909669 shutdown_connections 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 --2- 172.21.15.32:0/3713909669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc1047b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 --2- 172.21.15.32:0/3713909669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 0x7fb2dc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 -- 172.21.15.32:0/3713909669 >> 172.21.15.32:0/3713909669 conn(0x7fb2dc074e30 msgr2=0x7fb2dc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 -- 172.21.15.32:0/3713909669 shutdown_connections 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.345+0000 7fb2e4661700 1 -- 172.21.15.32:0/3713909669 wait complete. 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 Processor -- start 2024-06-03T01:32:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 -- start start 2024-06-03T01:32:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 0x7fb2dc101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1022d0 con 0x7fb2dc101060 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e4661700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc102410 con 0x7fb2dc1006b0 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e1bfc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e1bfc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32898/0 (socket says 172.21.15.32:32898) 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e1bfc700 1 -- 172.21.15.32:0/614050044 learned_addr learned my addr 172.21.15.32:0/614050044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.346+0000 7fb2e23fd700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 0x7fb2dc101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e1bfc700 1 -- 172.21.15.32:0/614050044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 msgr2=0x7fb2dc101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e1bfc700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 0x7fb2dc101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e1bfc700 1 -- 172.21.15.32:0/614050044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00095f0 con 0x7fb2dc101060 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e1bfc700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0005f50 tx=0x7fb2d000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:32:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d001d070 con 0x7fb2dc101060 2024-06-03T01:32:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc102690 con 0x7fb2dc101060 2024-06-03T01:32:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d000f460 con 0x7fb2dc101060 2024-06-03T01:32:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d0017630 con 0x7fb2dc101060 2024-06-03T01:32:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.347+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc1a2e70 con 0x7fb2dc101060 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.348+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc109150 con 0x7fb2dc101060 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.349+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb2d000f5d0 con 0x7fb2dc101060 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.349+0000 7fb2db7fe700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 0x7fb2c40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.349+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fb2d008f7f0 con 0x7fb2dc101060 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.349+0000 7fb2e23fd700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 0x7fb2c40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:32:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.350+0000 7fb2e23fd700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 0x7fb2c40706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00a400 tx=0x7fb2cc00d040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:32:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.354+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb2d00589c0 con 0x7fb2dc101060 2024-06-03T01:32:04.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:04 smithi044 ceph-mon[84214]: pgmap v243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.649+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2dc04f2e0 con 0x7fb2dc101060 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.650+0000 7fb2db7fe700 1 -- 172.21.15.32:0/614050044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fb2d005c080 con 0x7fb2dc101060 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:32:04.650 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 14 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:32:04.651 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 msgr2=0x7fb2c40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 0x7fb2c40706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00a400 tx=0x7fb2cc00d040 comp rx=0 tx=0).stop 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 msgr2=0x7fb2dc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0005f50 tx=0x7fb2d000bd70 comp rx=0 tx=0).stop 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 shutdown_connections 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb2c406e200 0x7fb2c40706c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2dc1006b0 0x7fb2dc101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 --2- 172.21.15.32:0/614050044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2dc101060 0x7fb2dc101d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:32:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 >> 172.21.15.32:0/614050044 conn(0x7fb2dc074e30 msgr2=0x7fb2dc103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:32:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 shutdown_connections 2024-06-03T01:32:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:32:04.652+0000 7fb2e4661700 1 -- 172.21.15.32:0/614050044 wait complete. 2024-06-03T01:32:05.262 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:05 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/614050044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:32:05.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-06-03T01:32:05.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:05 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/614050044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:32:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:32:06.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:06 smithi032 ceph-mon[111919]: pgmap v244: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:06.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:06 smithi044 ceph-mon[84214]: pgmap v244: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:06.981 INFO:teuthology.orchestra.run.smithi032.stdout:wait for servicemap items w/ changing names to refresh 2024-06-03T01:32:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:07.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:07.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'sleep 60' 2024-06-03T01:32:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:32:08.222 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:08 smithi032 ceph-mon[111919]: pgmap v245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:08 smithi044 ceph-mon[84214]: pgmap v245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:10.333 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:10 smithi032 ceph-mon[111919]: pgmap v246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:10 smithi044 ceph-mon[84214]: pgmap v246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:12 smithi044 ceph-mon[84214]: pgmap v247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:12.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:12 smithi032 ceph-mon[111919]: pgmap v247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:14 smithi044 ceph-mon[84214]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:14.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:14 smithi032 ceph-mon[111919]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:16 smithi044 ceph-mon[84214]: pgmap v249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:16.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:16 smithi032 ceph-mon[111919]: pgmap v249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:18.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:18 smithi032 ceph-mon[111919]: pgmap v250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:18.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:18 smithi044 ceph-mon[84214]: pgmap v250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:20 smithi044 ceph-mon[84214]: pgmap v251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:20.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:20 smithi032 ceph-mon[111919]: pgmap v251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:22 smithi044 ceph-mon[84214]: pgmap v252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:22.472 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:22 smithi032 ceph-mon[111919]: pgmap v252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:22.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:24 smithi044 ceph-mon[84214]: pgmap v253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:24.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:24 smithi032 ceph-mon[111919]: pgmap v253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:26.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:26 smithi032 ceph-mon[111919]: pgmap v254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:26 smithi044 ceph-mon[84214]: pgmap v254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:28 smithi032 ceph-mon[111919]: pgmap v255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:28.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:28 smithi044 ceph-mon[84214]: pgmap v255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:30.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:30 smithi044 ceph-mon[84214]: pgmap v256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:30.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:30 smithi032 ceph-mon[111919]: pgmap v256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:32.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:32 smithi044 ceph-mon[84214]: pgmap v257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:32.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:32 smithi032 ceph-mon[111919]: pgmap v257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:34.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:34 smithi044 ceph-mon[84214]: pgmap v258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:34.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:34 smithi032 ceph-mon[111919]: pgmap v258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:36.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:36 smithi044 ceph-mon[84214]: pgmap v259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:36.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:36 smithi032 ceph-mon[111919]: pgmap v259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:37.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:37.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:38.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:38 smithi032 ceph-mon[111919]: pgmap v260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:38.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:38 smithi044 ceph-mon[84214]: pgmap v260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:40.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:40 smithi044 ceph-mon[84214]: pgmap v261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:40.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:40 smithi032 ceph-mon[111919]: pgmap v261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:42.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:42 smithi044 ceph-mon[84214]: pgmap v262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:42.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:42 smithi032 ceph-mon[111919]: pgmap v262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:44 smithi044 ceph-mon[84214]: pgmap v263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:44.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:44 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:32:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:44 smithi032 ceph-mon[111919]: pgmap v263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:44.585 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:44 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:32:45.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:32:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:32:45.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:46.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:32:46.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:32:46.086 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:32:46.721 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:46 smithi044 ceph-mon[84214]: pgmap v264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:46.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:46 smithi032 ceph-mon[111919]: pgmap v264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:48.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:48 smithi044 ceph-mon[84214]: pgmap v265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:49.077 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:48 smithi032 ceph-mon[111919]: pgmap v265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:50.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:50 smithi044 ceph-mon[84214]: pgmap v266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:51.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:50 smithi032 ceph-mon[111919]: pgmap v266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:52.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:52 smithi044 ceph-mon[84214]: pgmap v267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:52.972 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:52 smithi032 ceph-mon[111919]: pgmap v267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:53.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:32:54.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:54 smithi044 ceph-mon[84214]: pgmap v268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:55.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:54 smithi032 ceph-mon[111919]: pgmap v268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:56.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:56 smithi044 ceph-mon[84214]: pgmap v269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:57.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:56 smithi032 ceph-mon[111919]: pgmap v269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:58.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:32:58 smithi044 ceph-mon[84214]: pgmap v270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:32:59.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:32:58 smithi032 ceph-mon[111919]: pgmap v270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:00.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:00 smithi044 ceph-mon[84214]: pgmap v271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:00 smithi032 ceph-mon[111919]: pgmap v271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:02.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:02 smithi044 ceph-mon[84214]: pgmap v272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:03.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:02 smithi032 ceph-mon[111919]: pgmap v272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:04.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:04 smithi044 ceph-mon[84214]: pgmap v273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:05.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:04 smithi032 ceph-mon[111919]: pgmap v273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:06.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:06 smithi044 ceph-mon[84214]: pgmap v274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:07.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:06 smithi032 ceph-mon[111919]: pgmap v274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:07.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:08.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:08.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:08 smithi044 ceph-mon[84214]: pgmap v275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:09.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:08 smithi032 ceph-mon[111919]: pgmap v275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:10.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph orch ps' 2024-06-03T01:33:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:10.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:10 smithi044 ceph-mon[84214]: pgmap v276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:11.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:10 smithi032 ceph-mon[111919]: pgmap v276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.049+0000 7efe27e5e700 1 -- 172.21.15.32:0/3745087742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20100ea0 msgr2=0x7efe20101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.049+0000 7efe27e5e700 1 --2- 172.21.15.32:0/3745087742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20100ea0 0x7efe20101280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b0a0 tx=0x7efe1c00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 -- 172.21.15.32:0/3745087742 shutdown_connections 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 --2- 172.21.15.32:0/3745087742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20101850 0x7efe201058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 --2- 172.21.15.32:0/3745087742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20100ea0 0x7efe20101280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 -- 172.21.15.32:0/3745087742 >> 172.21.15.32:0/3745087742 conn(0x7efe200fc750 msgr2=0x7efe200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 -- 172.21.15.32:0/3745087742 shutdown_connections 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 -- 172.21.15.32:0/3745087742 wait complete. 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 Processor -- start 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.050+0000 7efe27e5e700 1 -- start start 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe27e5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20100ea0 0x7efe2019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe27e5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe27e5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2019d3b0 con 0x7efe20101850 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe27e5e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20196860 con 0x7efe20100ea0 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42840/0 (socket says 172.21.15.32:42840) 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 -- 172.21.15.32:0/2977542453 learned_addr learned my addr 172.21.15.32:0/2977542453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe25bfa700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20100ea0 0x7efe2019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 -- 172.21.15.32:0/2977542453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20100ea0 msgr2=0x7efe2019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20100ea0 0x7efe2019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 -- 172.21.15.32:0/2977542453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c009d00 con 0x7efe20101850 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.051+0000 7efe253f9700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b480 tx=0x7efe1000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.052+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe10011800 con 0x7efe20101850 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.052+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe10011e20 con 0x7efe20101850 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.052+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe20196b40 con 0x7efe20101850 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.052+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe10010ea0 con 0x7efe20101850 2024-06-03T01:33:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.052+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe20197090 con 0x7efe20101850 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.053+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe20109240 con 0x7efe20101850 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.053+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe10010380 con 0x7efe20101850 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.054+0000 7efe16ffd700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 0x7efe0c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.054+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7efe1008dda0 con 0x7efe20101850 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.054+0000 7efe25bfa700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 0x7efe0c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.054+0000 7efe25bfa700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 0x7efe0c0706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe1c000f80 tx=0x7efe1c0097e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.059+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efe1005aaa0 con 0x7efe20101850 2024-06-03T01:33:12.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.299+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 --> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe20197370 con 0x7efe0c06e200 2024-06-03T01:33:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.311+0000 7efe16ffd700 1 -- 172.21.15.32:0/2977542453 <== mgr.34102 v2:172.21.15.32:6800/2336714546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efe20197370 con 0x7efe0c06e200 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (9m) 94s ago 20m 25.6M - 0.25.0 c8568f914cd2 f59023e086aa 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi032 smithi032 running (107s) 94s ago 20m 7344k - 19.0.0-2946-g3d89accf b045ac545f82 1afb2299e663 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-exporter.smithi044 smithi044 running (102s) 99s ago 19m 6937k - 19.0.0-2946-g3d89accf b045ac545f82 d96fb558e388 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (6m) 94s ago 20m 6669k - 19.0.0-2946-g3d89accf b045ac545f82 22ec23134533 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi044 smithi044 running (6m) 99s ago 19m 6665k - 19.0.0-2946-g3d89accf b045ac545f82 5990ab7cbd58 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (9m) 94s ago 20m 80.5M - 9.4.12 2bacad6d85d8 74199d6a3861 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.itqzrm smithi032 running (2m) 94s ago 14m 64.1M - 19.0.0-2946-g3d89accf b045ac545f82 aa391c5be983 2024-06-03T01:33:12.313 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.tegslf smithi032 running (2m) 94s ago 14m 71.9M - 19.0.0-2946-g3d89accf b045ac545f82 bda9b08ca07e 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.gkcyfr smithi044 running (2m) 99s ago 14m 15.0M - 19.0.0-2946-g3d89accf b045ac545f82 f5625d49b4b6 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi044.mjxnhh smithi044 running (2m) 99s ago 14m 12.5M - 19.0.0-2946-g3d89accf b045ac545f82 b8eec5ca5f6b 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.kfcfaa smithi032 *:8443,9283,8765 running (11m) 94s ago 21m 511M - 19.0.0-2946-g3d89accf b045ac545f82 84bf41ca6f7c 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi044.wszjlw smithi044 *:8443,9283,8765 running (10m) 99s ago 19m 453M - 19.0.0-2946-g3d89accf b045ac545f82 c13cc6d8b2cf 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (7m) 94s ago 21m 59.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d0b6e606b120 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi044 smithi044 running (7m) 99s ago 19m 46.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5c5eb244c04e 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (10m) 94s ago 20m 15.4M - 1.5.0 0da6a335fe13 13b990a44f66 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi044 smithi044 *:9100 running (10m) 99s ago 19m 16.0M - 1.5.0 0da6a335fe13 e2c6154cd63a 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) 94s ago 18m 144M 1217M 19.0.0-2946-g3d89accf b045ac545f82 d2cd0ee1b637 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 94s ago 18m 132M 1217M 19.0.0-2946-g3d89accf b045ac545f82 35e7b6d66100 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 94s ago 17m 93.7M 1217M 19.0.0-2946-g3d89accf b045ac545f82 0af7c9cdf9fa 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi044 running (4m) 99s ago 17m 113M 2241M 19.0.0-2946-g3d89accf b045ac545f82 8c54ffef38b6 2024-06-03T01:33:12.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi044 running (3m) 99s ago 16m 97.0M 2241M 19.0.0-2946-g3d89accf b045ac545f82 d71ecbc3e1ba 2024-06-03T01:33:12.315 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi044 running (3m) 99s ago 16m 100M 2241M 19.0.0-2946-g3d89accf b045ac545f82 a11e0341a922 2024-06-03T01:33:12.315 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (9m) 94s ago 19m 78.4M - 2.43.0 a07b618ecd1d 763daebb12c2 2024-06-03T01:33:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 msgr2=0x7efe0c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 0x7efe0c0706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe1c000f80 tx=0x7efe1c0097e0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 msgr2=0x7efe2019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b480 tx=0x7efe1000b790 comp rx=0 tx=0).stop 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 shutdown_connections 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7efe0c06e200 0x7efe0c0706c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe20100ea0 0x7efe2019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 --2- 172.21.15.32:0/2977542453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe20101850 0x7efe2019cd20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 >> 172.21.15.32:0/2977542453 conn(0x7efe200fc750 msgr2=0x7efe200fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 shutdown_connections 2024-06-03T01:33:12.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:12.313+0000 7efe27e5e700 1 -- 172.21.15.32:0/2977542453 wait complete. 2024-06-03T01:33:12.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:12 smithi032 ceph-mon[111919]: pgmap v277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:12.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:12 smithi044 ceph-mon[84214]: pgmap v277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:13.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions' 2024-06-03T01:33:13.232 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:13.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:13 smithi044 ceph-mon[84214]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:33:14.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:13 smithi032 ceph-mon[111919]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 -- 172.21.15.32:0/3412038097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c101e00 msgr2=0x7fb36c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3412038097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c101e00 0x7fb36c1021e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb368009910 tx=0x7fb368009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 -- 172.21.15.32:0/3412038097 shutdown_connections 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3412038097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c102720 0x7fb36c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3412038097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c101e00 0x7fb36c1021e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 -- 172.21.15.32:0/3412038097 >> 172.21.15.32:0/3412038097 conn(0x7fb36c0fb650 msgr2=0x7fb36c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 -- 172.21.15.32:0/3412038097 shutdown_connections 2024-06-03T01:33:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.895+0000 7fb373f0c700 1 -- 172.21.15.32:0/3412038097 wait complete. 2024-06-03T01:33:14.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 Processor -- start 2024-06-03T01:33:14.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 -- start start 2024-06-03T01:33:14.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c101e00 0x7fb36c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c19d1d0 con 0x7fb36c102720 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb373f0c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c196710 con 0x7fb36c101e00 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb3714a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb3714a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42864/0 (socket says 172.21.15.32:42864) 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb3714a7700 1 -- 172.21.15.32:0/3442398566 learned_addr learned my addr 172.21.15.32:0/3442398566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.896+0000 7fb371ca8700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c101e00 0x7fb36c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb3714a7700 1 -- 172.21.15.32:0/3442398566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c101e00 msgr2=0x7fb36c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb3714a7700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c101e00 0x7fb36c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb3714a7700 1 -- 172.21.15.32:0/3442398566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680095f0 con 0x7fb36c102720 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb3714a7700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c8f0 tx=0x7fb35c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb35c0041f0 con 0x7fb36c102720 2024-06-03T01:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb35c007dc0 con 0x7fb36c102720 2024-06-03T01:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c1969f0 con 0x7fb36c102720 2024-06-03T01:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb35c003ea0 con 0x7fb36c102720 2024-06-03T01:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.897+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c196f40 con 0x7fb36c102720 2024-06-03T01:33:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.898+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c108310 con 0x7fb36c102720 2024-06-03T01:33:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.898+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb35c004350 con 0x7fb36c102720 2024-06-03T01:33:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.899+0000 7fb362ffd700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 0x7fb3580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.899+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fb35c08df40 con 0x7fb36c102720 2024-06-03T01:33:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.899+0000 7fb371ca8700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 0x7fb3580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.900+0000 7fb371ca8700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 0x7fb3580706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb36800b5c0 tx=0x7fb36801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:14.904+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb35c0c2850 con 0x7fb36c102720 2024-06-03T01:33:14.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:14 smithi044 ceph-mon[84214]: pgmap v278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:15.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:14 smithi032 ceph-mon[111919]: pgmap v278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.194+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb36c04f2e0 con 0x7fb36c102720 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.195+0000 7fb362ffd700 1 -- 172.21.15.32:0/3442398566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7fb35c01d280 con 0x7fb36c102720 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-06-03T01:33:15.195 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 2 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 6 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 4 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)": 14 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-06-03T01:33:15.196 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 msgr2=0x7fb3580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 0x7fb3580706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb36800b5c0 tx=0x7fb36801a040 comp rx=0 tx=0).stop 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 msgr2=0x7fb36c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c8f0 tx=0x7fb35c00ccb0 comp rx=0 tx=0).stop 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 shutdown_connections 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb35806e200 0x7fb3580706c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb36c101e00 0x7fb36c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 --2- 172.21.15.32:0/3442398566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb36c102720 0x7fb36c19cbd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 >> 172.21.15.32:0/3442398566 conn(0x7fb36c0fb650 msgr2=0x7fb36c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 shutdown_connections 2024-06-03T01:33:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:15.197+0000 7fb373f0c700 1 -- 172.21.15.32:0/3442398566 wait complete. 2024-06-03T01:33:15.766 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:15 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3442398566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:15.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-06-03T01:33:15.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:15 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3442398566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:16.007 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:16.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:16 smithi032 ceph-mon[111919]: pgmap v279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:16.956 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:16 smithi044 ceph-mon[84214]: pgmap v279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 -- 172.21.15.32:0/2142630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec101850 msgr2=0x7feaec1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 --2- 172.21.15.32:0/2142630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec101850 0x7feaec1058a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feadc009910 tx=0x7feadc009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 -- 172.21.15.32:0/2142630925 shutdown_connections 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 --2- 172.21.15.32:0/2142630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec101850 0x7feaec1058a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 --2- 172.21.15.32:0/2142630925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec100ea0 0x7feaec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 -- 172.21.15.32:0/2142630925 >> 172.21.15.32:0/2142630925 conn(0x7feaec0fc750 msgr2=0x7feaec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 -- 172.21.15.32:0/2142630925 shutdown_connections 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.690+0000 7feaf40b1700 1 -- 172.21.15.32:0/2142630925 wait complete. 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.691+0000 7feaf40b1700 1 Processor -- start 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.691+0000 7feaf40b1700 1 -- start start 2024-06-03T01:33:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf40b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf40b1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101850 0x7feaec0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf40b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec073b10 con 0x7feaec100ea0 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf40b1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec073c50 con 0x7feaec101850 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37698/0 (socket says 172.21.15.32:37698) 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 -- 172.21.15.32:0/4023555116 learned_addr learned my addr 172.21.15.32:0/4023555116 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf164c700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101850 0x7feaec0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 -- 172.21.15.32:0/4023555116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101850 msgr2=0x7feaec0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101850 0x7feaec0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 -- 172.21.15.32:0/4023555116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc0095f0 con 0x7feaec100ea0 2024-06-03T01:33:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf1e4d700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feae800c930 tx=0x7feae800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae8007ad0 con 0x7feaec100ea0 2024-06-03T01:33:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec1a2a40 con 0x7feaec100ea0 2024-06-03T01:33:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae800ce80 con 0x7feaec100ea0 2024-06-03T01:33:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.692+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae8018630 con 0x7feaec100ea0 2024-06-03T01:33:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.693+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec1a2da0 con 0x7feaec100ea0 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.693+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec1092a0 con 0x7feaec100ea0 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.694+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feae8007c30 con 0x7feaec100ea0 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.694+0000 7feae2ffd700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 0x7fead80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.695+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7feae808e2f0 con 0x7feaec100ea0 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.695+0000 7feaf164c700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 0x7fead80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.695+0000 7feaf164c700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 0x7fead80706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feaec074880 tx=0x7feadc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:17.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.699+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feae80574c0 con 0x7feaec100ea0 2024-06-03T01:33:17.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.993+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaec108c80 con 0x7feaec100ea0 2024-06-03T01:33:17.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.993+0000 7feae2ffd700 1 -- 172.21.15.32:0/4023555116 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7feae805ab80 con 0x7feaec100ea0 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.995+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 msgr2=0x7fead80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 0x7fead80706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feaec074880 tx=0x7feadc01a040 comp rx=0 tx=0).stop 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 msgr2=0x7feaec073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feae800c930 tx=0x7feae800cc40 comp rx=0 tx=0).stop 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 shutdown_connections 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fead806e200 0x7fead80706c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaec100ea0 0x7feaec073090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 --2- 172.21.15.32:0/4023555116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaec101850 0x7feaec0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 >> 172.21.15.32:0/4023555116 conn(0x7feaec0fc750 msgr2=0x7feaec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 shutdown_connections 2024-06-03T01:33:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:17.996+0000 7feaf40b1700 1 -- 172.21.15.32:0/4023555116 wait complete. 2024-06-03T01:33:18.006 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-06-03T01:33:18.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-06-03T01:33:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:18.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:18 smithi032 ceph-mon[111919]: pgmap v280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:18.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:18 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/4023555116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:19.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:18 smithi044 ceph-mon[84214]: pgmap v280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:19.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:18 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/4023555116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:20.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 -- 172.21.15.32:0/2465831638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 msgr2=0x7ff17c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 --2- 172.21.15.32:0/2465831638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c101270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00b0a0 tx=0x7ff16c00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 -- 172.21.15.32:0/2465831638 shutdown_connections 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 --2- 172.21.15.32:0/2465831638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 0x7ff17c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 --2- 172.21.15.32:0/2465831638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c101270 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 -- 172.21.15.32:0/2465831638 >> 172.21.15.32:0/2465831638 conn(0x7ff17c0fc740 msgr2=0x7ff17c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 -- 172.21.15.32:0/2465831638 shutdown_connections 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.332+0000 7ff182108700 1 -- 172.21.15.32:0/2465831638 wait complete. 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 Processor -- start 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 -- start start 2024-06-03T01:33:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 0x7ff17c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c073b10 con 0x7ff17c100e90 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff182108700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c073c50 con 0x7ff17c101840 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37712/0 (socket says 172.21.15.32:37712) 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.333+0000 7ff17b7fe700 1 -- 172.21.15.32:0/3717771005 learned_addr learned my addr 172.21.15.32:0/3717771005 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff17affd700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 0x7ff17c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff17b7fe700 1 -- 172.21.15.32:0/3717771005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 msgr2=0x7ff17c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff17b7fe700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 0x7ff17c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff17b7fe700 1 -- 172.21.15.32:0/3717771005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c009d00 con 0x7ff17c100e90 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff17b7fe700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f80 tx=0x7ff16c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff16c00e040 con 0x7ff17c100e90 2024-06-03T01:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff16c003c10 con 0x7ff17c100e90 2024-06-03T01:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c073ed0 con 0x7ff17c100e90 2024-06-03T01:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff16c01d6b0 con 0x7ff17c100e90 2024-06-03T01:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.334+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c1a2db0 con 0x7ff17c100e90 2024-06-03T01:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.335+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c109230 con 0x7ff17c100e90 2024-06-03T01:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.336+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff16c004040 con 0x7ff17c100e90 2024-06-03T01:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.336+0000 7ff178ff9700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 0x7ff1640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.336+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7ff16c08fa20 con 0x7ff17c100e90 2024-06-03T01:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.336+0000 7ff17affd700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 0x7ff1640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.337+0000 7ff17affd700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 0x7ff1640706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff17c074880 tx=0x7ff17000b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:20.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.341+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff16c058bf0 con 0x7ff17c100e90 2024-06-03T01:33:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.632+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff17c074660 con 0x7ff17c100e90 2024-06-03T01:33:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.633+0000 7ff178ff9700 1 -- 172.21.15.32:0/3717771005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff16c05c2b0 con 0x7ff17c100e90 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 msgr2=0x7ff1640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 0x7ff1640706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff17c074880 tx=0x7ff17000b3c0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 msgr2=0x7ff17c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f80 tx=0x7ff16c00cd20 comp rx=0 tx=0).stop 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 shutdown_connections 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff16406e200 0x7ff1640706c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff17c100e90 0x7ff17c073090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 --2- 172.21.15.32:0/3717771005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff17c101840 0x7ff17c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:20.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 >> 172.21.15.32:0/3717771005 conn(0x7ff17c0fc740 msgr2=0x7ff17c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:20.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 shutdown_connections 2024-06-03T01:33:20.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:20.635+0000 7ff182108700 1 -- 172.21.15.32:0/3717771005 wait complete. 2024-06-03T01:33:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)" 2024-06-03T01:33:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:20 smithi032 ceph-mon[111919]: pgmap v281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:21.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:20 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3717771005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:21.112 DEBUG:teuthology.parallel:result is None 2024-06-03T01:33:21.112 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:33:21.125 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:33:21.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T01:33:21.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:20 smithi044 ceph-mon[84214]: pgmap v281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:21.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:20 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3717771005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-03T01:33:21.344 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 -- 172.21.15.32:0/1950040231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 msgr2=0x7fdc80101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 --2- 172.21.15.32:0/1950040231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc80101280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009910 tx=0x7fdc74009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 -- 172.21.15.32:0/1950040231 shutdown_connections 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 --2- 172.21.15.32:0/1950040231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc801058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 --2- 172.21.15.32:0/1950040231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc80101280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.989+0000 7fdc86e18700 1 -- 172.21.15.32:0/1950040231 >> 172.21.15.32:0/1950040231 conn(0x7fdc800fc750 msgr2=0x7fdc800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.990+0000 7fdc86e18700 1 -- 172.21.15.32:0/1950040231 shutdown_connections 2024-06-03T01:33:22.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.990+0000 7fdc86e18700 1 -- 172.21.15.32:0/1950040231 wait complete. 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.990+0000 7fdc86e18700 1 Processor -- start 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.990+0000 7fdc86e18700 1 -- start start 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc86e18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc86e18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc8019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc86e18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8019d2c0 con 0x7fdc80100ea0 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc86e18700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80196770 con 0x7fdc80101850 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37720/0 (socket says 172.21.15.32:37720) 2024-06-03T01:33:22.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 -- 172.21.15.32:0/3230674595 learned_addr learned my addr 172.21.15.32:0/3230674595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc7ffff700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc8019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 -- 172.21.15.32:0/3230674595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 msgr2=0x7fdc8019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc8019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 -- 172.21.15.32:0/3230674595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc740095f0 con 0x7fdc80100ea0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc7ffff700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc8019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.991+0000 7fdc84bb4700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc74006010 tx=0x7fdc7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.992+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7401d070 con 0x7fdc80100ea0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.992+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc74004fb0 con 0x7fdc80100ea0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.992+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc801969f0 con 0x7fdc80100ea0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.992+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7400f690 con 0x7fdc80100ea0 2024-06-03T01:33:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.992+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80196ee0 con 0x7fdc80100ea0 2024-06-03T01:33:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.993+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc80109240 con 0x7fdc80100ea0 2024-06-03T01:33:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.993+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc740228a0 con 0x7fdc80100ea0 2024-06-03T01:33:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.994+0000 7fdc7dffb700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 0x7fdc700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.994+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fdc7408fb00 con 0x7fdc80100ea0 2024-06-03T01:33:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.994+0000 7fdc7ffff700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 0x7fdc700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:22.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.995+0000 7fdc7ffff700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 0x7fdc700706c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc80197be0 tx=0x7fdc6c005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:22.999+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdc7405c800 con 0x7fdc80100ea0 2024-06-03T01:33:23.023 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:22 smithi032 ceph-mon[111919]: pgmap v282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:23.023 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:22 smithi044 ceph-mon[84214]: pgmap v282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:23.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.253+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc80197d20 con 0x7fdc80100ea0 2024-06-03T01:33:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.253+0000 7fdc7dffb700 1 -- 172.21.15.32:0/3230674595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1950 (secure 0 0 0) 0x7fdc7405c390 con 0x7fdc80100ea0 2024-06-03T01:33:23.253 INFO:teuthology.orchestra.run.smithi032.stdout:e38 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 38 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-06-03T01:18:11.768168+0000 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-06-03T01:31:42.784421+0000 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-06-03T01:33:23.254 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 91 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 2 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=34414,1=44287} 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-06-03T01:33:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 34414 members: 44287,34414 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.itqzrm{0:34414} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.32:6828/1487470155,v1:172.21.15.32:6829/1487470155] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.tegslf{1:44287} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.32:6826/4048608139,v1:172.21.15.32:6827/4048608139] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.gkcyfr{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/2797390272,v1:172.21.15.44:6825/2797390272] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:33:23.256 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi044.mjxnhh{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6826/4018885964,v1:172.21.15.44:6827/4018885964] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.255+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 msgr2=0x7fdc700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.255+0000 7fdc86e18700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 0x7fdc700706c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc80197be0 tx=0x7fdc6c005f50 comp rx=0 tx=0).stop 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 msgr2=0x7fdc8019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc74006010 tx=0x7fdc7400be20 comp rx=0 tx=0).stop 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 shutdown_connections 2024-06-03T01:33:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fdc7006e200 0x7fdc700706c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc80100ea0 0x7fdc8019c6f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 --2- 172.21.15.32:0/3230674595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc80101850 0x7fdc8019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 >> 172.21.15.32:0/3230674595 conn(0x7fdc800fc750 msgr2=0x7fdc800fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 shutdown_connections 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:23.256+0000 7fdc86e18700 1 -- 172.21.15.32:0/3230674595 wait complete. 2024-06-03T01:33:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 38 2024-06-03T01:33:23.867 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:23 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3230674595' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:33:23.868 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-06-03T01:33:23.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:33:24.092 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:24.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:23 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3230674595' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:33:24.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:24 smithi032 ceph-mon[111919]: pgmap v283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:25.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:24 smithi044 ceph-mon[84214]: pgmap v283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 -- 172.21.15.32:0/1926355161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 msgr2=0x7fcf28105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 --2- 172.21.15.32:0/1926355161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28105fb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009910 tx=0x7fcf18009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 -- 172.21.15.32:0/1926355161 shutdown_connections 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 --2- 172.21.15.32:0/1926355161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 0x7fcf28100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 --2- 172.21.15.32:0/1926355161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28105fb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 -- 172.21.15.32:0/1926355161 >> 172.21.15.32:0/1926355161 conn(0x7fcf28075670 msgr2=0x7fcf28075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 -- 172.21.15.32:0/1926355161 shutdown_connections 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.829+0000 7fcf2df49700 1 -- 172.21.15.32:0/1926355161 wait complete. 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 Processor -- start 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 -- start start 2024-06-03T01:33:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 0x7fcf281955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28196070 con 0x7fcf28105bd0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.830+0000 7fcf2df49700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf281961e0 con 0x7fcf280ffc20 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37742/0 (socket says 172.21.15.32:37742) 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 -- 172.21.15.32:0/3244838346 learned_addr learned my addr 172.21.15.32:0/3244838346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf277fe700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 0x7fcf281955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 -- 172.21.15.32:0/3244838346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 msgr2=0x7fcf281955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 0x7fcf281955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 -- 172.21.15.32:0/3244838346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180095f0 con 0x7fcf28105bd0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf1effd700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000b4b0 tx=0x7fcf1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf10011800 con 0x7fcf28105bd0 2024-06-03T01:33:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf281a2c60 con 0x7fcf28105bd0 2024-06-03T01:33:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf10011e20 con 0x7fcf28105bd0 2024-06-03T01:33:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1000f2e0 con 0x7fcf28105bd0 2024-06-03T01:33:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.831+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf281a2fc0 con 0x7fcf28105bd0 2024-06-03T01:33:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.832+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf281080d0 con 0x7fcf28105bd0 2024-06-03T01:33:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.833+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcf10010380 con 0x7fcf28105bd0 2024-06-03T01:33:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.833+0000 7fcf257fa700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 0x7fcf140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.833+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fcf1008dde0 con 0x7fcf28105bd0 2024-06-03T01:33:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.834+0000 7fcf277fe700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 0x7fcf140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.834+0000 7fcf277fe700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 0x7fcf140706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b5c0 tx=0x7fcf1801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:25.838+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcf1005aae0 con 0x7fcf28105bd0 2024-06-03T01:33:26.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.091+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf2804f2e0 con 0x7fcf28105bd0 2024-06-03T01:33:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.092+0000 7fcf257fa700 1 -- 172.21.15.32:0/3244838346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5230 (secure 0 0 0) 0x7fcf1005a670 con 0x7fcf28105bd0 2024-06-03T01:33:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:26.092 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:42.784421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:active","state_seq":18,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34414,"qdb_cluster":[44287,34414]},"id":1}]} 2024-06-03T01:33:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 msgr2=0x7fcf140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 0x7fcf140706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b5c0 tx=0x7fcf1801a040 comp rx=0 tx=0).stop 2024-06-03T01:33:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 msgr2=0x7fcf28195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000b4b0 tx=0x7fcf1000b870 comp rx=0 tx=0).stop 2024-06-03T01:33:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 shutdown_connections 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fcf1406e200 0x7fcf140706c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf280ffc20 0x7fcf281955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 --2- 172.21.15.32:0/3244838346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf28105bd0 0x7fcf28195b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 >> 172.21.15.32:0/3244838346 conn(0x7fcf28075670 msgr2=0x7fcf280fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 shutdown_connections 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:26.094+0000 7fcf2df49700 1 -- 172.21.15.32:0/3244838346 wait complete. 2024-06-03T01:33:26.095 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 38 2024-06-03T01:33:26.787 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-06-03T01:33:26.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 18 2024-06-03T01:33:27.007 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:27.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:26 smithi032 ceph-mon[111919]: pgmap v284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:27.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:26 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3244838346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:33:27.119 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:26 smithi044 ceph-mon[84214]: pgmap v284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:27.120 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:26 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3244838346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:33:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 -- 172.21.15.32:0/480487270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a80ffc20 msgr2=0x7f56a8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 --2- 172.21.15.32:0/480487270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a80ffc20 0x7f56a8100080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5698009910 tx=0x7f5698009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 -- 172.21.15.32:0/480487270 shutdown_connections 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 --2- 172.21.15.32:0/480487270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a80ffc20 0x7f56a8100080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 --2- 172.21.15.32:0/480487270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a8105bd0 0x7f56a8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 -- 172.21.15.32:0/480487270 >> 172.21.15.32:0/480487270 conn(0x7f56a8075670 msgr2=0x7f56a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 -- 172.21.15.32:0/480487270 shutdown_connections 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.469+0000 7f56b05c1700 1 -- 172.21.15.32:0/480487270 wait complete. 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 Processor -- start 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 -- start start 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 0x7f56a819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a819d350 con 0x7f56a8105bd0 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56b05c1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8196800 con 0x7f56a80ffc20 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56adb5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56adb5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50248/0 (socket says 172.21.15.32:50248) 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.470+0000 7f56adb5c700 1 -- 172.21.15.32:0/3561631747 learned_addr learned my addr 172.21.15.32:0/3561631747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56ae35d700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 0x7f56a819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56adb5c700 1 -- 172.21.15.32:0/3561631747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 msgr2=0x7f56a819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56adb5c700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 0x7f56a819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56adb5c700 1 -- 172.21.15.32:0/3561631747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980095f0 con 0x7f56a8105bd0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56ae35d700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 0x7f56a819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56adb5c700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5698006010 tx=0x7f569800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f569801d070 con 0x7f56a8105bd0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5698004fb0 con 0x7f56a8105bd0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a8196a80 con 0x7f56a8105bd0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5698022ea0 con 0x7f56a8105bd0 2024-06-03T01:33:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.471+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a8196f70 con 0x7f56a8105bd0 2024-06-03T01:33:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.472+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a804f2e0 con 0x7f56a8105bd0 2024-06-03T01:33:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.473+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5698022470 con 0x7f56a8105bd0 2024-06-03T01:33:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.473+0000 7f569f7fe700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 0x7f56940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.473+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f569808f720 con 0x7f56a8105bd0 2024-06-03T01:33:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.473+0000 7f56ae35d700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 0x7f56940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.474+0000 7f56ae35d700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 0x7f56940706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56a4007900 tx=0x7f56a4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.478+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f569805c670 con 0x7f56a8105bd0 2024-06-03T01:33:28.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.747+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f56a81972a0 con 0x7f56a8105bd0 2024-06-03T01:33:28.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.748+0000 7f569f7fe700 1 -- 172.21.15.32:0/3561631747 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v38) v1 ==== 107+0+4873 (secure 0 0 0) 0x7f5698022720 con 0x7f56a8105bd0 2024-06-03T01:33:28.748 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:28.748 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:20:21.980280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6827/1879817357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1879817357},{"type":"v1","addr":"172.21.15.32:6827","nonce":1879817357}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6829/3721991653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3721991653},{"type":"v1","addr":"172.21.15.32:6829","nonce":3721991653}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 msgr2=0x7f56940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 0x7f56940706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56a4007900 tx=0x7f56a4008040 comp rx=0 tx=0).stop 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 msgr2=0x7f56a819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5698006010 tx=0x7f569800be20 comp rx=0 tx=0).stop 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 shutdown_connections 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f569406e200 0x7f56940706c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a80ffc20 0x7f56a819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 --2- 172.21.15.32:0/3561631747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105bd0 0x7f56a819ccc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 >> 172.21.15.32:0/3561631747 conn(0x7f56a8075670 msgr2=0x7f56a80686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 shutdown_connections 2024-06-03T01:33:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:28.750+0000 7f56b05c1700 1 -- 172.21.15.32:0/3561631747 wait complete. 2024-06-03T01:33:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-06-03T01:33:29.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:28 smithi032 ceph-mon[111919]: pgmap v285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:29.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:28 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3561631747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-06-03T01:33:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:28 smithi044 ceph-mon[84214]: pgmap v285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:29.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:28 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3561631747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-06-03T01:33:29.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 19 2024-06-03T01:33:29.600 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:30.975 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:30 smithi032 ceph-mon[111919]: pgmap v286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 -- 172.21.15.32:0/4002873027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4100ea0 msgr2=0x7f8ae4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 --2- 172.21.15.32:0/4002873027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4100ea0 0x7f8ae4101280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009910 tx=0x7f8ad4009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 -- 172.21.15.32:0/4002873027 shutdown_connections 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 --2- 172.21.15.32:0/4002873027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4101850 0x7f8ae41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 --2- 172.21.15.32:0/4002873027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4100ea0 0x7f8ae4101280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 -- 172.21.15.32:0/4002873027 >> 172.21.15.32:0/4002873027 conn(0x7f8ae40fc750 msgr2=0x7f8ae40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 -- 172.21.15.32:0/4002873027 shutdown_connections 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.211+0000 7f8aeae09700 1 -- 172.21.15.32:0/4002873027 wait complete. 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 Processor -- start 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 -- start start 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4100ea0 0x7f8ae419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419d2c0 con 0x7f8ae4101850 2024-06-03T01:33:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8aeae09700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4196770 con 0x7f8ae4100ea0 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50260/0 (socket says 172.21.15.32:50260) 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 -- 172.21.15.32:0/2539308767 learned_addr learned my addr 172.21.15.32:0/2539308767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae8ba5700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4100ea0 0x7f8ae419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 -- 172.21.15.32:0/2539308767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4100ea0 msgr2=0x7f8ae419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4100ea0 0x7f8ae419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.212+0000 7f8ae3fff700 1 -- 172.21.15.32:0/2539308767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40095f0 con 0x7f8ae4101850 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8ae3fff700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800d900 tx=0x7f8ad800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad8009900 con 0x7f8ae4101850 2024-06-03T01:33:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad8010460 con 0x7f8ae4101850 2024-06-03T01:33:31.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae4196a50 con 0x7f8ae4101850 2024-06-03T01:33:31.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad800f5f0 con 0x7f8ae4101850 2024-06-03T01:33:31.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.213+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae4196fa0 con 0x7f8ae4101850 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.214+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4109240 con 0x7f8ae4101850 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.215+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8ad80105d0 con 0x7f8ae4101850 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.215+0000 7f8ae1ffb700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 0x7f8acc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.215+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8ad808e100 con 0x7f8ae4101850 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.215+0000 7f8ae8ba5700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 0x7f8acc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:31.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.216+0000 7f8ae8ba5700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 0x7f8acc0706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006010 tx=0x7f8ad401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:31.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.220+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8ad805ae00 con 0x7f8ae4101850 2024-06-03T01:33:31.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:30 smithi044 ceph-mon[84214]: pgmap v286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.472+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f8ae404f2e0 con 0x7f8ae4101850 2024-06-03T01:33:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.473+0000 7f8ae1ffb700 1 -- 172.21.15.32:0/2539308767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v38) v1 ==== 107+0+4874 (secure 0 0 0) 0x7f8ad805a990 con 0x7f8ae4101850 2024-06-03T01:33:31.473 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:31.473 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:21:35.850986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6827/1879817357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1879817357},{"type":"v1","addr":"172.21.15.32:6827","nonce":1879817357}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6829/3721991653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3721991653},{"type":"v1","addr":"172.21.15.32:6829","nonce":3721991653}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 msgr2=0x7f8acc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 0x7f8acc0706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006010 tx=0x7f8ad401a040 comp rx=0 tx=0).stop 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 msgr2=0x7f8ae419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800d900 tx=0x7f8ad800dcc0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 shutdown_connections 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8acc06e200 0x7f8acc0706c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ae4100ea0 0x7f8ae419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 --2- 172.21.15.32:0/2539308767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae4101850 0x7f8ae419cc30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 >> 172.21.15.32:0/2539308767 conn(0x7f8ae40fc750 msgr2=0x7f8ae40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.475+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 shutdown_connections 2024-06-03T01:33:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:31.476+0000 7f8aeae09700 1 -- 172.21.15.32:0/2539308767 wait complete. 2024-06-03T01:33:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2024-06-03T01:33:32.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:31 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2539308767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-06-03T01:33:32.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 20 2024-06-03T01:33:32.201 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:31 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2539308767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-06-03T01:33:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:32.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:32 smithi032 ceph-mon[111919]: pgmap v287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:33.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:32 smithi044 ceph-mon[84214]: pgmap v287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.912+0000 7ff429095700 1 -- 172.21.15.32:0/1102129202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424100ea0 msgr2=0x7ff424101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.912+0000 7ff429095700 1 --2- 172.21.15.32:0/1102129202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424100ea0 0x7ff424101280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff414009910 tx=0x7ff414009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 -- 172.21.15.32:0/1102129202 shutdown_connections 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 --2- 172.21.15.32:0/1102129202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424101850 0x7ff4241058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 --2- 172.21.15.32:0/1102129202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424100ea0 0x7ff424101280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 -- 172.21.15.32:0/1102129202 >> 172.21.15.32:0/1102129202 conn(0x7ff4240fc750 msgr2=0x7ff4240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 -- 172.21.15.32:0/1102129202 shutdown_connections 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 -- 172.21.15.32:0/1102129202 wait complete. 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.913+0000 7ff429095700 1 Processor -- start 2024-06-03T01:33:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff429095700 1 -- start start 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff429095700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424100ea0 0x7ff42419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff429095700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff429095700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419d2e0 con 0x7ff424101850 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff429095700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424196790 con 0x7ff424100ea0 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50264/0 (socket says 172.21.15.32:50264) 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 -- 172.21.15.32:0/633173795 learned_addr learned my addr 172.21.15.32:0/633173795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff422d9d700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424100ea0 0x7ff42419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 -- 172.21.15.32:0/633173795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424100ea0 msgr2=0x7ff42419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424100ea0 0x7ff42419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.914+0000 7ff42259c700 1 -- 172.21.15.32:0/633173795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140095f0 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff42259c700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c8f0 tx=0x7ff40c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40c007ad0 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff40c00ce90 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff424196a70 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40c005600 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.915+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff424196fc0 con 0x7ff424101850 2024-06-03T01:33:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.916+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424109240 con 0x7ff424101850 2024-06-03T01:33:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.917+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff40c004750 con 0x7ff424101850 2024-06-03T01:33:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.917+0000 7ff41bfff700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 0x7ff4100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.917+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7ff40c08df70 con 0x7ff424101850 2024-06-03T01:33:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.917+0000 7ff422d9d700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 0x7ff4100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.918+0000 7ff422d9d700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 0x7ff4100706c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff414006010 tx=0x7ff41400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:33.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:33.922+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff40c0c2850 con 0x7ff424101850 2024-06-03T01:33:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.174+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7ff42404f2e0 con 0x7ff424101850 2024-06-03T01:33:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.175+0000 7ff41bfff700 1 -- 172.21.15.32:0/633173795 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+4873 (secure 0 0 0) 0x7ff40c01d090 con 0x7ff424101850 2024-06-03T01:33:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:34.175 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:24:41.116802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6827/1879817357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1879817357},{"type":"v1","addr":"172.21.15.32:6827","nonce":1879817357}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6829/3721991653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3721991653},{"type":"v1","addr":"172.21.15.32:6829","nonce":3721991653}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 msgr2=0x7ff4100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 0x7ff4100706c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff414006010 tx=0x7ff41400b540 comp rx=0 tx=0).stop 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 msgr2=0x7ff42419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c8f0 tx=0x7ff40c00ccb0 comp rx=0 tx=0).stop 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 shutdown_connections 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7ff41006e200 0x7ff4100706c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff424100ea0 0x7ff42419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 --2- 172.21.15.32:0/633173795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff424101850 0x7ff42419cc50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.177+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 >> 172.21.15.32:0/633173795 conn(0x7ff4240fc750 msgr2=0x7ff4240fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.178+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 shutdown_connections 2024-06-03T01:33:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:34.178+0000 7ff429095700 1 -- 172.21.15.32:0/633173795 wait complete. 2024-06-03T01:33:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-06-03T01:33:34.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 21 2024-06-03T01:33:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:35.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:34 smithi032 ceph-mon[111919]: pgmap v288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:35.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:34 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/633173795' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-06-03T01:33:35.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:34 smithi044 ceph-mon[84214]: pgmap v288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:35.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:34 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/633173795' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1026597809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0ffc20 msgr2=0x7fd4ec100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1026597809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec100080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009910 tx=0x7fd4d4009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1026597809 shutdown_connections 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1026597809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec100080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1026597809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1026597809 >> 172.21.15.32:0/1026597809 conn(0x7fd4ec075670 msgr2=0x7fd4ec075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1026597809 shutdown_connections 2024-06-03T01:33:36.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.582+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1026597809 wait complete. 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 Processor -- start 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 -- start start 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec19d2b0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.583+0000 7fd4f26c5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec196730 con 0x7fd4ec0ffc20 2024-06-03T01:33:36.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50292/0 (socket says 172.21.15.32:50292) 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 -- 172.21.15.32:0/1504372968 learned_addr learned my addr 172.21.15.32:0/1504372968 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4ebfff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 -- 172.21.15.32:0/1504372968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 msgr2=0x7fd4ec19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 -- 172.21.15.32:0/1504372968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40095f0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4ebfff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e3fff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40092e0 tx=0x7fd4d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d401d070 con 0x7fd4ec105bd0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4d4005e00 con 0x7fd4ec105bd0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec1969b0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d4017400 con 0x7fd4ec105bd0 2024-06-03T01:33:36.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.584+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec196ea0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.585+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec04f2e0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.586+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd4d400f460 con 0x7fd4ec105bd0 2024-06-03T01:33:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.586+0000 7fd4e9ffb700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 0x7fd4d80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:36.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.586+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fd4d408f6c0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.586+0000 7fd4ebfff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 0x7fd4d80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:36.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.587+0000 7fd4ebfff700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 0x7fd4d80706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005950 tx=0x7fd4dc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:36.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.591+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd4d405c440 con 0x7fd4ec105bd0 2024-06-03T01:33:36.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.841+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fd4ec1971d0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.842+0000 7fd4e9ffb700 1 -- 172.21.15.32:0/1504372968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+4872 (secure 0 0 0) 0x7fd4d405bfd0 con 0x7fd4ec105bd0 2024-06-03T01:33:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:24:42.122740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6827/1879817357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1879817357},{"type":"v1","addr":"172.21.15.32:6827","nonce":1879817357}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6829/3721991653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3721991653},{"type":"v1","addr":"172.21.15.32:6829","nonce":3721991653}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 msgr2=0x7fd4d80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 0x7fd4d80706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005950 tx=0x7fd4dc0058e0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 msgr2=0x7fd4ec19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40092e0 tx=0x7fd4d400be20 comp rx=0 tx=0).stop 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 shutdown_connections 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd4d806e200 0x7fd4d80706c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ec0ffc20 0x7fd4ec19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 --2- 172.21.15.32:0/1504372968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4ec105bd0 0x7fd4ec19cbd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 >> 172.21.15.32:0/1504372968 conn(0x7fd4ec075670 msgr2=0x7fd4ec068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 shutdown_connections 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:36.844+0000 7fd4f26c5700 1 -- 172.21.15.32:0/1504372968 wait complete. 2024-06-03T01:33:36.845 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-06-03T01:33:37.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:36 smithi032 ceph-mon[111919]: pgmap v289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:37.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:36 smithi044 ceph-mon[84214]: pgmap v289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:37.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 22 2024-06-03T01:33:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:38.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:37 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1504372968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-06-03T01:33:38.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:37 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:37 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1504372968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-06-03T01:33:38.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:37 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:38.987 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:38 smithi032 ceph-mon[111919]: pgmap v290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 -- 172.21.15.32:0/3708077794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00101850 msgr2=0x7f5a001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/3708077794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00101850 0x7f5a001058a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009910 tx=0x7f59f0009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 -- 172.21.15.32:0/3708077794 shutdown_connections 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/3708077794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00101850 0x7f5a001058a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/3708077794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00100ea0 0x7f5a00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 -- 172.21.15.32:0/3708077794 >> 172.21.15.32:0/3708077794 conn(0x7f5a000fc750 msgr2=0x7f5a000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 -- 172.21.15.32:0/3708077794 shutdown_connections 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.208+0000 7f5a07a5a700 1 -- 172.21.15.32:0/3708077794 wait complete. 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 Processor -- start 2024-06-03T01:33:39.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 -- start start 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00101850 0x7f5a0019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0019f410 con 0x7f5a00100ea0 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a07a5a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00198950 con 0x7f5a00101850 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a057f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a057f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49936/0 (socket says 172.21.15.32:49936) 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.209+0000 7f5a057f6700 1 -- 172.21.15.32:0/461226810 learned_addr learned my addr 172.21.15.32:0/461226810 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a04ff5700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00101850 0x7f5a0019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a057f6700 1 -- 172.21.15.32:0/461226810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00101850 msgr2=0x7f5a0019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a057f6700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00101850 0x7f5a0019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a057f6700 1 -- 172.21.15.32:0/461226810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00095f0 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a057f6700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c930 tx=0x7f59fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc007ad0 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00198c30 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59fc007c30 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc018610 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.210+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a00199180 con 0x7f5a00100ea0 2024-06-03T01:33:39.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.211+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a00105020 con 0x7f5a00100ea0 2024-06-03T01:33:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.211+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59fc00f450 con 0x7f5a00100ea0 2024-06-03T01:33:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.212+0000 7f59f67fc700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 0x7f59ec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.212+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f59fc08e4b0 con 0x7f5a00100ea0 2024-06-03T01:33:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.212+0000 7f5a04ff5700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 0x7f59ec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:39.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.212+0000 7f5a04ff5700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 0x7f59ec0706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006010 tx=0x7f59f0005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:39.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.217+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f59fc05b1b0 con 0x7f5a00100ea0 2024-06-03T01:33:39.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:38 smithi044 ceph-mon[84214]: pgmap v290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.467+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f5a00199eb0 con 0x7f5a00100ea0 2024-06-03T01:33:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.468+0000 7f59f67fc700 1 -- 172.21.15.32:0/461226810 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+3293 (secure 0 0 0) 0x7f59fc05ad40 con 0x7f5a00100ea0 2024-06-03T01:33:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:39.468 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 msgr2=0x7f59ec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 0x7f59ec0706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006010 tx=0x7f59f0005960 comp rx=0 tx=0).stop 2024-06-03T01:33:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 msgr2=0x7f5a0019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c930 tx=0x7f59fc00ccf0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 shutdown_connections 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f59ec06e200 0x7f59ec0706c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a00100ea0 0x7f5a0019e8d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 --2- 172.21.15.32:0/461226810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a00101850 0x7f5a0019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 >> 172.21.15.32:0/461226810 conn(0x7f5a000fc750 msgr2=0x7f5a000feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 shutdown_connections 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:39.470+0000 7f5a07a5a700 1 -- 172.21.15.32:0/461226810 wait complete. 2024-06-03T01:33:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2024-06-03T01:33:40.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:39 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/461226810' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-06-03T01:33:40.094 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:40.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 23 2024-06-03T01:33:40.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:39 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/461226810' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-06-03T01:33:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:41.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:40 smithi032 ceph-mon[111919]: pgmap v291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:41.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:40 smithi044 ceph-mon[84214]: pgmap v291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 -- 172.21.15.32:0/2316197454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 msgr2=0x7fe36c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 --2- 172.21.15.32:0/2316197454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c0689e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe360009910 tx=0x7fe360009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 -- 172.21.15.32:0/2316197454 shutdown_connections 2024-06-03T01:33:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 --2- 172.21.15.32:0/2316197454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c0689e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 --2- 172.21.15.32:0/2316197454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 0x7fe36c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 -- 172.21.15.32:0/2316197454 >> 172.21.15.32:0/2316197454 conn(0x7fe36c075670 msgr2=0x7fe36c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 -- 172.21.15.32:0/2316197454 shutdown_connections 2024-06-03T01:33:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.015+0000 7fe37283e700 1 -- 172.21.15.32:0/2316197454 wait complete. 2024-06-03T01:33:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 Processor -- start 2024-06-03T01:33:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 -- start start 2024-06-03T01:33:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 0x7fe36c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c197550 con 0x7fe36c068580 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe37283e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c19b360 con 0x7fe36c106cd0 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49956/0 (socket says 172.21.15.32:49956) 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe36bfff700 1 -- 172.21.15.32:0/446136287 learned_addr learned my addr 172.21.15.32:0/446136287 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.016+0000 7fe36b7fe700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 0x7fe36c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe36bfff700 1 -- 172.21.15.32:0/446136287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 msgr2=0x7fe36c196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe36bfff700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 0x7fe36c196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe36bfff700 1 -- 172.21.15.32:0/446136287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600095f0 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe36bfff700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b4b0 tx=0x7fe35c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35c011800 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c19b640 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe35c011e20 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35c00f2e0 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.017+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c19bb90 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.018+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c1091d0 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.019+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe35c010380 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.019+0000 7fe3697fa700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 0x7fe3540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.020+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fe35c08ddf0 con 0x7fe36c068580 2024-06-03T01:33:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.020+0000 7fe36b7fe700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 0x7fe3540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.020+0000 7fe36b7fe700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 0x7fe3540706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe36c197fa0 tx=0x7fe36001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.024+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe35c05aaf0 con 0x7fe36c068580 2024-06-03T01:33:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.279+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fe36c04f2e0 con 0x7fe36c068580 2024-06-03T01:33:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.279+0000 7fe3697fa700 1 -- 172.21.15.32:0/446136287 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+4851 (secure 0 0 0) 0x7fe35c05a680 con 0x7fe36c068580 2024-06-03T01:33:42.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:42.280 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1054828690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1054828690},{"type":"v1","addr":"172.21.15.32:6827","nonce":1054828690}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34382,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/4125418426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4125418426},{"type":"v1","addr":"172.21.15.32:6829","nonce":4125418426}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 msgr2=0x7fe3540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 0x7fe3540706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe36c197fa0 tx=0x7fe36001a040 comp rx=0 tx=0).stop 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 msgr2=0x7fe36c196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b4b0 tx=0x7fe35c00b7c0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 shutdown_connections 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe35406e200 0x7fe3540706c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe36c068580 0x7fe36c196980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 --2- 172.21.15.32:0/446136287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe36c106cd0 0x7fe36c196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 >> 172.21.15.32:0/446136287 conn(0x7fe36c075670 msgr2=0x7fe36c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 shutdown_connections 2024-06-03T01:33:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:42.282+0000 7fe37283e700 1 -- 172.21.15.32:0/446136287 wait complete. 2024-06-03T01:33:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2024-06-03T01:33:42.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:42 smithi032 ceph-mon[111919]: pgmap v292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:42.946 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:42 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/446136287' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-06-03T01:33:42.969 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:42.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 24 2024-06-03T01:33:43.185 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:43.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:42 smithi044 ceph-mon[84214]: pgmap v292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:43.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:42 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/446136287' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-06-03T01:33:44.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 -- 172.21.15.32:0/588077650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 msgr2=0x7f5b4c10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/588077650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c10d290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009910 tx=0x7f5b34009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 -- 172.21.15.32:0/588077650 shutdown_connections 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/588077650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c10d290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/588077650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 0x7f5b4c068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 -- 172.21.15.32:0/588077650 >> 172.21.15.32:0/588077650 conn(0x7f5b4c075670 msgr2=0x7f5b4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 -- 172.21.15.32:0/588077650 shutdown_connections 2024-06-03T01:33:44.870 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:44 smithi032 ceph-mon[111919]: pgmap v293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.869+0000 7f5b51f2e700 1 -- 172.21.15.32:0/588077650 wait complete. 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 Processor -- start 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 -- start start 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 0x7f5b4c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c19f4b0 con 0x7f5b4c068ea0 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.870+0000 7f5b51f2e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c198960 con 0x7f5b4c068580 2024-06-03T01:33:44.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49974/0 (socket says 172.21.15.32:49974) 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 -- 172.21.15.32:0/914659189 learned_addr learned my addr 172.21.15.32:0/914659189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b4b7fe700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 0x7f5b4c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 -- 172.21.15.32:0/914659189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 msgr2=0x7f5b4c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 0x7f5b4c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 -- 172.21.15.32:0/914659189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340095f0 con 0x7f5b4c068ea0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b42ffd700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005f50 tx=0x7f5b3400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3401d070 con 0x7f5b4c068ea0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b3400b610 con 0x7f5b4c068ea0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.871+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b34022470 con 0x7f5b4c068ea0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.872+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c198be0 con 0x7f5b4c068ea0 2024-06-03T01:33:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.872+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c199080 con 0x7f5b4c068ea0 2024-06-03T01:33:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.873+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c10aa00 con 0x7f5b4c068ea0 2024-06-03T01:33:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.873+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5b3400fca0 con 0x7f5b4c068ea0 2024-06-03T01:33:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.874+0000 7f5b497fa700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 0x7f5b380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.874+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f5b3408f750 con 0x7f5b4c068ea0 2024-06-03T01:33:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.874+0000 7f5b4b7fe700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 0x7f5b380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.874+0000 7f5b4b7fe700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 0x7f5b380706c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009bf0 tx=0x7f5b3c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:44.878+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5b3405c450 con 0x7f5b4c068ea0 2024-06-03T01:33:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.130+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f5b4c199e70 con 0x7f5b4c068ea0 2024-06-03T01:33:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.131+0000 7f5b497fa700 1 -- 172.21.15.32:0/914659189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+4072 (secure 0 0 0) 0x7f5b3405bfe0 con 0x7f5b4c068ea0 2024-06-03T01:33:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:45.131 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1054828690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1054828690},{"type":"v1","addr":"172.21.15.32:6827","nonce":1054828690}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 msgr2=0x7f5b380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 0x7f5b380706c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009bf0 tx=0x7f5b3c009260 comp rx=0 tx=0).stop 2024-06-03T01:33:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 msgr2=0x7f5b4c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005f50 tx=0x7f5b3400f740 comp rx=0 tx=0).stop 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 shutdown_connections 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f5b3806e200 0x7f5b380706c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c068580 0x7f5b4c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 --2- 172.21.15.32:0/914659189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b4c068ea0 0x7f5b4c19ee20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.133+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 >> 172.21.15.32:0/914659189 conn(0x7f5b4c075670 msgr2=0x7f5b4c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.134+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 shutdown_connections 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:45.134+0000 7f5b51f2e700 1 -- 172.21.15.32:0/914659189 wait complete. 2024-06-03T01:33:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-06-03T01:33:45.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:44 smithi044 ceph-mon[84214]: pgmap v293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:45.742 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:45.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 25 2024-06-03T01:33:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:46.083 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:45 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/914659189' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-06-03T01:33:46.083 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:45 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:33:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:45 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/914659189' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-06-03T01:33:46.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:45 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:33:47.002 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: pgmap v294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:47.002 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.002 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.002 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:33:47.002 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:33:47.003 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:46 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: pgmap v294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:33:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:33:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:46 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 -- 172.21.15.32:0/3381955532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338101850 msgr2=0x7f73381058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 --2- 172.21.15.32:0/3381955532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338101850 0x7f73381058a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f732c009910 tx=0x7f732c009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 -- 172.21.15.32:0/3381955532 shutdown_connections 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 --2- 172.21.15.32:0/3381955532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338101850 0x7f73381058a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 --2- 172.21.15.32:0/3381955532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338100ea0 0x7f7338101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 -- 172.21.15.32:0/3381955532 >> 172.21.15.32:0/3381955532 conn(0x7f73380fc750 msgr2=0x7f73380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 -- 172.21.15.32:0/3381955532 shutdown_connections 2024-06-03T01:33:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.494+0000 7f733ddb8700 1 -- 172.21.15.32:0/3381955532 wait complete. 2024-06-03T01:33:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 Processor -- start 2024-06-03T01:33:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 -- start start 2024-06-03T01:33:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 0x7f733819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338198890 con 0x7f7338100ea0 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f733ddb8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338198a00 con 0x7f7338101850 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f73377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.495+0000 7f73377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58206/0 (socket says 172.21.15.32:58206) 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f73377fe700 1 -- 172.21.15.32:0/824018945 learned_addr learned my addr 172.21.15.32:0/824018945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f7336ffd700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 0x7f733819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f73377fe700 1 -- 172.21.15.32:0/824018945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 msgr2=0x7f733819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f73377fe700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 0x7f733819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f73377fe700 1 -- 172.21.15.32:0/824018945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0095f0 con 0x7f7338100ea0 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f7336ffd700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 0x7f733819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f73377fe700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f732800ef30 tx=0x7f732800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7328018070 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7338198c80 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f732800f040 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7328004020 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.496+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381991a0 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.497+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7338105730 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.498+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7328004ad0 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.498+0000 7f7334ff9700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 0x7f73200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.498+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f732808e160 con 0x7f7338100ea0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.498+0000 7f7336ffd700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 0x7f73200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.499+0000 7f7336ffd700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 0x7f73200706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f732c0059d0 tx=0x7f732c005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.503+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7328057330 con 0x7f7338100ea0 2024-06-03T01:33:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.756+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f7338199dc0 con 0x7f7338100ea0 2024-06-03T01:33:47.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.757+0000 7f7334ff9700 1 -- 172.21.15.32:0/824018945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4919 (secure 0 0 0) 0x7f732805a9f0 con 0x7f7338100ea0 2024-06-03T01:33:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:47.757 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1054828690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1054828690},{"type":"v1","addr":"172.21.15.32:6827","nonce":1054828690}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:47.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 msgr2=0x7f73200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:47.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 0x7f73200706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f732c0059d0 tx=0x7f732c005940 comp rx=0 tx=0).stop 2024-06-03T01:33:47.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 msgr2=0x7f733819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f732800ef30 tx=0x7f732800c3a0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 shutdown_connections 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f732006e200 0x7f73200706c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7338100ea0 0x7f733819e8a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 --2- 172.21.15.32:0/824018945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338101850 0x7f733819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 >> 172.21.15.32:0/824018945 conn(0x7f73380fc750 msgr2=0x7f7338102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 shutdown_connections 2024-06-03T01:33:47.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:47.759+0000 7f733ddb8700 1 -- 172.21.15.32:0/824018945 wait complete. 2024-06-03T01:33:47.761 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-06-03T01:33:48.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:47 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/824018945' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-06-03T01:33:48.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:47 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/824018945' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-06-03T01:33:48.359 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:48.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 26 2024-06-03T01:33:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:48.963 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:48 smithi032 ceph-mon[111919]: pgmap v295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:49.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:48 smithi044 ceph-mon[84214]: pgmap v295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/2841540964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68100ea0 msgr2=0x7f3f68101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/2841540964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68100ea0 0x7f3f68101280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009910 tx=0x7f3f58009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/2841540964 shutdown_connections 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/2841540964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68101850 0x7f3f681058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/2841540964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68100ea0 0x7f3f68101280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/2841540964 >> 172.21.15.32:0/2841540964 conn(0x7f3f680fc750 msgr2=0x7f3f680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/2841540964 shutdown_connections 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/2841540964 wait complete. 2024-06-03T01:33:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 Processor -- start 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.102+0000 7f3f6e9f3700 1 -- start start 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f6e9f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68100ea0 0x7f3f6819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f6e9f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f6e9f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819f550 con 0x7f3f68101850 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f6e9f3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68198a00 con 0x7f3f68100ea0 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58226/0 (socket says 172.21.15.32:58226) 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 -- 172.21.15.32:0/3315543300 learned_addr learned my addr 172.21.15.32:0/3315543300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f67fff700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68100ea0 0x7f3f6819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 -- 172.21.15.32:0/3315543300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68100ea0 msgr2=0x7f3f6819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68100ea0 0x7f3f6819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 -- 172.21.15.32:0/3315543300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580095f0 con 0x7f3f68101850 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.103+0000 7f3f677fe700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00b480 tx=0x7f3f5c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.104+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c011800 con 0x7f3f68101850 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.104+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f5c011e20 con 0x7f3f68101850 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.104+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c010ea0 con 0x7f3f68101850 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.104+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68198ce0 con 0x7f3f68101850 2024-06-03T01:33:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.104+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f68199230 con 0x7f3f68101850 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.105+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f68105020 con 0x7f3f68101850 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.105+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3f5c010380 con 0x7f3f68101850 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.106+0000 7f3f657fa700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 0x7f3f500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.106+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f3f5c08dd00 con 0x7f3f68101850 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.106+0000 7f3f67fff700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 0x7f3f500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.106+0000 7f3f67fff700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 0x7f3f500706c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006010 tx=0x7f3f5801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:50.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.111+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3f5c091030 con 0x7f3f68101850 2024-06-03T01:33:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.364+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f3f681994b0 con 0x7f3f68101850 2024-06-03T01:33:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.365+0000 7f3f657fa700 1 -- 172.21.15.32:0/3315543300 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+4140 (secure 0 0 0) 0x7f3f5c05aa80 con 0x7f3f68101850 2024-06-03T01:33:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:50.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 msgr2=0x7f3f500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 0x7f3f500706c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006010 tx=0x7f3f5801a040 comp rx=0 tx=0).stop 2024-06-03T01:33:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 msgr2=0x7f3f6819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00b480 tx=0x7f3f5c00b790 comp rx=0 tx=0).stop 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 shutdown_connections 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f3f5006e200 0x7f3f500706c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f68100ea0 0x7f3f6819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 --2- 172.21.15.32:0/3315543300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f68101850 0x7f3f6819eec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 >> 172.21.15.32:0/3315543300 conn(0x7f3f680fc750 msgr2=0x7f3f680fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 shutdown_connections 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:50.367+0000 7f3f6e9f3700 1 -- 172.21.15.32:0/3315543300 wait complete. 2024-06-03T01:33:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2024-06-03T01:33:50.949 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:50 smithi032 ceph-mon[111919]: pgmap v296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:50.949 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:50 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3315543300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-06-03T01:33:50.973 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:50.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 27 2024-06-03T01:33:51.189 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:50 smithi044 ceph-mon[84214]: pgmap v296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:51.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:50 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3315543300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-06-03T01:33:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 -- 172.21.15.32:0/4104877999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 msgr2=0x7f4dd8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/4104877999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8100080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009910 tx=0x7f4dcc009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 -- 172.21.15.32:0/4104877999 shutdown_connections 2024-06-03T01:33:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/4104877999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8100080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/4104877999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 0x7f4dd8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 -- 172.21.15.32:0/4104877999 >> 172.21.15.32:0/4104877999 conn(0x7f4dd8075670 msgr2=0x7f4dd8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 -- 172.21.15.32:0/4104877999 shutdown_connections 2024-06-03T01:33:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.859+0000 7f4dde7fd700 1 -- 172.21.15.32:0/4104877999 wait complete. 2024-06-03T01:33:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.860+0000 7f4dde7fd700 1 Processor -- start 2024-06-03T01:33:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.860+0000 7f4dde7fd700 1 -- start start 2024-06-03T01:33:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.860+0000 7f4dde7fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dde7fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 0x7f4dd81021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dde7fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd81026f0 con 0x7f4dd80ffc20 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dde7fd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8102830 con 0x7f4dd8105bd0 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58238/0 (socket says 172.21.15.32:58238) 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 -- 172.21.15.32:0/2813534689 learned_addr learned my addr 172.21.15.32:0/2813534689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd77fe700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 0x7f4dd81021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 -- 172.21.15.32:0/2813534689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 msgr2=0x7f4dd81021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 0x7f4dd81021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 -- 172.21.15.32:0/2813534689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0095f0 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.861+0000 7f4dd7fff700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800d8d0 tx=0x7f4dc800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.862+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc80098a0 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.862+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dc8010460 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.862+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd81a2ae0 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.862+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc800f5f0 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.862+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd81a2e40 con 0x7f4dd80ffc20 2024-06-03T01:33:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.863+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd8102a80 con 0x7f4dd80ffc20 2024-06-03T01:33:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.863+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4dc8009a00 con 0x7f4dd80ffc20 2024-06-03T01:33:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.864+0000 7f4dd57fa700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.864+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f4dc808e180 con 0x7f4dd80ffc20 2024-06-03T01:33:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.864+0000 7f4dd77fe700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.864+0000 7f4dd77fe700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 0x7f4dc00706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81033f0 tx=0x7f4dcc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:52.873+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4dd8102a80 con 0x7f4dd80ffc20 2024-06-03T01:33:52.874 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:52 smithi032 ceph-mon[111919]: pgmap v297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:52.874 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:52 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.132+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f4dd81029f0 con 0x7f4dd80ffc20 2024-06-03T01:33:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.133+0000 7f4dd57fa700 1 -- 172.21.15.32:0/2813534689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+4987 (secure 0 0 0) 0x7f4dc8057350 con 0x7f4dd80ffc20 2024-06-03T01:33:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:53.134 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24295,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/3478864978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":3478864978},{"type":"v1","addr":"172.21.15.44:6825","nonce":3478864978}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 msgr2=0x7f4dc00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 0x7f4dc00706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81033f0 tx=0x7f4dcc01a040 comp rx=0 tx=0).stop 2024-06-03T01:33:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 msgr2=0x7f4dd8101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800d8d0 tx=0x7f4dc800dbe0 comp rx=0 tx=0).stop 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 shutdown_connections 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd80ffc20 0x7f4dd8101c70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 --2- 172.21.15.32:0/2813534689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd8105bd0 0x7f4dd81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 >> 172.21.15.32:0/2813534689 conn(0x7f4dd8075670 msgr2=0x7f4dd8103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 shutdown_connections 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:53.135+0000 7f4dde7fd700 1 -- 172.21.15.32:0/2813534689 wait complete. 2024-06-03T01:33:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-06-03T01:33:53.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:52 smithi044 ceph-mon[84214]: pgmap v297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:53.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:52 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:33:53.754 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:53.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 28 2024-06-03T01:33:53.974 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:54.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:53 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2813534689' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-06-03T01:33:54.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:53 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2813534689' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-06-03T01:33:54.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:54 smithi032 ceph-mon[111919]: pgmap v298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:55.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:54 smithi044 ceph-mon[84214]: pgmap v298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- 172.21.15.32:0/2761320443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c100ea0 msgr2=0x7fc29c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2761320443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c100ea0 0x7fc29c101280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009910 tx=0x7fc28c009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- 172.21.15.32:0/2761320443 shutdown_connections 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2761320443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c101850 0x7fc29c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2761320443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c100ea0 0x7fc29c101280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- 172.21.15.32:0/2761320443 >> 172.21.15.32:0/2761320443 conn(0x7fc29c0fc750 msgr2=0x7fc29c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- 172.21.15.32:0/2761320443 shutdown_connections 2024-06-03T01:33:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- 172.21.15.32:0/2761320443 wait complete. 2024-06-03T01:33:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 Processor -- start 2024-06-03T01:33:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.473+0000 7fc2a2273700 1 -- start start 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc2a2273700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c100ea0 0x7fc29c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc2a2273700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc2a2273700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c19d2c0 con 0x7fc29c101850 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc2a2273700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c196770 con 0x7fc29c100ea0 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58254/0 (socket says 172.21.15.32:58254) 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 -- 172.21.15.32:0/2038999263 learned_addr learned my addr 172.21.15.32:0/2038999263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29bfff700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c100ea0 0x7fc29c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 -- 172.21.15.32:0/2038999263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c100ea0 msgr2=0x7fc29c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c100ea0 0x7fc29c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 -- 172.21.15.32:0/2038999263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0095f0 con 0x7fc29c101850 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.474+0000 7fc29b7fe700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc29000d900 tx=0x7fc29000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.475+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc290009900 con 0x7fc29c101850 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.475+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c196a50 con 0x7fc29c101850 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.475+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc290010460 con 0x7fc29c101850 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.475+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29000f5f0 con 0x7fc29c101850 2024-06-03T01:33:55.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.475+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c196fa0 con 0x7fc29c101850 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.476+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c109240 con 0x7fc29c101850 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.476+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc290009a60 con 0x7fc29c101850 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.477+0000 7fc2997fa700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 0x7fc2840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.477+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fc29008de60 con 0x7fc29c101850 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.477+0000 7fc29bfff700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 0x7fc2840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:55.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.477+0000 7fc29bfff700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 0x7fc2840706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006010 tx=0x7fc28c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.482+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc29005ab60 con 0x7fc29c101850 2024-06-03T01:33:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.733+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fc29c04f2e0 con 0x7fc29c101850 2024-06-03T01:33:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.733+0000 7fc2997fa700 1 -- 172.21.15.32:0/2038999263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+4208 (secure 0 0 0) 0x7fc29005a6f0 con 0x7fc29c101850 2024-06-03T01:33:55.734 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:55.734 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.735+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 msgr2=0x7fc2840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.735+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 0x7fc2840706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006010 tx=0x7fc28c01a040 comp rx=0 tx=0).stop 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.735+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 msgr2=0x7fc29c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.735+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc29000d900 tx=0x7fc29000dc10 comp rx=0 tx=0).stop 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 shutdown_connections 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fc28406e200 0x7fc2840706c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc29c100ea0 0x7fc29c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 --2- 172.21.15.32:0/2038999263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc29c101850 0x7fc29c19cc30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 >> 172.21.15.32:0/2038999263 conn(0x7fc29c0fc750 msgr2=0x7fc29c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 shutdown_connections 2024-06-03T01:33:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:55.736+0000 7fc2a2273700 1 -- 172.21.15.32:0/2038999263 wait complete. 2024-06-03T01:33:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-06-03T01:33:56.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:55 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2038999263' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-06-03T01:33:56.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:55 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2038999263' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-06-03T01:33:56.372 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:56.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 29 2024-06-03T01:33:56.589 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:33:57.050 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:56 smithi032 ceph-mon[111919]: pgmap v299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:57.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:56 smithi044 ceph-mon[84214]: pgmap v299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 -- 172.21.15.32:0/3581808444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26240ff340 msgr2=0x7f26240ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 --2- 172.21.15.32:0/3581808444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26240ff340 0x7f26240ff7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2618009910 tx=0x7f2618009c20 comp rx=0 tx=0).stop 2024-06-03T01:33:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 -- 172.21.15.32:0/3581808444 shutdown_connections 2024-06-03T01:33:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 --2- 172.21.15.32:0/3581808444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26240ff340 0x7f26240ff7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 --2- 172.21.15.32:0/3581808444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2624105350 0x7f2624105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.256+0000 7f2629c97700 1 -- 172.21.15.32:0/3581808444 >> 172.21.15.32:0/3581808444 conn(0x7f2624074e30 msgr2=0x7f2624075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.257+0000 7f2629c97700 1 -- 172.21.15.32:0/3581808444 shutdown_connections 2024-06-03T01:33:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.257+0000 7f2629c97700 1 -- 172.21.15.32:0/3581808444 wait complete. 2024-06-03T01:33:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.257+0000 7f2629c97700 1 Processor -- start 2024-06-03T01:33:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.257+0000 7f2629c97700 1 -- start start 2024-06-03T01:33:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2629c97700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff340 0x7f2624100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2629c97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2629c97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624101a40 con 0x7f2624105350 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2629c97700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624101b80 con 0x7f26240ff340 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50634/0 (socket says 172.21.15.32:50634) 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 -- 172.21.15.32:0/124236407 learned_addr learned my addr 172.21.15.32:0/124236407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f26237fe700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff340 0x7f2624100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 -- 172.21.15.32:0/124236407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff340 msgr2=0x7f2624100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff340 0x7f2624100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 -- 172.21.15.32:0/124236407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180095f0 con 0x7f2624105350 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2622ffd700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2618006010 tx=0x7f261800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.258+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261801d070 con 0x7f2624105350 2024-06-03T01:33:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.259+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2618005dc0 con 0x7f2624105350 2024-06-03T01:33:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.259+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2624101e00 con 0x7f2624105350 2024-06-03T01:33:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.259+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2618017400 con 0x7f2624105350 2024-06-03T01:33:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.259+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26241a2d30 con 0x7f2624105350 2024-06-03T01:33:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.260+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624109860 con 0x7f2624105350 2024-06-03T01:33:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.260+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2618003a60 con 0x7f2624105350 2024-06-03T01:33:58.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.261+0000 7f2620ff9700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 0x7f260c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:33:58.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.261+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f261808f670 con 0x7f2624105350 2024-06-03T01:33:58.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.261+0000 7f26237fe700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 0x7f260c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:33:58.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.261+0000 7f26237fe700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 0x7f260c0706c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2614005950 tx=0x7f261400b510 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:33:58.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.265+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2618058860 con 0x7f2624105350 2024-06-03T01:33:58.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.515+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f262402cc70 con 0x7f2624105350 2024-06-03T01:33:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.516+0000 7f2620ff9700 1 -- 172.21.15.32:0/124236407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+5055 (secure 0 0 0) 0x7f261805bf20 con 0x7f2624105350 2024-06-03T01:33:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:33:58.516 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24287,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/1615260462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1615260462},{"type":"v1","addr":"172.21.15.44:6827","nonce":1615260462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:33:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 msgr2=0x7f260c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 0x7f260c0706c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2614005950 tx=0x7f261400b510 comp rx=0 tx=0).stop 2024-06-03T01:33:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 msgr2=0x7f2624101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2618006010 tx=0x7f261800bd70 comp rx=0 tx=0).stop 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 shutdown_connections 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f260c06e200 0x7f260c0706c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26240ff340 0x7f2624100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 --2- 172.21.15.32:0/124236407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624105350 0x7f2624101500 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 >> 172.21.15.32:0/124236407 conn(0x7f2624074e30 msgr2=0x7f2624103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 shutdown_connections 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:33:58.519+0000 7f2629c97700 1 -- 172.21.15.32:0/124236407 wait complete. 2024-06-03T01:33:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-06-03T01:33:59.162 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:58 smithi032 ceph-mon[111919]: pgmap v300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:59.162 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:33:58 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/124236407' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-06-03T01:33:59.186 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:33:59.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 30 2024-06-03T01:33:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:58 smithi044 ceph-mon[84214]: pgmap v300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:33:59.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:33:58 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/124236407' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-06-03T01:33:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 -- 172.21.15.32:0/2797003438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb00ffb40 msgr2=0x7f8bb00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/2797003438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb00ffb40 0x7f8bb00fffc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009910 tx=0x7f8ba4009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 -- 172.21.15.32:0/2797003438 shutdown_connections 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/2797003438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb00ffb40 0x7f8bb00fffc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/2797003438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 -- 172.21.15.32:0/2797003438 >> 172.21.15.32:0/2797003438 conn(0x7f8bb00fb650 msgr2=0x7f8bb00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.991+0000 7f8bb54d4700 1 -- 172.21.15.32:0/2797003438 shutdown_connections 2024-06-03T01:34:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.992+0000 7f8bb54d4700 1 -- 172.21.15.32:0/2797003438 wait complete. 2024-06-03T01:34:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.992+0000 7f8bb54d4700 1 Processor -- start 2024-06-03T01:34:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.992+0000 7f8bb54d4700 1 -- start start 2024-06-03T01:34:00.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bb54d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb00ffb40 0x7f8bb01954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:00.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bb54d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:00.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bb54d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0195f40 con 0x7f8bb0105bf0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bb54d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb01960b0 con 0x7f8bb00ffb40 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50666/0 (socket says 172.21.15.32:50666) 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 -- 172.21.15.32:0/3615910803 learned_addr learned my addr 172.21.15.32:0/3615910803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8baeffd700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb00ffb40 0x7f8bb01954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 -- 172.21.15.32:0/3615910803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb00ffb40 msgr2=0x7f8bb01954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb00ffb40 0x7f8bb01954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 -- 172.21.15.32:0/3615910803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40095f0 con 0x7f8bb0105bf0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8bae7fc700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40092e0 tx=0x7f8ba400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.993+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ba401d070 con 0x7f8bb0105bf0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.994+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ba4005e00 con 0x7f8bb0105bf0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.994+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb0196330 con 0x7f8bb0105bf0 2024-06-03T01:34:00.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.994+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ba4017400 con 0x7f8bb0105bf0 2024-06-03T01:34:00.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.994+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb01a2db0 con 0x7f8bb0105bf0 2024-06-03T01:34:00.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.995+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb004f2e0 con 0x7f8bb0105bf0 2024-06-03T01:34:00.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.995+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8ba400f6f0 con 0x7f8bb0105bf0 2024-06-03T01:34:00.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.996+0000 7f8b97fff700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 0x7f8b980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:00.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.996+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8ba408f590 con 0x7f8bb0105bf0 2024-06-03T01:34:00.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.996+0000 7f8baeffd700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 0x7f8b980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:00.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:00.996+0000 7f8baeffd700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 0x7f8b980706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005950 tx=0x7f8ba000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:01.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.001+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8ba405c290 con 0x7f8bb0105bf0 2024-06-03T01:34:01.085 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:00 smithi032 ceph-mon[111919]: pgmap v301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:01.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:00 smithi044 ceph-mon[84214]: pgmap v301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.256+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f8bb0196330 con 0x7f8bb0105bf0 2024-06-03T01:34:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.257+0000 7f8b97fff700 1 -- 172.21.15.32:0/3615910803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+4276 (secure 0 0 0) 0x7f8bb0196330 con 0x7f8bb0105bf0 2024-06-03T01:34:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:01.257 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 msgr2=0x7f8b980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 0x7f8b980706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005950 tx=0x7f8ba000b410 comp rx=0 tx=0).stop 2024-06-03T01:34:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 msgr2=0x7f8bb0195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40092e0 tx=0x7f8ba400bd70 comp rx=0 tx=0).stop 2024-06-03T01:34:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 shutdown_connections 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8b9806e200 0x7f8b980706c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bb00ffb40 0x7f8bb01954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 --2- 172.21.15.32:0/3615910803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105bf0 0x7f8bb0195a00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 >> 172.21.15.32:0/3615910803 conn(0x7f8bb00fb650 msgr2=0x7f8bb00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 shutdown_connections 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:01.259+0000 7f8bb54d4700 1 -- 172.21.15.32:0/3615910803 wait complete. 2024-06-03T01:34:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2024-06-03T01:34:01.904 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:34:01.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 31 2024-06-03T01:34:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:02.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:01 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3615910803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-06-03T01:34:02.244 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:01 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3615910803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-06-03T01:34:02.971 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:02 smithi044 ceph-mon[84214]: pgmap v302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:03.276 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:02 smithi032 ceph-mon[111919]: pgmap v302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.813+0000 7f513a38b700 1 -- 172.21.15.32:0/2636285403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134101850 msgr2=0x7f51341058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.813+0000 7f513a38b700 1 --2- 172.21.15.32:0/2636285403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134101850 0x7f51341058a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5128009910 tx=0x7f5128009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 -- 172.21.15.32:0/2636285403 shutdown_connections 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 --2- 172.21.15.32:0/2636285403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134101850 0x7f51341058a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 --2- 172.21.15.32:0/2636285403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134100ea0 0x7f5134101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 -- 172.21.15.32:0/2636285403 >> 172.21.15.32:0/2636285403 conn(0x7f51340fc750 msgr2=0x7f51340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 -- 172.21.15.32:0/2636285403 shutdown_connections 2024-06-03T01:34:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 -- 172.21.15.32:0/2636285403 wait complete. 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.814+0000 7f513a38b700 1 Processor -- start 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f513a38b700 1 -- start start 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f513a38b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f513a38b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134101850 0x7f513419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f513a38b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513419d2d0 con 0x7f5134100ea0 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f513a38b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5134196780 con 0x7f5134101850 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50684/0 (socket says 172.21.15.32:50684) 2024-06-03T01:34:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 -- 172.21.15.32:0/3861704063 learned_addr learned my addr 172.21.15.32:0/3861704063 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f51337fe700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134101850 0x7f513419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 -- 172.21.15.32:0/3861704063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134101850 msgr2=0x7f513419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134101850 0x7f513419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.815+0000 7f5133fff700 1 -- 172.21.15.32:0/3861704063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280095f0 con 0x7f5134100ea0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f5133fff700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f512400c8a0 tx=0x7f512400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f512400cea0 con 0x7f5134100ea0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5124004830 con 0x7f5134100ea0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5134196a60 con 0x7f5134100ea0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51240055b0 con 0x7f5134100ea0 2024-06-03T01:34:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.816+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5134196fb0 con 0x7f5134100ea0 2024-06-03T01:34:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.817+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51341092a0 con 0x7f5134100ea0 2024-06-03T01:34:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.817+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5124007ad0 con 0x7f5134100ea0 2024-06-03T01:34:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.818+0000 7f51317fa700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 0x7f511c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.818+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f512408dee0 con 0x7f5134100ea0 2024-06-03T01:34:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.818+0000 7f51337fe700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 0x7f511c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.818+0000 7f51337fe700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 0x7f511c0706c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f51280059d0 tx=0x7f5128005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:03.823+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f512405ac60 con 0x7f5134100ea0 2024-06-03T01:34:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.073+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f5134197d60 con 0x7f5134100ea0 2024-06-03T01:34:04.074 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:03 smithi032 ceph-mon[111919]: pgmap v303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.074+0000 7f51317fa700 1 -- 172.21.15.32:0/3861704063 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5123 (secure 0 0 0) 0x7f512401d020 con 0x7f5134100ea0 2024-06-03T01:34:04.074 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:04.075 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:30:06.909705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 msgr2=0x7f511c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 0x7f511c0706c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f51280059d0 tx=0x7f5128005960 comp rx=0 tx=0).stop 2024-06-03T01:34:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 msgr2=0x7f513419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f512400c8a0 tx=0x7f512400cc60 comp rx=0 tx=0).stop 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 shutdown_connections 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f511c06e200 0x7f511c0706c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5134100ea0 0x7f513419c700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 --2- 172.21.15.32:0/3861704063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5134101850 0x7f513419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 >> 172.21.15.32:0/3861704063 conn(0x7f51340fc750 msgr2=0x7f51340fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 shutdown_connections 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:04.076+0000 7f513a38b700 1 -- 172.21.15.32:0/3861704063 wait complete. 2024-06-03T01:34:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-06-03T01:34:04.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:03 smithi044 ceph-mon[84214]: pgmap v303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:04.685 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-03T01:34:04.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 32 2024-06-03T01:34:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:05.026 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:04 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3861704063' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-06-03T01:34:05.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:04 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3861704063' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-06-03T01:34:06.170 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:05 smithi032 ceph-mon[111919]: pgmap v304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:06.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:05 smithi044 ceph-mon[84214]: pgmap v304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 -- 172.21.15.32:0/369856206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 msgr2=0x7f38e4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 --2- 172.21.15.32:0/369856206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e4073540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b0a0 tx=0x7f38d400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 -- 172.21.15.32:0/369856206 shutdown_connections 2024-06-03T01:34:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 --2- 172.21.15.32:0/369856206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 0x7f38e410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 --2- 172.21.15.32:0/369856206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e4073540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 -- 172.21.15.32:0/369856206 >> 172.21.15.32:0/369856206 conn(0x7f38e40fc740 msgr2=0x7f38e40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 -- 172.21.15.32:0/369856206 shutdown_connections 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.709+0000 7f38eac48700 1 -- 172.21.15.32:0/369856206 wait complete. 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 Processor -- start 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 -- start start 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 0x7f38e419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e419f460 con 0x7f38e4073160 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38eac48700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4198960 con 0x7f38e4073a80 2024-06-03T01:34:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38e89e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38e89e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50698/0 (socket says 172.21.15.32:50698) 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.710+0000 7f38e89e4700 1 -- 172.21.15.32:0/2457600400 learned_addr learned my addr 172.21.15.32:0/2457600400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e89e4700 1 -- 172.21.15.32:0/2457600400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 msgr2=0x7f38e419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e3fff700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 0x7f38e419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e89e4700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 0x7f38e419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e89e4700 1 -- 172.21.15.32:0/2457600400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4009d00 con 0x7f38e4073160 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e89e4700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b070 tx=0x7f38d4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d400e040 con 0x7f38e4073160 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d40096e0 con 0x7f38e4073160 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e4198be0 con 0x7f38e4073160 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d401d6b0 con 0x7f38e4073160 2024-06-03T01:34:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.711+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41990d0 con 0x7f38e4073160 2024-06-03T01:34:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.712+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e410a690 con 0x7f38e4073160 2024-06-03T01:34:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.713+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f38d4004590 con 0x7f38e4073160 2024-06-03T01:34:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.713+0000 7f38e1ffb700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 0x7f38d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.713+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f38d408f9f0 con 0x7f38e4073160 2024-06-03T01:34:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.713+0000 7f38e3fff700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 0x7f38d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.714+0000 7f38e3fff700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 0x7f38d00706c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005950 tx=0x7f38cc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.718+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f38d4091e60 con 0x7f38e4073160 2024-06-03T01:34:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.986+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f38e404f2e0 con 0x7f38e4073160 2024-06-03T01:34:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.987+0000 7f38e1ffb700 1 -- 172.21.15.32:0/2457600400 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+5122 (secure 0 0 0) 0x7f38d4061e60 con 0x7f38e4073160 2024-06-03T01:34:06.987 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:06.987 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:16.974545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 msgr2=0x7f38d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 0x7f38d00706c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005950 tx=0x7f38cc0058e0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 msgr2=0x7f38e419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b070 tx=0x7f38d4003ce0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 shutdown_connections 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f38d006e200 0x7f38d00706c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e4073160 0x7f38e419e890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 --2- 172.21.15.32:0/2457600400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38e4073a80 0x7f38e419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 >> 172.21.15.32:0/2457600400 conn(0x7f38e40fc740 msgr2=0x7f38e41077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 shutdown_connections 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:06.989+0000 7f38eac48700 1 -- 172.21.15.32:0/2457600400 wait complete. 2024-06-03T01:34:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2024-06-03T01:34:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:07 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/2457600400' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-06-03T01:34:07.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:07 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:34:07.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:07 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/2457600400' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-06-03T01:34:07.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:07 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:34:07.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 33 2024-06-03T01:34:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:08.157 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:08 smithi032 ceph-mon[111919]: pgmap v305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:08.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:08 smithi044 ceph-mon[84214]: pgmap v305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- 172.21.15.32:0/3622833361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 msgr2=0x7fe1f80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 --2- 172.21.15.32:0/3622833361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80689e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009910 tx=0x7fe1e8009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- 172.21.15.32:0/3622833361 shutdown_connections 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 --2- 172.21.15.32:0/3622833361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80689e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 --2- 172.21.15.32:0/3622833361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 0x7fe1f81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- 172.21.15.32:0/3622833361 >> 172.21.15.32:0/3622833361 conn(0x7fe1f8075670 msgr2=0x7fe1f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- 172.21.15.32:0/3622833361 shutdown_connections 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- 172.21.15.32:0/3622833361 wait complete. 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 Processor -- start 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.415+0000 7fe200384700 1 -- start start 2024-06-03T01:34:09.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe200384700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe200384700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 0x7fe1f8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe200384700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8100770 con 0x7fe1f8068580 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe200384700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81008b0 con 0x7fe1f8106cd0 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33410/0 (socket says 172.21.15.32:33410) 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 -- 172.21.15.32:0/1043912164 learned_addr learned my addr 172.21.15.32:0/1043912164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fd91f700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 0x7fe1f8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 -- 172.21.15.32:0/1043912164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 msgr2=0x7fe1f8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 0x7fe1f8100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 -- 172.21.15.32:0/1043912164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80095f0 con 0x7fe1f8068580 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.416+0000 7fe1fe120700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400d8d0 tx=0x7fe1f400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.417+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f40098a0 con 0x7fe1f8068580 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.417+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f81a2a30 con 0x7fe1f8068580 2024-06-03T01:34:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.417+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1f4010460 con 0x7fe1f8068580 2024-06-03T01:34:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.417+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f81a2e40 con 0x7fe1f8068580 2024-06-03T01:34:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.417+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f400f5f0 con 0x7fe1f8068580 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.418+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f81091d0 con 0x7fe1f8068580 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.419+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe1f4009a00 con 0x7fe1f8068580 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.419+0000 7fe1ef7fe700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 0x7fe1e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.419+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fe1f408e150 con 0x7fe1f8068580 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.419+0000 7fe1fd91f700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 0x7fe1e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.420+0000 7fe1fd91f700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 0x7fe1e40706c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1f81014e0 tx=0x7fe1e801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.428+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe1f4057320 con 0x7fe1f8068580 2024-06-03T01:34:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.685+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fe1f8108bb0 con 0x7fe1f8068580 2024-06-03T01:34:09.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.686+0000 7fe1ef7fe700 1 -- 172.21.15.32:0/1043912164 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+5213 (secure 0 0 0) 0x7fe1f405a9e0 con 0x7fe1f8068580 2024-06-03T01:34:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:09.686 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:17.977743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 msgr2=0x7fe1e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 0x7fe1e40706c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1f81014e0 tx=0x7fe1e801a040 comp rx=0 tx=0).stop 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 msgr2=0x7fe1f80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400d8d0 tx=0x7fe1f400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 shutdown_connections 2024-06-03T01:34:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fe1e406e200 0x7fe1e40706c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1f8068580 0x7fe1f80ffcf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 --2- 172.21.15.32:0/1043912164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f8106cd0 0x7fe1f8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 >> 172.21.15.32:0/1043912164 conn(0x7fe1f8075670 msgr2=0x7fe1f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 shutdown_connections 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:09.688+0000 7fe200384700 1 -- 172.21.15.32:0/1043912164 wait complete. 2024-06-03T01:34:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-06-03T01:34:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:10 smithi032 ceph-mon[111919]: pgmap v306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:10.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:10 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1043912164' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-06-03T01:34:10.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 34 2024-06-03T01:34:10.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:10 smithi044 ceph-mon[84214]: pgmap v306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:10.487 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:10 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1043912164' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-06-03T01:34:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:12.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 -- 172.21.15.32:0/4127081760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280fff50 msgr2=0x7fa528105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4127081760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280fff50 0x7fa528105840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009910 tx=0x7fa51c009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 -- 172.21.15.32:0/4127081760 shutdown_connections 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4127081760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280fff50 0x7fa528105840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4127081760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280ff5a0 0x7fa5280ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 -- 172.21.15.32:0/4127081760 >> 172.21.15.32:0/4127081760 conn(0x7fa528074e30 msgr2=0x7fa528075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 -- 172.21.15.32:0/4127081760 shutdown_connections 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.182+0000 7fa52de7b700 1 -- 172.21.15.32:0/4127081760 wait complete. 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 Processor -- start 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 -- start start 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280fff50 0x7fa528196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528197560 con 0x7fa5280ff5a0 2024-06-03T01:34:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa52de7b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52819b370 con 0x7fa5280fff50 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.183+0000 7fa5277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33426/0 (socket says 172.21.15.32:33426) 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 -- 172.21.15.32:0/4129140604 learned_addr learned my addr 172.21.15.32:0/4129140604 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa526ffd700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280fff50 0x7fa528196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 -- 172.21.15.32:0/4129140604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280fff50 msgr2=0x7fa528196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280fff50 0x7fa528196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 -- 172.21.15.32:0/4129140604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0095f0 con 0x7fa5280ff5a0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa5277fe700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa51800b4b0 tx=0x7fa51800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa518011800 con 0x7fa5280ff5a0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa518011e20 con 0x7fa5280ff5a0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52819b650 con 0x7fa5280ff5a0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51800f2e0 con 0x7fa5280ff5a0 2024-06-03T01:34:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.184+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52819bb70 con 0x7fa5280ff5a0 2024-06-03T01:34:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.185+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa528109200 con 0x7fa5280ff5a0 2024-06-03T01:34:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.186+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa518010380 con 0x7fa5280ff5a0 2024-06-03T01:34:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.186+0000 7fa524ff9700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 0x7fa5100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.186+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fa51808de60 con 0x7fa5280ff5a0 2024-06-03T01:34:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.187+0000 7fa526ffd700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 0x7fa5100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.187+0000 7fa526ffd700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 0x7fa5100706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa528197900 tx=0x7fa51c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:12.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.191+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa51805ab60 con 0x7fa5280ff5a0 2024-06-03T01:34:12.191 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:12 smithi032 ceph-mon[111919]: pgmap v307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.442+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fa52804f2e0 con 0x7fa5280ff5a0 2024-06-03T01:34:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.443+0000 7fa524ff9700 1 -- 172.21.15.32:0/4129140604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+5217 (secure 0 0 0) 0x7fa51805a6f0 con 0x7fa5280ff5a0 2024-06-03T01:34:12.443 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:12.443 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:26.923513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:resolve","state_seq":19,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:resolve","state_seq":15,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 msgr2=0x7fa5100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 0x7fa5100706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa528197900 tx=0x7fa51c01a040 comp rx=0 tx=0).stop 2024-06-03T01:34:12.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 msgr2=0x7fa528196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa51800b4b0 tx=0x7fa51800b870 comp rx=0 tx=0).stop 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 shutdown_connections 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fa51006e200 0x7fa5100706c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5280ff5a0 0x7fa528196990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 --2- 172.21.15.32:0/4129140604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5280fff50 0x7fa528196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 >> 172.21.15.32:0/4129140604 conn(0x7fa528074e30 msgr2=0x7fa5280fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 shutdown_connections 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:12.445+0000 7fa52de7b700 1 -- 172.21.15.32:0/4129140604 wait complete. 2024-06-03T01:34:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-06-03T01:34:12.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:12 smithi044 ceph-mon[84214]: pgmap v307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:13.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 35 2024-06-03T01:34:13.307 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:13.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:13 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/4129140604' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-06-03T01:34:13.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:13 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/4129140604' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-06-03T01:34:14.370 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:14 smithi032 ceph-mon[111919]: pgmap v308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:14.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:14 smithi044 ceph-mon[84214]: pgmap v308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 -- 172.21.15.32:0/945847443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 msgr2=0x7fbcb010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 --2- 172.21.15.32:0/945847443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb010abf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009910 tx=0x7fbca0009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 -- 172.21.15.32:0/945847443 shutdown_connections 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 --2- 172.21.15.32:0/945847443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb010abf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 --2- 172.21.15.32:0/945847443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 0x7fbcb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 -- 172.21.15.32:0/945847443 >> 172.21.15.32:0/945847443 conn(0x7fbcb00fb650 msgr2=0x7fbcb00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 -- 172.21.15.32:0/945847443 shutdown_connections 2024-06-03T01:34:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.906+0000 7fbcb771f700 1 -- 172.21.15.32:0/945847443 wait complete. 2024-06-03T01:34:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 Processor -- start 2024-06-03T01:34:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 -- start start 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 0x7fbcb019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb019d270 con 0x7fbcb0102700 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb771f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb0196720 con 0x7fbcb0101de0 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33452/0 (socket says 172.21.15.32:33452) 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 -- 172.21.15.32:0/1344595502 learned_addr learned my addr 172.21.15.32:0/1344595502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb54bb700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 0x7fbcb019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 -- 172.21.15.32:0/1344595502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 msgr2=0x7fbcb019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 0x7fbcb019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 -- 172.21.15.32:0/1344595502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca00095f0 con 0x7fbcb0102700 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.907+0000 7fbcb4cba700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbca00092e0 tx=0x7fbca000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.908+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca001d070 con 0x7fbcb0102700 2024-06-03T01:34:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.908+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbca0005dc0 con 0x7fbcb0102700 2024-06-03T01:34:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.908+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb01969a0 con 0x7fbcb0102700 2024-06-03T01:34:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.908+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca0017400 con 0x7fbcb0102700 2024-06-03T01:34:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.908+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb0196e90 con 0x7fbcb0102700 2024-06-03T01:34:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.909+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb01082f0 con 0x7fbcb0102700 2024-06-03T01:34:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.909+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbca000f6f0 con 0x7fbcb0102700 2024-06-03T01:34:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.910+0000 7fbca67fc700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 0x7fbc9c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.910+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fbca008f680 con 0x7fbcb0102700 2024-06-03T01:34:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.910+0000 7fbcb54bb700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 0x7fbc9c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.910+0000 7fbcb54bb700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 0x7fbc9c0706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009af0 tx=0x7fbcac0094f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:14.914+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbca005c380 con 0x7fbcb0102700 2024-06-03T01:34:15.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.167+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fbcb004f2e0 con 0x7fbcb0102700 2024-06-03T01:34:15.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.167+0000 7fbca67fc700 1 -- 172.21.15.32:0/1344595502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5221 (secure 0 0 0) 0x7fbca005bf10 con 0x7fbcb0102700 2024-06-03T01:34:15.167 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:15.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:26.976237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:15.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.169+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 msgr2=0x7fbc9c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.169+0000 7fbcb771f700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 0x7fbc9c0706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009af0 tx=0x7fbcac0094f0 comp rx=0 tx=0).stop 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.169+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 msgr2=0x7fbcb019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.169+0000 7fbcb771f700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbca00092e0 tx=0x7fbca000be20 comp rx=0 tx=0).stop 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 shutdown_connections 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fbc9c06e200 0x7fbc9c0706c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcb0101de0 0x7fbcb019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 --2- 172.21.15.32:0/1344595502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb0102700 0x7fbcb019cbe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 >> 172.21.15.32:0/1344595502 conn(0x7fbcb00fb650 msgr2=0x7fbcb0105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 shutdown_connections 2024-06-03T01:34:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:15.170+0000 7fbcb771f700 1 -- 172.21.15.32:0/1344595502 wait complete. 2024-06-03T01:34:15.171 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-06-03T01:34:15.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 36 2024-06-03T01:34:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:16.373 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:16 smithi032 ceph-mon[111919]: pgmap v309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:16.373 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:16 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/1344595502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-06-03T01:34:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:16 smithi044 ceph-mon[84214]: pgmap v309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:16.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:16 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/1344595502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-06-03T01:34:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3311408036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40fff50 msgr2=0x7f8ff4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3311408036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40fff50 0x7f8ff4105840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009910 tx=0x7f8fdc009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3311408036 shutdown_connections 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3311408036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40fff50 0x7f8ff4105840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3311408036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3311408036 >> 172.21.15.32:0/3311408036 conn(0x7f8ff4074e30 msgr2=0x7f8ff4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3311408036 shutdown_connections 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.672+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3311408036 wait complete. 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 Processor -- start 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 -- start start 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fff50 0x7f8ff419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff419d210 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.673+0000 7f8ffa06d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff41966c0 con 0x7f8ff40fff50 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37046/0 (socket says 172.21.15.32:37046) 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 -- 172.21.15.32:0/3065254282 learned_addr learned my addr 172.21.15.32:0/3065254282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8feaffd700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fff50 0x7f8ff419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 -- 172.21.15.32:0/3065254282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fff50 msgr2=0x7f8ff419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fff50 0x7f8ff419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 -- 172.21.15.32:0/3065254282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0095f0 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff37fe700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d8d0 tx=0x7f8fe400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe40098a0 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fe4010460 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff41969a0 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe400f5f0 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.674+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff4196ec0 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.675+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4109200 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.676+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8fe4009a00 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.677+0000 7f8ff17fa700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 0x7f8fe00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.677+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8fe408e080 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.677+0000 7f8feaffd700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 0x7f8fe00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.677+0000 7f8feaffd700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 0x7f8fe00706c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:17.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.681+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8fe405ae00 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.934+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f8ff4197130 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.935+0000 7f8ff17fa700 1 -- 172.21.15.32:0/3065254282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f8fe405a990 con 0x7f8ff40ff5a0 2024-06-03T01:34:17.935 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:17.935 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:27.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-06-03T01:34:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.937+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 msgr2=0x7f8fe00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.937+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 0x7f8fe00706c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc01a040 comp rx=0 tx=0).stop 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.937+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 msgr2=0x7f8ff419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.937+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d8d0 tx=0x7f8fe400dbe0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 shutdown_connections 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8fe006e200 0x7f8fe00706c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff40ff5a0 0x7f8ff419c640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 --2- 172.21.15.32:0/3065254282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff40fff50 0x7f8ff419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 >> 172.21.15.32:0/3065254282 conn(0x7f8ff4074e30 msgr2=0x7f8ff40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 shutdown_connections 2024-06-03T01:34:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:17.938+0000 7f8ffa06d700 1 -- 172.21.15.32:0/3065254282 wait complete. 2024-06-03T01:34:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-06-03T01:34:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:18 smithi044 ceph-mon[84214]: pgmap v310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:18.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:18 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3065254282' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-06-03T01:34:18.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:18 smithi032 ceph-mon[111919]: pgmap v310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:18.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:18 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3065254282' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-06-03T01:34:18.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 37 2024-06-03T01:34:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:20.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:20 smithi032 ceph-mon[111919]: pgmap v311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/2679172630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0101850 msgr2=0x7f8ea01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/2679172630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0101850 0x7f8ea01058a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009910 tx=0x7f8e90009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/2679172630 shutdown_connections 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/2679172630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0101850 0x7f8ea01058a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/2679172630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/2679172630 >> 172.21.15.32:0/2679172630 conn(0x7f8ea00fc750 msgr2=0x7f8ea00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/2679172630 shutdown_connections 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.440+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/2679172630 wait complete. 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 Processor -- start 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 -- start start 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 0x7f8ea019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea019d250 con 0x7f8ea0100ea0 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea7ce8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0196700 con 0x7f8ea0101850 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea5a84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea5a84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37068/0 (socket says 172.21.15.32:37068) 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.441+0000 7f8ea5a84700 1 -- 172.21.15.32:0/4237381903 learned_addr learned my addr 172.21.15.32:0/4237381903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5283700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 0x7f8ea019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5a84700 1 -- 172.21.15.32:0/4237381903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 msgr2=0x7f8ea019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5a84700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 0x7f8ea019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5a84700 1 -- 172.21.15.32:0/4237381903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e900095f0 con 0x7f8ea0100ea0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5283700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 0x7f8ea019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea5a84700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00d900 tx=0x7f8e9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c004a00 con 0x7f8ea0100ea0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e9c0054e0 con 0x7f8ea0100ea0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea01969e0 con 0x7f8ea0100ea0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c009d60 con 0x7f8ea0100ea0 2024-06-03T01:34:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.442+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea0196f30 con 0x7f8ea0100ea0 2024-06-03T01:34:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.443+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea01091b0 con 0x7f8ea0100ea0 2024-06-03T01:34:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.444+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8e9c005020 con 0x7f8ea0100ea0 2024-06-03T01:34:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.444+0000 7f8e96ffd700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 0x7f8e8c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.444+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8e9c08e550 con 0x7f8ea0100ea0 2024-06-03T01:34:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.444+0000 7f8ea5283700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 0x7f8e8c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.445+0000 7f8ea5283700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 0x7f8e8c0706c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0197b70 tx=0x7f8e9001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:20.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.449+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e9c05b1b0 con 0x7f8ea0100ea0 2024-06-03T01:34:20.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:20 smithi044 ceph-mon[84214]: pgmap v311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.703+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f8ea0197d10 con 0x7f8ea0100ea0 2024-06-03T01:34:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.703+0000 7f8e96ffd700 1 -- 172.21.15.32:0/4237381903 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5230 (secure 0 0 0) 0x7f8ea0197d10 con 0x7f8ea0100ea0 2024-06-03T01:34:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-06-03T01:34:20.704 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi044.gkcyfr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/2797390272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2797390272},{"type":"v1","addr":"172.21.15.44:6825","nonce":2797390272}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi044.mjxnhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/4018885964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":4018885964},{"type":"v1","addr":"172.21.15.44:6827","nonce":4018885964}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:18:11.768168+0000","modified":"2024-06-03T01:31:29.943482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":44287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi032.itqzrm","rank":0,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.32:6829/1487470155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1487470155},{"type":"v1","addr":"172.21.15.32:6829","nonce":1487470155}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44287":{"gid":44287,"name":"cephfs.smithi032.tegslf","rank":1,"incarnation":33,"state":"up:active","state_seq":18,"addr":"172.21.15.32:6827/4048608139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4048608139},{"type":"v1","addr":"172.21.15.32:6827","nonce":4048608139}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34414,"qdb_cluster":[34414,44287]},"id":1}]} 2024-06-03T01:34:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 msgr2=0x7f8e8c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 0x7f8e8c0706c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0197b70 tx=0x7f8e9001a040 comp rx=0 tx=0).stop 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 msgr2=0x7f8ea019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00d900 tx=0x7f8e9c00dcc0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 shutdown_connections 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7f8e8c06e200 0x7f8e8c0706c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ea0100ea0 0x7f8ea019c680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 --2- 172.21.15.32:0/4237381903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea0101850 0x7f8ea019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.705+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 >> 172.21.15.32:0/4237381903 conn(0x7f8ea00fc750 msgr2=0x7f8ea0105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.706+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 shutdown_connections 2024-06-03T01:34:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:20.706+0000 7f8ea7ce8700 1 -- 172.21.15.32:0/4237381903 wait complete. 2024-06-03T01:34:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2024-06-03T01:34:21.344 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-03T01:34:21.355 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-03T01:34:21.355 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:34:21.355 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:34:21.376 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-06-03T01:34:21.376 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:34:21.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T01:34:21.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:21 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/4237381903' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-06-03T01:34:21.499 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:21 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/4237381903' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-06-03T01:34:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:22 smithi032 ceph-mon[111919]: pgmap v312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:22.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:22 smithi032 ceph-mon[111919]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:34:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:22 smithi044 ceph-mon[84214]: pgmap v312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:22.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:22 smithi044 ceph-mon[84214]: from='mgr.34102 172.21.15.32:0/1529876886' entity='mgr.smithi032.kfcfaa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-03T01:34:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3066348124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8106cd0 msgr2=0x7fd3a81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3066348124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8106cd0 0x7fd3a81070b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd398009910 tx=0x7fd398009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3066348124 shutdown_connections 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3066348124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8068580 0x7fd3a80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3066348124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8106cd0 0x7fd3a81070b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3066348124 >> 172.21.15.32:0/3066348124 conn(0x7fd3a8075670 msgr2=0x7fd3a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3066348124 shutdown_connections 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.313+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3066348124 wait complete. 2024-06-03T01:34:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 Processor -- start 2024-06-03T01:34:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 -- start start 2024-06-03T01:34:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8106cd0 0x7fd3a8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8100770 con 0x7fd3a8068580 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3ad6bd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81008b0 con 0x7fd3a8106cd0 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37072/0 (socket says 172.21.15.32:37072) 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3a6ffd700 1 -- 172.21.15.32:0/3662167871 learned_addr learned my addr 172.21.15.32:0/3662167871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.314+0000 7fd3a67fc700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8106cd0 0x7fd3a8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3a6ffd700 1 -- 172.21.15.32:0/3662167871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8106cd0 msgr2=0x7fd3a8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3a6ffd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8106cd0 0x7fd3a8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3a6ffd700 1 -- 172.21.15.32:0/3662167871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3980095f0 con 0x7fd3a8068580 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3a6ffd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3980092e0 tx=0x7fd39800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd39801d070 con 0x7fd3a8068580 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8100b30 con 0x7fd3a8068580 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd398005e00 con 0x7fd3a8068580 2024-06-03T01:34:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd398017400 con 0x7fd3a8068580 2024-06-03T01:34:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.315+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a81a2d90 con 0x7fd3a8068580 2024-06-03T01:34:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.316+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a81091f0 con 0x7fd3a8068580 2024-06-03T01:34:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.317+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd39800f460 con 0x7fd3a8068580 2024-06-03T01:34:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.317+0000 7fd38ffff700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 0x7fd3900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.317+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fd39808f7c0 con 0x7fd3a8068580 2024-06-03T01:34:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.317+0000 7fd3a67fc700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 0x7fd3900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.318+0000 7fd3a67fc700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 0x7fd3900706c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005950 tx=0x7fd39c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:23.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.322+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd398058990 con 0x7fd3a8068580 2024-06-03T01:34:23.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.545+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd3a804f2e0 con 0x7fd3a8068580 2024-06-03T01:34:23.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.545+0000 7fd38ffff700 1 -- 172.21.15.32:0/3662167871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v96) v1 ==== 81+0+2426 (secure 0 0 0) 0x7fd39805c050 con 0x7fd3a8068580 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6828/3721991653 2024-06-04T01:30:06.909697+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6826/1879817357 2024-06-04T01:30:06.909675+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2195293934 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2322410533 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1166022518 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3711300187 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4112439105 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1129696663 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3841470814 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6827/1879817357 2024-06-04T01:30:06.909675+0000 2024-06-03T01:34:23.546 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1891444588 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3680901021 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3836531275 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/84030829 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1080069639 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3147998267 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6828/2860560309 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2872286231 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6824/372165216 2024-06-04T01:18:29.688792+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2139490011 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2914664298 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/4275841113 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6825/372165216 2024-06-04T01:18:29.688792+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3674210877 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3147998267 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/458796220 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/458796220 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1578498944 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2007986688 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1578498944 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2112372769 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2678186916 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6829/3721991653 2024-06-04T01:30:06.909697+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2678186916 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6829/2860560309 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1695685766 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3335786446 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/1927345634 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:23.548 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1434819821 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:23.549 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2112372769 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.549 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1172809786 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 msgr2=0x7fd3900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 0x7fd3900706c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005950 tx=0x7fd39c0058e0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 msgr2=0x7fd3a80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3980092e0 tx=0x7fd39800be20 comp rx=0 tx=0).stop 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 shutdown_connections 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fd39006e200 0x7fd3900706c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3a8068580 0x7fd3a80ffcf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 --2- 172.21.15.32:0/3662167871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a8106cd0 0x7fd3a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 >> 172.21.15.32:0/3662167871 conn(0x7fd3a8075670 msgr2=0x7fd3a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 shutdown_connections 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:23.548+0000 7fd3ad6bd700 1 -- 172.21.15.32:0/3662167871 wait complete. 2024-06-03T01:34:23.550 INFO:teuthology.orchestra.run.smithi032.stderr:listed 41 entries 2024-06-03T01:34:24.182 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-03T01:34:24.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:24.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-03T01:34:24.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:34:24.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:24.253 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-03T01:34:24.327 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:34:24.327 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:34:24.348 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-06-03T01:34:24.348 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:34:24.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T01:34:24.432 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:24 smithi032 ceph-mon[111919]: pgmap v313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:24.432 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:24 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3662167871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:34:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:24 smithi044 ceph-mon[84214]: pgmap v313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-03T01:34:24.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:24 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3662167871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:34:24.623 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/mon.smithi032/config 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 -- 172.21.15.32:0/157387388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 msgr2=0x7fb86c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 --2- 172.21.15.32:0/157387388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c101280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb868009910 tx=0x7fb868009c20 comp rx=0 tx=0).stop 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 -- 172.21.15.32:0/157387388 shutdown_connections 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 --2- 172.21.15.32:0/157387388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 --2- 172.21.15.32:0/157387388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c101280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 -- 172.21.15.32:0/157387388 >> 172.21.15.32:0/157387388 conn(0x7fb86c0fc750 msgr2=0x7fb86c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 -- 172.21.15.32:0/157387388 shutdown_connections 2024-06-03T01:34:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.213+0000 7fb873d61700 1 -- 172.21.15.32:0/157387388 wait complete. 2024-06-03T01:34:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 Processor -- start 2024-06-03T01:34:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 -- start start 2024-06-03T01:34:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c197560 con 0x7fb86c100ea0 2024-06-03T01:34:26.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb873d61700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c19b370 con 0x7fb86c101850 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb871afd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb871afd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37080/0 (socket says 172.21.15.32:37080) 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.214+0000 7fb871afd700 1 -- 172.21.15.32:0/3142029477 learned_addr learned my addr 172.21.15.32:0/3142029477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb8712fc700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb871afd700 1 -- 172.21.15.32:0/3142029477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 msgr2=0x7fb86c196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb871afd700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb871afd700 1 -- 172.21.15.32:0/3142029477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680095f0 con 0x7fb86c100ea0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb8712fc700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb871afd700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb868006010 tx=0x7fb86800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86801d070 con 0x7fb86c100ea0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb868004fb0 con 0x7fb86c100ea0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c19b5f0 con 0x7fb86c100ea0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb868022ea0 con 0x7fb86c100ea0 2024-06-03T01:34:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.215+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c19bae0 con 0x7fb86c100ea0 2024-06-03T01:34:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.216+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c109240 con 0x7fb86c100ea0 2024-06-03T01:34:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.217+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb868022470 con 0x7fb86c100ea0 2024-06-03T01:34:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.217+0000 7fb85effd700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 0x7fb8580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:34:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.217+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fb86808f8c0 con 0x7fb86c100ea0 2024-06-03T01:34:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.217+0000 7fb8712fc700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 0x7fb8580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:34:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.218+0000 7fb8712fc700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 0x7fb8580706c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb86c197fb0 tx=0x7fb860008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:34:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.222+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb86805c5d0 con 0x7fb86c100ea0 2024-06-03T01:34:26.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:26 smithi032 ceph-mon[111919]: pgmap v314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:34:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.447+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb86c197dc0 con 0x7fb86c100ea0 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.448+0000 7fb85effd700 1 -- 172.21.15.32:0/3142029477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v96) v1 ==== 81+0+2426 (secure 0 0 0) 0x7fb868027990 con 0x7fb86c100ea0 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6828/3721991653 2024-06-04T01:30:06.909697+0000 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6826/1879817357 2024-06-04T01:30:06.909675+0000 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2195293934 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2322410533 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1166022518 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3711300187 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4112439105 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1129696663 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3841470814 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6827/1879817357 2024-06-04T01:30:06.909675+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1891444588 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3680901021 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/3836531275 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/84030829 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1080069639 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3147998267 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:26.449 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6828/2860560309 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2872286231 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6824/372165216 2024-06-04T01:18:29.688792+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2139490011 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2914664298 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/4275841113 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6825/372165216 2024-06-04T01:18:29.688792+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3674210877 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3147998267 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:26.450 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/458796220 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/458796220 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1578498944 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2007986688 2024-06-04T01:12:14.539594+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1578498944 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2112372769 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2678186916 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6829/3721991653 2024-06-04T01:30:06.909697+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2678186916 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:6829/2860560309 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1695685766 2024-06-04T01:13:28.845604+0000 2024-06-03T01:34:26.451 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3335786446 2024-06-04T01:20:55.768514+0000 2024-06-03T01:34:26.452 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.44:0/1927345634 2024-06-04T01:22:15.016430+0000 2024-06-03T01:34:26.452 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1434819821 2024-06-04T01:11:41.610662+0000 2024-06-03T01:34:26.452 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2112372769 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.452 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1172809786 2024-06-04T01:25:36.924136+0000 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 msgr2=0x7fb8580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 0x7fb8580706c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb86c197fb0 tx=0x7fb860008040 comp rx=0 tx=0).stop 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 msgr2=0x7fb86c196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb868006010 tx=0x7fb86800be20 comp rx=0 tx=0).stop 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 shutdown_connections 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:6800/2336714546,v1:172.21.15.32:6801/2336714546] conn(0x7fb85806e200 0x7fb8580706c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86c100ea0 0x7fb86c196990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 --2- 172.21.15.32:0/3142029477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb86c101850 0x7fb86c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 >> 172.21.15.32:0/3142029477 conn(0x7fb86c0fc750 msgr2=0x7fb86c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 shutdown_connections 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-06-03T01:34:26.450+0000 7fb873d61700 1 -- 172.21.15.32:0/3142029477 wait complete. 2024-06-03T01:34:26.453 INFO:teuthology.orchestra.run.smithi032.stderr:listed 41 entries 2024-06-03T01:34:26.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:26 smithi044 ceph-mon[84214]: pgmap v314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:34:27.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-06-03T01:34:27.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-06-03T01:34:27.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-06-03T01:34:27.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.138 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-06-03T01:34:27.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.209 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-06-03T01:34:27.228 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-03T01:34:27.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.228 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T01:34:27.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-06-03T01:34:27.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:27 smithi032 ceph-mon[111919]: from='client.? 172.21.15.32:0/3142029477' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:34:27.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.397 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-06-03T01:34:27.417 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-06-03T01:34:27.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-06-03T01:34:27.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:27 smithi044 ceph-mon[84214]: from='client.? 172.21.15.32:0/3142029477' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:34:27.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:34:27.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-06-03T01:34:27.592 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-03T01:34:27.603 INFO:tasks.cephadm:Teardown begin 2024-06-03T01:34:27.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:34:27.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:34:27.661 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-03T01:34:27.661 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:34:27.690 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:34:27.717 INFO:tasks.cephadm:Stopping all daemons... 2024-06-03T01:34:27.717 INFO:tasks.cephadm.mon.smithi032:Stopping mon.smithi032... 2024-06-03T01:34:27.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032 2024-06-03T01:34:28.059 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:27 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[111915]: 2024-06-03T01:34:28.074+0000 7fd357a77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032[111915]: 2024-06-03T01:34:28.074+0000 7fd357a77700 -1 mon.smithi032@0(leader) e3 *** Got Signal Terminated *** 2024-06-03T01:34:28.335 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 podman[159768]: 2024-06-03 01:34:28.207887511 +0000 UTC m=+0.265750520 container died d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T01:34:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 podman[159768]: 2024-06-03 01:34:28.633185167 +0000 UTC m=+0.691048165 container cleanup d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T01:34:28.835 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 bash[159768]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032 2024-06-03T01:34:29.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service' 2024-06-03T01:34:29.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:28 smithi032 podman[159790]: 2024-06-03 01:34:28.876666178 +0000 UTC m=+0.668251856 container remove d0b6e606b1205b58a24ec4612281ad2d84dd9424d43c382f0dd154eeb3acddf4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi032, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:34:29.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:29 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Deactivated successfully. 2024-06-03T01:34:29.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:29 smithi032 systemd[1]: Stopped Ceph mon.smithi032 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:34:29.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:Jun 03 01:34:29 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi032.service: Consumed 9.103s CPU time. 2024-06-03T01:34:29.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:34:29.169 INFO:tasks.cephadm.mon.smithi032:Stopped mon.smithi032 2024-06-03T01:34:29.169 INFO:tasks.cephadm.mon.smithi044:Stopping mon.smithi044... 2024-06-03T01:34:29.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi044 2024-06-03T01:34:29.471 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:29 smithi044 systemd[1]: Stopping Ceph mon.smithi044 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:29.863 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:29 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044[84210]: 2024-06-03T01:34:29.503+0000 7fb866d17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi044 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:29.863 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:29 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044[84210]: 2024-06-03T01:34:29.503+0000 7fb866d17700 -1 mon.smithi044@1(peon) e3 *** Got Signal Terminated *** 2024-06-03T01:34:29.863 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:29 smithi044 podman[120520]: 2024-06-03 01:34:29.556081096 +0000 UTC m=+0.163282523 container died 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T01:34:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:30 smithi044 podman[120520]: 2024-06-03 01:34:30.030072131 +0000 UTC m=+0.637273549 container cleanup 5c5eb244c04e8c1763de332ae32d48e7d67ca11d46636b9e3dcb0abf9681d1f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:34:30.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Jun 03 01:34:30 smithi044 bash[120520]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-mon-smithi044 2024-06-03T01:34:30.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@mon.smithi044.service' 2024-06-03T01:34:30.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:34:30.676 INFO:tasks.cephadm.mon.smithi044:Stopped mon.smithi044 2024-06-03T01:34:30.676 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-03T01:34:30.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.0 2024-06-03T01:34:30.997 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:30 smithi032 systemd[1]: Stopping Ceph osd.0 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:31.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:30 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:34:30.996+0000 7f0940112700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:31.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:30 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:34:30.996+0000 7f0940112700 -1 osd.0 96 *** Got signal Terminated *** 2024-06-03T01:34:31.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:30 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0[123235]: 2024-06-03T01:34:30.996+0000 7f0940112700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:34:36.674 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:36 smithi032 podman[159901]: 2024-06-03 01:34:36.398331309 +0000 UTC m=+5.516743768 container died d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T01:34:37.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:36 smithi032 podman[159901]: 2024-06-03 01:34:36.76431494 +0000 UTC m=+5.882727388 container cleanup d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-03T01:34:37.085 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:36 smithi032 bash[159901]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0 2024-06-03T01:34:37.556 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[159918]: 2024-06-03 01:34:37.224424239 +0000 UTC m=+0.818446029 container remove d2cd0ee1b637e86a41bc9ddd8e548948dbe47f8af00b8de6baf859b4156af31b (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:34:37.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[160005]: 2024-06-03 01:34:37.459944692 +0000 UTC m=+0.018433532 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:34:37.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[160005]: 2024-06-03 01:34:37.575563095 +0000 UTC m=+0.134051909 container create 6cd87521590a5b4d4f0f9f1ad6b15523911742417e8b28da9e5616a955a4645a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-03T01:34:37.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[160005]: 2024-06-03 01:34:37.786455455 +0000 UTC m=+0.344944268 container init 6cd87521590a5b4d4f0f9f1ad6b15523911742417e8b28da9e5616a955a4645a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T01:34:37.835 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[160005]: 2024-06-03 01:34:37.790026091 +0000 UTC m=+0.348514907 container start 6cd87521590a5b4d4f0f9f1ad6b15523911742417e8b28da9e5616a955a4645a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T01:34:38.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:37 smithi032 podman[160005]: 2024-06-03 01:34:37.886758839 +0000 UTC m=+0.445247668 container attach 6cd87521590a5b4d4f0f9f1ad6b15523911742417e8b28da9e5616a955a4645a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:34:38.335 INFO:journalctl@ceph.osd.0.smithi032.stdout:Jun 03 01:34:38 smithi032 podman[160031]: 2024-06-03 01:34:38.12552086 +0000 UTC m=+0.018542746 container died 6cd87521590a5b4d4f0f9f1ad6b15523911742417e8b28da9e5616a955a4645a (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-03T01:34:38.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-03T01:34:38.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:34:38.702 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-03T01:34:38.702 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-03T01:34:38.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1 2024-06-03T01:34:39.060 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:38 smithi032 systemd[1]: Stopping Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:39.335 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:39 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:34:39.060+0000 7f59d2d15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:39.335 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:39 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:34:39.060+0000 7f59d2d15700 -1 osd.1 96 *** Got signal Terminated *** 2024-06-03T01:34:39.335 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:39 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1[128910]: 2024-06-03T01:34:39.060+0000 7f59d2d15700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:34:44.912 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:44 smithi032 podman[160110]: 2024-06-03 01:34:44.604057509 +0000 UTC m=+5.662965381 container died 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-03T01:34:45.212 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160110]: 2024-06-03 01:34:45.002647349 +0000 UTC m=+6.061555216 container cleanup 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-03T01:34:45.212 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 bash[160110]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1 2024-06-03T01:34:45.585 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160139]: 2024-06-03 01:34:45.21304377 +0000 UTC m=+0.598642120 container remove 35e7b6d66100a9ab152bb97825c9bed6ef31f28ffc95c88f60169daad12f2dfd (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T01:34:45.949 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160226]: 2024-06-03 01:34:45.551502488 +0000 UTC m=+0.017420644 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:34:45.949 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160226]: 2024-06-03 01:34:45.663958248 +0000 UTC m=+0.129876407 container create c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T01:34:45.949 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160226]: 2024-06-03 01:34:45.874693737 +0000 UTC m=+0.340611894 container init c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528) 2024-06-03T01:34:45.949 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160226]: 2024-06-03 01:34:45.879139642 +0000 UTC m=+0.345057808 container start c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-03T01:34:45.949 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:45 smithi032 podman[160226]: 2024-06-03 01:34:45.949607471 +0000 UTC m=+0.415525642 container attach c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-03T01:34:46.208 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:46 smithi032 podman[160226]: 2024-06-03 01:34:46.181091515 +0000 UTC m=+0.647009672 container died c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T01:34:46.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-03T01:34:46.746 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:46 smithi032 podman[160252]: 2024-06-03 01:34:46.6957724 +0000 UTC m=+0.505166126 container remove c1a6733159e531c17c62aaa27927e2fe89f47b4703080dc1137e3c0a5fb982e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:34:46.746 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:46 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service: Deactivated successfully. 2024-06-03T01:34:46.746 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:46 smithi032 systemd[1]: Stopped Ceph osd.1 for 12a144f0-2146-11ef-bc9b-c7b262605968. 2024-06-03T01:34:46.746 INFO:journalctl@ceph.osd.1.smithi032.stdout:Jun 03 01:34:46 smithi032 systemd[1]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.1.service: Consumed 12.269s CPU time. 2024-06-03T01:34:46.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:34:46.763 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-03T01:34:46.764 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-03T01:34:46.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.2 2024-06-03T01:34:47.085 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:46 smithi032 systemd[1]: Stopping Ceph osd.2 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:47.585 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:47 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:34:47.132+0000 7fb577021700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:47.585 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:47 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:34:47.132+0000 7fb577021700 -1 osd.2 96 *** Got signal Terminated *** 2024-06-03T01:34:47.585 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:47 smithi032 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2[134748]: 2024-06-03T01:34:47.132+0000 7fb577021700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:34:52.959 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:52 smithi032 podman[160332]: 2024-06-03 01:34:52.673683999 +0000 UTC m=+5.673894085 container died 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-03T01:34:53.300 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160332]: 2024-06-03 01:34:53.04940613 +0000 UTC m=+6.049616197 container cleanup 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T01:34:53.300 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 bash[160332]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2 2024-06-03T01:34:53.585 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160348]: 2024-06-03 01:34:53.300921589 +0000 UTC m=+0.623376624 container remove 0af7c9cdf9fa053480ae8e303fe98ac6bf76da46d04e6b3197b40fe45c1b80a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528) 2024-06-03T01:34:53.937 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160439]: 2024-06-03 01:34:53.551477601 +0000 UTC m=+0.018935682 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:34:53.937 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160439]: 2024-06-03 01:34:53.660307394 +0000 UTC m=+0.127765516 container create 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T01:34:53.937 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160439]: 2024-06-03 01:34:53.86319312 +0000 UTC m=+0.330651200 container init 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-03T01:34:53.937 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160439]: 2024-06-03 01:34:53.866617758 +0000 UTC m=+0.334075838 container start 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T01:34:53.937 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:53 smithi032 podman[160439]: 2024-06-03 01:34:53.938030215 +0000 UTC m=+0.405488299 container attach 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:34:54.335 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:54 smithi032 podman[160439]: 2024-06-03 01:34:54.17328678 +0000 UTC m=+0.640744861 container died 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:34:54.729 INFO:journalctl@ceph.osd.2.smithi032.stdout:Jun 03 01:34:54 smithi032 podman[160465]: 2024-06-03 01:34:54.716797513 +0000 UTC m=+0.533851041 container remove 64825c57cc62d516ec2c51ae6df6b79ad5cb0702a3a56fc668af10dacca2cdee (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-03T01:34:54.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-03T01:34:54.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:34:54.776 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-03T01:34:54.776 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-03T01:34:54.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.3 2024-06-03T01:34:55.101 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:34:54 smithi044 systemd[1]: Stopping Ceph osd.3 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:34:55.471 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:34:55 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:34:55.101+0000 7f3ba16e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:34:55.471 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:34:55 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:34:55.101+0000 7f3ba16e3700 -1 osd.3 96 *** Got signal Terminated *** 2024-06-03T01:34:55.471 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:34:55 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:34:55.101+0000 7f3ba16e3700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:34:57.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:34:57 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:34:57.515+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:34:58.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:34:58 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:34:58.524+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:34:59.471 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:34:59 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3[93882]: 2024-06-03T01:34:59.186+0000 7f3b9a4d0700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:35.022928+0000 front 2024-06-03T01:34:35.022874+0000 (oldest deadline 2024-06-03T01:34:59.122730+0000) 2024-06-03T01:34:59.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:34:59 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:34:59.525+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:00.972 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:00 smithi044 podman[120652]: 2024-06-03 01:35:00.660290651 +0000 UTC m=+5.679147325 container died 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T01:35:00.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:00 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:00.572+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:01.344 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:01 smithi044 podman[120652]: 2024-06-03 01:35:01.210167093 +0000 UTC m=+6.229023757 container cleanup 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-03T01:35:01.344 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:01 smithi044 bash[120652]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3 2024-06-03T01:35:01.721 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:01 smithi044 podman[120669]: 2024-06-03 01:35:01.344981882 +0000 UTC m=+0.679637570 container remove 8c54ffef38b6d156981b07ccd3c4d7238663e7b61369e3514c19a6b7cf0d5cba (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-03T01:35:01.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:01 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:01.575+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:02.039 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:01 smithi044 podman[120758]: 2024-06-03 01:35:01.676581213 +0000 UTC m=+0.018078590 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:35:02.040 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:01 smithi044 podman[120758]: 2024-06-03 01:35:01.78771768 +0000 UTC m=+0.129215044 container create 8b6a4fd649337614b17d1d1b743fe9a185d1a3ea7b6ad1389f4bad14460243be (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T01:35:02.040 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:02 smithi044 podman[120758]: 2024-06-03 01:35:02.040564673 +0000 UTC m=+0.382062050 container init 8b6a4fd649337614b17d1d1b743fe9a185d1a3ea7b6ad1389f4bad14460243be (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T01:35:02.354 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:02 smithi044 podman[120758]: 2024-06-03 01:35:02.044528722 +0000 UTC m=+0.386026092 container start 8b6a4fd649337614b17d1d1b743fe9a185d1a3ea7b6ad1389f4bad14460243be (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T01:35:02.354 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:02 smithi044 podman[120758]: 2024-06-03 01:35:02.14890089 +0000 UTC m=+0.490398254 container attach 8b6a4fd649337614b17d1d1b743fe9a185d1a3ea7b6ad1389f4bad14460243be (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:35:02.644 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:02 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:02.541+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:02.644 INFO:journalctl@ceph.osd.3.smithi044.stdout:Jun 03 01:35:02 smithi044 podman[120758]: 2024-06-03 01:35:02.355098039 +0000 UTC m=+0.696595412 container died 8b6a4fd649337614b17d1d1b743fe9a185d1a3ea7b6ad1389f4bad14460243be (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-03T01:35:02.936 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-03T01:35:02.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:35:02.981 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-03T01:35:02.981 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-03T01:35:02.981 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.4 2024-06-03T01:35:03.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:03 smithi044 systemd[1]: Stopping Ceph osd.4 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:35:03.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:03.155+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:03.559 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:03.331+0000 7f2ddf80c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:35:03.560 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:03.331+0000 7f2ddf80c700 -1 osd.4 96 *** Got signal Terminated *** 2024-06-03T01:35:03.560 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:03.331+0000 7f2ddf80c700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:35:03.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:03 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:03.560+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:04.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:04 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:04.192+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:04.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:04 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:04.536+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:05.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:05.172+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:05.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:05.492+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:05.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:05 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:05.492+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:06.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:06 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:06.162+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:06.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:06 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:06.529+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:06.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:06 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:06.529+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:07.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:07 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:07.187+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:07.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:07 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:07.519+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:07.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:07 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:07.519+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:08.515 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:08 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:08.231+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.777438+0000 front 2024-06-03T01:34:32.777547+0000 (oldest deadline 2024-06-03T01:35:02.177842+0000) 2024-06-03T01:35:08.515 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:08 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4[98789]: 2024-06-03T01:35:08.231+0000 7f2ddc601700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:42.178332+0000 front 2024-06-03T01:34:42.178427+0000 (oldest deadline 2024-06-03T01:35:08.078257+0000) 2024-06-03T01:35:08.860 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:08 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:08.515+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:08.860 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:08 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:08.515+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:09.221 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:08 smithi044 podman[120864]: 2024-06-03 01:35:08.861304424 +0000 UTC m=+5.641039470 container died d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:35:09.513 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:09 smithi044 podman[120864]: 2024-06-03 01:35:09.256552672 +0000 UTC m=+6.036287701 container cleanup d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-06-03T01:35:09.513 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:09 smithi044 bash[120864]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4 2024-06-03T01:35:09.513 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:09 smithi044 podman[120893]: 2024-06-03 01:35:09.499861843 +0000 UTC m=+0.630600972 container remove d71ecbc3e1ba9cc506caa5f87bf717444335a4fbe63040f0d2b0b1df0e80afc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-03T01:35:09.841 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:09 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:09.513+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:09.841 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:09 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:09.513+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:10.119 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:09 smithi044 podman[120982]: 2024-06-03 01:35:09.842539657 +0000 UTC m=+0.104046502 container create dc47b9f2448a4ef66ba2db83a2af8096a82570118658b3b7b361fb6a16ed5c90 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-03T01:35:10.119 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:09 smithi044 podman[120982]: 2024-06-03 01:35:09.756301649 +0000 UTC m=+0.017808504 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:35:10.119 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:10 smithi044 podman[120982]: 2024-06-03 01:35:10.045115152 +0000 UTC m=+0.306621995 container init dc47b9f2448a4ef66ba2db83a2af8096a82570118658b3b7b361fb6a16ed5c90 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True) 2024-06-03T01:35:10.119 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:10 smithi044 podman[120982]: 2024-06-03 01:35:10.049113552 +0000 UTC m=+0.310620395 container start dc47b9f2448a4ef66ba2db83a2af8096a82570118658b3b7b361fb6a16ed5c90 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:35:10.120 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:10 smithi044 podman[120982]: 2024-06-03 01:35:10.120023306 +0000 UTC m=+0.381530150 container attach dc47b9f2448a4ef66ba2db83a2af8096a82570118658b3b7b361fb6a16ed5c90 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:35:10.471 INFO:journalctl@ceph.osd.4.smithi044.stdout:Jun 03 01:35:10 smithi044 podman[120982]: 2024-06-03 01:35:10.35069965 +0000 UTC m=+0.612206494 container died dc47b9f2448a4ef66ba2db83a2af8096a82570118658b3b7b361fb6a16ed5c90 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T01:35:10.948 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:10 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:10.543+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:10.948 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:10 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:10.543+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:10.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-03T01:35:11.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:35:11.023 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-03T01:35:11.024 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-03T01:35:11.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.5 2024-06-03T01:35:11.221 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 systemd[1]: Stopping Ceph osd.5 for 12a144f0-2146-11ef-bc9b-c7b262605968... 2024-06-03T01:35:11.721 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:11.351+0000 7efe988c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:35:11.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:11.352+0000 7efe988c4700 -1 osd.5 96 *** Got signal Terminated *** 2024-06-03T01:35:11.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:11.352+0000 7efe988c4700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:35:11.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:11.585+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:11.722 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:11 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:11.585+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:12.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:12 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:12.574+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:12.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:12 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:12.574+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:13.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:13 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:13.607+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:13.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:13 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:13.607+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:14.957 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:14 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:14.596+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:14.957 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:14 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:14.596+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:15.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:15 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:15.628+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-06-03T01:34:32.636934+0000 front 2024-06-03T01:34:32.636834+0000 (oldest deadline 2024-06-03T01:34:56.637165+0000) 2024-06-03T01:35:15.971 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:15 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:15.628+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-06-03T01:34:41.238531+0000 front 2024-06-03T01:34:41.238495+0000 (oldest deadline 2024-06-03T01:35:04.738411+0000) 2024-06-03T01:35:15.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:15 smithi044 ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5[103673]: 2024-06-03T01:35:15.628+0000 7efe916b1700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.32:6822 osd.2 since back 2024-06-03T01:34:50.539594+0000 front 2024-06-03T01:34:50.539538+0000 (oldest deadline 2024-06-03T01:35:14.639452+0000) 2024-06-03T01:35:17.154 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:16 smithi044 podman[121088]: 2024-06-03 01:35:16.857955111 +0000 UTC m=+5.602124207 container died a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T01:35:17.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:17 smithi044 podman[121088]: 2024-06-03 01:35:17.244173561 +0000 UTC m=+5.988342658 container cleanup a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T01:35:17.471 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:17 smithi044 bash[121088]: ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5 2024-06-03T01:35:17.821 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:17 smithi044 podman[121105]: 2024-06-03 01:35:17.478672183 +0000 UTC m=+0.618906859 container remove a11e0341a92226b54a43fc9f14098f716a3f4301acdde67132deb050af5ab2f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-03T01:35:18.126 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:17 smithi044 podman[121194]: 2024-06-03 01:35:17.821077122 +0000 UTC m=+0.113835869 container create 36761fed891500262c22d7924d01660e6979d91a4bc64a77e275c99f63be967f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T01:35:18.126 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:17 smithi044 podman[121194]: 2024-06-03 01:35:17.724725967 +0000 UTC m=+0.017484739 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:35:18.126 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:18 smithi044 podman[121194]: 2024-06-03 01:35:18.032364504 +0000 UTC m=+0.325123257 container init 36761fed891500262c22d7924d01660e6979d91a4bc64a77e275c99f63be967f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T01:35:18.126 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:18 smithi044 podman[121194]: 2024-06-03 01:35:18.035766246 +0000 UTC m=+0.328525000 container start 36761fed891500262c22d7924d01660e6979d91a4bc64a77e275c99f63be967f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-03T01:35:18.126 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:18 smithi044 podman[121194]: 2024-06-03 01:35:18.126548179 +0000 UTC m=+0.419306970 container attach 36761fed891500262c22d7924d01660e6979d91a4bc64a77e275c99f63be967f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-03T01:35:18.472 INFO:journalctl@ceph.osd.5.smithi044.stdout:Jun 03 01:35:18 smithi044 podman[121194]: 2024-06-03 01:35:18.342201046 +0000 UTC m=+0.634959813 container died 36761fed891500262c22d7924d01660e6979d91a4bc64a77e275c99f63be967f (image=quay.ceph.io/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-12a144f0-2146-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-06-03T01:35:18.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a144f0-2146-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-03T01:35:18.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:35:18.957 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-03T01:35:18.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T01:35:19.125 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:35:43.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T01:35:43.778 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:35:58.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:35:58.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:35:58.378 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-03T01:35:58.380 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi032/crash 2024-06-03T01:35:58.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T01:35:58.417 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T01:35:58.417 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2024-06-03T01:35:58.419 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi044/crash 2024-06-03T01:35:58.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T01:35:58.444 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T01:35:58.444 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-06-03T01:35:58.445 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-03T01:35:58.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-03T01:35:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:2024-06-03T01:30:00.000380+0000 mon.smithi032 (mon.0) 550 : cluster [WRN] osd.5 (root=default,host=smithi044) is down 2024-06-03T01:35:58.505 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-03T01:35:58.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[SEC\]' /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-03T01:35:58.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]' /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-03T01:35:58.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[WRN\]' /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-03T01:35:58.685 INFO:teuthology.orchestra.run.smithi032.stdout:2024-06-03T01:30:00.000380+0000 mon.smithi032 (mon.0) 550 : cluster [WRN] osd.5 (root=default,host=smithi044) is down 2024-06-03T01:35:58.686 INFO:tasks.cephadm:Compressing logs... 2024-06-03T01:35:58.686 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T01:35:58.729 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T01:35:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T01:35:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:: No such file or directory 2024-06-03T01:35:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.tegslf.log 2024-06-03T01:35:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-03T01:35:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.tegslf.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T01:35:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi032.log 2024-06-03T01:35:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-03T01:35:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi032.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-03T01:35:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi032.kfcfaa.log 2024-06-03T01:35:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.0.log: /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi032.kfcfaa.log: 87.9% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T01:35:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi032.log 2024-06-03T01:35:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T01:35:58.758 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.gkcyfr.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:: No such file or directory 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.gkcyfr.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi044.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi044.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi044.log: /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi044.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.mjxnhh.log 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.4.log: 87.9% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T01:35:58.759 INFO:teuthology.orchestra.run.smithi044.stderr: 94.1% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi044.log.gz 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi032.log: 94.1% 91.5% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi032.log.gz 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.itqzrm.log 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log: 85.9% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T01:35:58.764 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr: 91.6% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi044.wszjlw.log 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.mjxnhh.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi044.wszjlw.log: gzip -5 --verbose -- /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T01:35:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T01:35:58.766 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log: 94.0% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.mjxnhh.log.gz 2024-06-03T01:35:58.767 INFO:teuthology.orchestra.run.smithi044.stderr: 91.7% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi044.gkcyfr.log.gz 2024-06-03T01:35:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.itqzrm.log: 91.0% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T01:35:58.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T01:35:58.784 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T01:35:58.815 INFO:teuthology.orchestra.run.smithi044.stderr: 89.0% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi044.wszjlw.log.gz 2024-06-03T01:35:58.831 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cephadm.log: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T01:35:59.049 INFO:teuthology.orchestra.run.smithi044.stderr: 92.5% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi044.log.gz 2024-06-03T01:35:59.151 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mgr.smithi032.kfcfaa.log.gz 2024-06-03T01:35:59.937 INFO:teuthology.orchestra.run.smithi032.stderr: 90.6% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mon.smithi032.log.gz 2024-06-03T01:36:02.986 INFO:teuthology.orchestra.run.smithi044.stderr: 93.5% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-03T01:36:04.167 INFO:teuthology.orchestra.run.smithi032.stderr: 93.6% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-03T01:36:04.517 INFO:teuthology.orchestra.run.smithi032.stderr: 93.5% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-03T01:36:04.741 INFO:teuthology.orchestra.run.smithi044.stderr: 93.3% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-03T01:36:05.687 INFO:teuthology.orchestra.run.smithi044.stderr: 93.1% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-03T01:36:05.688 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-06-03T01:36:05.689 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m6.944s 2024-06-03T01:36:05.689 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m16.955s 2024-06-03T01:36:05.689 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.582s 2024-06-03T01:36:05.822 INFO:teuthology.orchestra.run.smithi032.stderr: 93.3% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-03T01:36:49.406 INFO:teuthology.orchestra.run.smithi032.stderr: 93.2% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.tegslf.log.gz 2024-06-03T01:36:50.912 INFO:teuthology.orchestra.run.smithi032.stderr: 93.1% -- replaced with /var/log/ceph/12a144f0-2146-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi032.itqzrm.log.gz 2024-06-03T01:36:50.914 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-06-03T01:36:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m52.172s 2024-06-03T01:36:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:user 1m59.028s 2024-06-03T01:36:50.914 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m3.431s 2024-06-03T01:36:50.915 INFO:tasks.cephadm:Archiving logs... 2024-06-03T01:36:50.915 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi032/log 2024-06-03T01:36:50.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T01:37:07.572 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi044/log 2024-06-03T01:37:07.574 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T01:37:10.200 INFO:tasks.cephadm:Removing cluster... 2024-06-03T01:37:10.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 --force 2024-06-03T01:37:10.363 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:37:10.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a144f0-2146-11ef-bc9b-c7b262605968 --force 2024-06-03T01:37:10.943 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 12a144f0-2146-11ef-bc9b-c7b262605968 2024-06-03T01:37:11.247 INFO:tasks.cephadm:Removing cephadm ... 2024-06-03T01:37:11.247 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T01:37:11.264 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T01:37:11.280 INFO:tasks.cephadm:Teardown complete 2024-06-03T01:37:11.281 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-03T01:37:11.293 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-03T01:37:11.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T01:37:11.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T01:37:11.397 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-03T01:37:11.407 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-03T01:37:11.408 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:37:11.410 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:37:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:37:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:37:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:37:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-06-03T01:37:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +409us[ +442us] +/- 65ms 2024-06-03T01:37:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -671us[ -671us] +/- 65ms 2024-06-03T01:37:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +307us[ +307us] +/- 70ms 2024-06-03T01:37:11.429 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +377us[ +387us] +/- 65ms 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -692us[ -682us] +/- 65ms 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 95 +318us[ +341us] +/- 69ms 2024-06-03T01:37:11.430 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:37:11.431 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-03T01:37:11.441 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-03T01:37:11.442 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-03T01:37:11.521 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T01:37:11.607 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T01:37:11.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:37:11.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:37:11.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:37:11.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:37:11.743 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-03T01:37:11.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-03T01:37:11.765 INFO:teuthology.task.internal:Duration was 1984.242940 seconds 2024-06-03T01:37:11.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-03T01:37:11.775 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-03T01:37:11.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T01:37:11.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T01:37:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:37:11.823 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:37:12.126 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-03T01:37:12.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-06-03T01:37:12.126 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T01:37:12.188 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-06-03T01:37:12.189 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T01:37:12.218 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-03T01:37:12.218 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T01:37:12.231 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T01:37:12.286 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-03T01:37:12.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T01:37:12.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T01:37:12.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-03T01:37:13.004 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-03T01:37:13.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T01:37:13.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T01:37:13.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-03T01:37:13.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:37:13.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:37:13.095 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-06-03T01:37:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-06-03T01:37:13.136 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:37:13.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:37:13.165 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:37:13.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:37:13.192 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-03T01:37:13.204 INFO:teuthology.task.internal:Transferring archived files... 2024-06-03T01:37:13.205 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi032 2024-06-03T01:37:13.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T01:37:13.258 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738764/remote/smithi044 2024-06-03T01:37:13.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T01:37:13.297 INFO:teuthology.task.internal:Removing archive directory... 2024-06-03T01:37:13.297 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T01:37:13.300 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T01:37:13.347 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-03T01:37:13.360 INFO:teuthology.task.internal:Not uploading archives. 2024-06-03T01:37:13.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-03T01:37:13.371 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-03T01:37:13.372 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T01:37:13.374 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T01:37:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 01:37 /home/ubuntu/cephtest 2024-06-03T01:37:13.403 INFO:teuthology.orchestra.run.smithi044.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 01:37 /home/ubuntu/cephtest 2024-06-03T01:37:13.404 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-03T01:37:13.416 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-03T01:37:13.486 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1984.2429399490356 failure_reason: '"2024-06-03T01:30:00.000380+0000 mon.smithi032 (mon.0) 550 : cluster [WRN] osd.5 (root=default,host=smithi044) is down" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-03T01:37:13.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T01:37:13.578 INFO:teuthology.run:FAIL